new library boost.fiber in vault

I've re factored boost.fiber (-> boost vault http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Concurrent%20Programming). The library provides a so called lightweight thread of execution (also known as user-space thread or fiber on WIN32). The API is modeled after boost.thread. Beside a scheduler (currently simple round-robin) for the fibers the library provides sync. primitives like mutex, condition- and event-variables (auto-reset, manual-reset, count-down). Message can be passed between fibers via bounded-/unbounded-fifo. Some feedback would be nice. regards, Oliver

Oliver Kowalke escribió:
I've re factored boost.fiber (-> boost vault http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Concurrent%20Programming).
regards, Oliver _______________________________________________
Could you include built documentation? Agustín K-ballo Bergé.-

2009/11/20 Agustín K-ballo Bergé <kaballo86@hotmail.com>:
Oliver Kowalke escribió:
I've re factored boost.fiber (-> boost vault http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Concurrent%20Programming).
regards, Oliver _______________________________________________
Could you include built documentation?
Yes, *please*. Not home right now, the computer I am on does not have any form of a compiler or anything, and reading QBK files in a text editor is "not fun". :)

OvermindDL1 wrote:
2009/11/20 Agustín K-ballo Bergé <kaballo86@hotmail.com>:
Oliver Kowalke escribió:
I've re factored boost.fiber (-> boost vault http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Concurrent%20Programming).
regards, Oliver _______________________________________________
Could you include built documentation?
Yes, *please*. Not home right now, the computer I am on does not have any form of a compiler or anything, and reading QBK files in a text editor is "not fun". :) _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost will be done soon

Sounds very similar to something I did a while back: http://www.mr-edd.co.uk/?page_id=91 It's rather late now, so I'll take a closer look tomorrow, perhaps. Kind regards, Edd Oliver Kowalke wrote:
I've re factored boost.fiber (-> boost vault http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Concurrent%20Programming).
The library provides a so called lightweight thread of execution (also known as user-space thread or fiber on WIN32). The API is modeled after boost.thread.
Beside a scheduler (currently simple round-robin) for the fibers the library provides sync. primitives like mutex, condition- and event-variables (auto-reset, manual-reset, count-down). Message can be passed between fibers via bounded-/unbounded-fifo.
Some feedback would be nice.
regards, Oliver

On Fri, Nov 20, 2009 at 9:39 PM, Edd Dawson <lists@mr-edd.co.uk> wrote:
Sounds very similar to something I did a while back: http://www.mr-edd.co.uk/?page_id=91
It's rather late now, so I'll take a closer look tomorrow, perhaps.
Yea, I see that Boost.Coroutine in the vault right under that Boost.Fiber. :) Different styles of handling different things. They might be able to be merged together?

OvermindDL1 wrote:
On Fri, Nov 20, 2009 at 9:39 PM, Edd Dawson <lists@mr-edd.co.uk> wrote:
Sounds very similar to something I did a while back: http://www.mr-edd.co.uk/?page_id=91
It's rather late now, so I'll take a closer look tomorrow, perhaps.
Yea, I see that Boost.Coroutine in the vault right under that Boost.Fiber. :) Different styles of handling different things. They might be able to be merged together? _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Beside fibers conceptually equivalent constructs are co-routines. A co-routine can be seen as a language-level construct while a fiber is a system-level construct. Boost.Fiber is modeled after the interface of Boost.Thread and this API is at least for me more convenient than co-routines (that means I'm more familiar with the usage of threads) and Boost.Fiber was written for support of Boost.Task (non-blocking tasks inside the thread-pool). I don't think that the libs should be merged. regards, Oliver

Oliver Kowalke wrote:
Boost.Fiber is modeled after the interface of Boost.Thread and this API is at least for me more convenient than co-routines (that means I'm more familiar with the usage of threads) and Boost.Fiber was written for support of Boost.Task (non-blocking tasks inside the thread-pool).
I don't think that the libs should be merged.
Code that uses fibers can be somewhat gnarly by necessity, especially when exception handling and multiple platforms are thrown in to the mix. For example, I found that MinGW GCC's older SJLJ mechanism is particularly problematic (http://lists-archives.org/mingw-users/09843-bug-with-win32-fibers.html), though I suspect you're far more qualified to work around that than I, if you haven't already. Anyway, should boost really have two lots of code for this stuff? In other words, would it not be beneficial if one library was written in terms of the other? Kind regards, Edd

Edd Dawson wrote:
Oliver Kowalke wrote:
Boost.Fiber is modeled after the interface of Boost.Thread and this API is at least for me more convenient than co-routines (that means I'm more familiar with the usage of threads) and Boost.Fiber was written for support of Boost.Task (non-blocking tasks inside the thread-pool).
I don't think that the libs should be merged.
Code that uses fibers can be somewhat gnarly by necessity, especially when exception handling and multiple platforms are thrown in to the mix.
For example, I found that MinGW GCC's older SJLJ mechanism is particularly problematic (http://lists-archives.org/mingw-users/09843-bug-with-win32-fibers.html), though I suspect you're far more qualified to work around that than I, if you haven't already.
thx for the hint. I know that fibers are problematic - even on Windows fibers are not supported by the CLR/.NET stuff (because hitting some concepts). maybe I should collect the DONTs in the docu.
Anyway, should boost really have two lots of code for this stuff? In other words, would it not be beneficial if one library was written in terms of the other? maybe! I didn't looked into coroutine much because I needed another interface.
Kind regards,
Edd
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
best regards, Oliver

Hi Oliver, Oliver Kowalke-2 wrote:
I've re factored boost.fiber (-> boost vault http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Concurrent%20Programming).
The library provides a so called lightweight thread of execution (also known as user-space thread or fiber on WIN32). The API is modeled after boost.thread.
Beside a scheduler (currently simple round-robin) for the fibers the library provides sync. primitives like mutex, condition- and event-variables (auto-reset, manual-reset, count-down). Message can be passed between fibers via bounded-/unbounded-fifo.
Some feedback would be nice.
regards, Oliver _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Happy to see that you have announced Boost.Fifer as a stand alone library. Best, Vicente -- View this message in context: http://old.nabble.com/new-library-boost.fiber-in-vault-tp26453013p26455856.h... Sent from the Boost - Dev mailing list archive at Nabble.com.

Vicente Botet Escriba wrote:
Hi Oliver,
Oliver Kowalke-2 wrote:
I've re factored boost.fiber (-> boost vault http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Concurrent%20Programming).
The library provides a so called lightweight thread of execution (also known as user-space thread or fiber on WIN32). The API is modeled after boost.thread.
Beside a scheduler (currently simple round-robin) for the fibers the library provides sync. primitives like mutex, condition- and event-variables (auto-reset, manual-reset, count-down). Message can be passed between fibers via bounded-/unbounded-fifo.
Some feedback would be nice.
regards, Oliver _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Happy to see that you have announced Boost.Fifer as a stand alone library.
Best, Vicente
Hello Vicente, thanks! I'll start to integrate boost.fiber into boost.task and if all works we can start with the review of boost.task. best regards, Oliver

Oliver Kowalke-2 wrote:
I've re factored boost.fiber (-> boost vault http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Concurrent%20Programming).
The library provides a so called lightweight thread of execution (also known as user-space thread or fiber on WIN32). The API is modeled after boost.thread.
Beside a scheduler (currently simple round-robin) for the fibers the library provides sync. primitives like mutex, condition- and event-variables (auto-reset, manual-reset, count-down). Message can be passed between fibers via bounded-/unbounded-fifo.
Some feedback would be nice.
Just a remarks related to name space. Your namespace approach is mixed. From one side you have boost::this_fiber namespace and boost::fiber class and the namespace boost::fibers. IMO you should either put all on boost or on boost::fibers but not both. Instead of boost::fiber f1=boost::fibers::make_fiber( some_fn); I would prefer boost::fiber f1=boost::make_fiber( some_fn); I have also some questions related to the scheduler. How many schedulers can be run on a thread? Do you have a use case of any interest to be able to run more than one? Could you talk about the need to migrate fibers from one thread to another? Thanks, Vicente -- View this message in context: http://old.nabble.com/new-library-boost.fiber-in-vault-tp26453013p26456641.h... Sent from the Boost - Dev mailing list archive at Nabble.com.

Vicente Botet Escriba wrote:
Oliver Kowalke-2 wrote:
I've re factored boost.fiber (-> boost vault http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Concurrent%20Programming).
The library provides a so called lightweight thread of execution (also known as user-space thread or fiber on WIN32). The API is modeled after boost.thread.
Beside a scheduler (currently simple round-robin) for the fibers the library provides sync. primitives like mutex, condition- and event-variables (auto-reset, manual-reset, count-down). Message can be passed between fibers via bounded-/unbounded-fifo.
Some feedback would be nice.
Just a remarks related to name space. Your namespace approach is mixed. From one side you have boost::this_fiber namespace and boost::fiber class and the namespace boost::fibers. IMO you should either put all on boost or on boost::fibers but not both.
I choosed the same style as boost.uuid and other libs: namespace uuids { class uuid ... } using uuids::uuid; The problem is that some classes as mutex of boost.thread reside in namespace boost (->boost::mutex). Thatswhy I put all classes into ns boost::fibers. The class boost::fiber is a using clause in ns boost (using boost::fibers::fiber). I choosed boost::this_fiber because of the analogy to boost.thread (boost::this_thread).
Instead of
boost::fiber f1=boost::fibers::make_fiber( some_fn);
I would prefer boost::fiber f1=boost::make_fiber( some_fn);
name collisions with boost::mutex, boost::condition etc.
I have also some questions related to the scheduler. How many schedulers can be run on a thread? Do you have a use case of any interest to be able to run more than one?
The intention is to use only one scheduler in a thread, but you can use one scheduler in multiple threads.
Could you talk about the need to migrate fibers from one thread to another?
Be aware that sharing data between threads/cpus costs a lot of cpu cycles. In future work fibers maybe explicit migrated to other threads/schedulers -> load balancing.
Thanks, Vicente
regards, Oliver

Oliver Kowalke-2 wrote:
Vicente Botet Escriba wrote:
Oliver Kowalke-2 wrote:
I've re factored boost.fiber (-> boost vault http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Concurrent%20Programming).
The library provides a so called lightweight thread of execution (also known as user-space thread or fiber on WIN32). The API is modeled after boost.thread.
Beside a scheduler (currently simple round-robin) for the fibers the library provides sync. primitives like mutex, condition- and event-variables (auto-reset, manual-reset, count-down). Message can be passed between fibers via bounded-/unbounded-fifo.
Some feedback would be nice.
Just a remarks related to name space. Your namespace approach is mixed. From one side you have boost::this_fiber namespace and boost::fiber class and the namespace boost::fibers. IMO you should either put all on boost or on boost::fibers but not both.
I choosed the same style as boost.uuid and other libs:
namespace uuids { class uuid ... }
using uuids::uuid;
The problem is that some classes as mutex of boost.thread reside in namespace boost (->boost::mutex). Thatswhy I put all classes into ns boost::fibers. The class boost::fiber is a using clause in ns boost (using boost::fibers::fiber).
I choosed boost::this_fiber because of the analogy to boost.thread (boost::this_thread).
Instead of
boost::fiber f1=boost::fibers::make_fiber( some_fn);
I would prefer boost::fiber f1=boost::make_fiber( some_fn);
name collisions with boost::mutex, boost::condition etc.
Ok I see. I'll put all the conflicting stuff on the namespace fibers, but I would let Non-member template function make_fiber() on boost namespace. BTW, it is not clear in the reference documentation on which namespace this function is in. I have found also an incoherence between the implementation and the documentation concerning the classes disable_interruption and restore_interruption. They should be defined on the this_fiber namespace, but I found them declared on boost::fibers. Am I missing a using declaration?
I have also some questions related to the scheduler. How many schedulers can be run on a thread? Do you have a use case of any interest to be able to run more than one?
The intention is to use only one scheduler in a thread, but you can use one scheduler in multiple threads.
Could we say that we can have thread specific schedulers and multi-threaded schedulers? I see that you plan to implement other scheduler features, e.g. taking care of priorities. I see also that there are some functions that depends on the scheduler class. Of course we can have a single scheduler class that would take care of all the scheduler intricacies, but I think that it will interesting to explore the possibility to have several models of a Scheduler concept, and see if making this possible don't would decrease the performances or even if this could allow to improve them. If we would have specific schedulers, the current scheduler class would be just one of them, and should be named with its current specific purpose.
Could you talk about the need to migrate fibers from one thread to another?
Be aware that sharing data between threads/cpus costs a lot of cpu cycles. In future work fibers maybe explicit migrated to other threads/schedulers -> load balancing.
Maybe it is worth considering migrating and non-migrating fibers respect to threads and schedulers. What do you think? What about having fiber specific storage, something like fiber_specific_ptr? Would this be possible without breaking the current design? What about a fiber barrier? Have you explored how fibers and futures interact? Let me know if I can help you on any task, I'm really interested in this library. Continue the good work, Vicente -- View this message in context: http://old.nabble.com/new-library-boost.fiber-in-vault-tp26453013p26465136.h... Sent from the Boost - Dev mailing list archive at Nabble.com.

Vicente Botet Escriba wrote:
Ok I see. I'll put all the conflicting stuff on the namespace fibers, but I would let Non-member template function make_fiber() on boost namespace. BTW, it is not clear in the reference documentation on which namespace this function is in.
It's not a big thing to move make_fiber() etc. to boost namespace. I did found any advice in the boost documentation what is the preferred way.
I have found also an incoherence between the implementation and the documentation concerning the classes disable_interruption and restore_interruption. They should be defined on the this_fiber namespace, but I found them declared on boost::fibers. Am I missing a using declaration?
Thx - I'll correct this soon (including adding namspace).
Could we say that we can have thread specific schedulers and multi-threaded schedulers?
Would be nice but requires a careful design.
I see that you plan to implement other scheduler features, e.g. taking care of priorities. I see also that there are some functions that depends on the scheduler class. Of course we can have a single scheduler class that would take care of all the scheduler intricacies, but I think that it will interesting to explore the possibility to have several models of a Scheduler concept, and see if making this possible don't would decrease the performances or even if this could allow to improve them. If we would have specific schedulers, the current scheduler class would be just one of them, and should be named with its current specific purpose.
I plan to implement several scheduling algorithms selectable via an enumeration passed to the ctor of the scheduler class. Another possibility would be to do make scheduler to an template and the template argument specifies the scheduling algorithm (implementation). template< typename Strategy > class scheduler; scheduler< round_robin > rr_sched; scheduler< bfs > bfs_sched; This would break the usage of boost::this_fiber::yield() for example, because the functions in namespace boost::this_fiber access the scheduler via a static member and need to know the exact type of the scheduler.
Maybe it is worth considering migrating and non-migrating fibers respect to threads and schedulers. What do you think?
Sounds reasonable if we can get a correct working implementation.
What about having fiber specific storage, something like fiber_specific_ptr? Would this be possible without breaking the current design?
What about a fiber barrier?
would be nice
Have you explored how fibers and futures interact?
I'll start to integrate boost.fiber into boost.task: boost::fibers::scheduler sched; boost::task::handle< int > h = boost::task::async( boost::task::make_task( my_fn), sched); ...
Let me know if I can help you on any task, I'm really interested in this library.
Thx - I think of what should be the next steps (together with boost.task). best regards, Oliver

Oliver Kowalke-2 wrote:
Vicente Botet Escriba wrote:
Ok I see. I'll put all the conflicting stuff on the namespace fibers, but I would let Non-member template function make_fiber() on boost namespace. BTW, it is not clear in the reference documentation on which namespace this function is in.
It's not a big thing to move make_fiber() etc. to boost namespace. I did found any advice in the boost documentation what is the preferred way.
I have found also an incoherence between the implementation and the documentation concerning the classes disable_interruption and restore_interruption. They should be defined on the this_fiber namespace, but I found them declared on boost::fibers. Am I missing a using declaration?
Thx - I'll correct this soon (including adding namspace).
Could we say that we can have thread specific schedulers and multi-threaded schedulers?
Would be nice but requires a careful design.
I see that you plan to implement other scheduler features, e.g. taking care of priorities. I see also that there are some functions that depends on the scheduler class. Of course we can have a single scheduler class that would take care of all the scheduler intricacies, but I think that it will interesting to explore the possibility to have several models of a Scheduler concept, and see if making this possible don't would decrease the performances or even if this could allow to improve them. If we would have specific schedulers, the current scheduler class would be just one of them, and should be named with its current specific purpose.
I plan to implement several scheduling algorithms selectable via an enumeration passed to the ctor of the scheduler class. Another possibility would be to do make scheduler to an template and the template argument specifies the scheduling algorithm (implementation).
template< typename Strategy > class scheduler;
scheduler< round_robin > rr_sched; scheduler< bfs > bfs_sched;
This would break the usage of boost::this_fiber::yield() for example, because the functions in namespace boost::this_fiber access the scheduler via a static member and need to know the exact type of the scheduler.
Yes I have already remarked this fact. If we restraint to having only one scheduler instance running on a thread, we can transform these static calls to polymorphic calls. This will opens the introduction of schedulers external to the library. The call of virtual functions could decrease the performances of some functions, but a test on the enumeration should be equivalent.
Maybe it is worth considering migrating and non-migrating fibers respect to threads and schedulers. What do you think?
Sounds reasonable if we can get a correct working implementation.
Well if I understand, up to now you have implemented non-migrating fibers. The idea could be to introduce migrating fibers on a separated class, and don't decrease the performances of a simple tool. I have yet another question. I understand we need fibers::mutex and fibers::condition_variable, but could you explain why do we need a separated fibers::lock_guard and fibers::unique_lock template classes? Why the ones from Boost.Thread are not usable in the fiber context, at the end the Mutex parameter can be any model of lockable. Best, Vicente -- View this message in context: http://old.nabble.com/new-library-boost.fiber-in-vault-tp26453013p26467888.h... Sent from the Boost - Dev mailing list archive at Nabble.com.
participants (5)
-
Agustín K-ballo Bergé
-
Edd Dawson
-
Oliver Kowalke
-
OvermindDL1
-
Vicente Botet Escriba