I wrote:
I'm certainly interested in a Boost library that offers efficient flow-based multithreading with an interface that is as simple as possible. In fact I have some ideas of my own which I would gladly present to the mailing list when I find more time.
In the meanwhile I would like to refer to FastFlow [1], which as far as I can tell hasn't been mentioned yet in this thread. It's similar to DSPatch with respect to OOD, but it offers two particular features that I think are more powerful and would be more fitting for a Boost submission. The first is the use of producer-consumer queues as the primary means to transfer data between two threads. I believe this is really THE way to go, because the data are buffered and neither producer nor consumer will need to wait for the other unless the buffer is full or empty. Note that in a sense the DSPatch wire is a special case of the producer-consumer queue where the buffer has only one slot, so it's always either full or empty, forcing the producer and consumer components to work strictly in tandem. The other feature is the use of dedicated threads to create one-to- many and many-to-one connections out of only pure one-to-one connections. The FastFlow authors refer to these as "emitters" and "collectors", respectively. This simplifies the implementation and improves throughput in scenarios where a thread has multiple inputs or multiple outputs. Moreover, in combination with generic design this would allow for some nifty off-the-shelf tools such as tuple zipping/unzipping, distributing/interlacing, etcetera. Topher Cooper independently came up with a similar idea in [2]. Nonetheless FastFlow still isn't my "dream library". One reason for that is that it isn't generic. Another reason is that to my taste it behaves too much like a framework instead of like a toolkit. One symptom of that is the requirement on users to derive from a class (like in DSPatch) and to override methods. Finally, while FastFlow makes an admirable attempt at providing abstractions for complex high- level use cases ("skeletons"), doing simple things is not simple (thankfully adopting this way of talking about simplicity from [2]). Hope this will help the discussion! -Julian _______ [1] http://calvados.di.unipi.it/dokuwiki/doku.php?id=ffnamespace:about [2] http://lists.boost.org/Archives/boost/2012/12/199333.php