Marcus Tomlinson wrote:
Note that in a sense the DSPatch wire is a special case of the producer-consumer queue where the buffer has only one slot, so it's always either full or empty, forcing the producer and consumer components to work strictly in tandem.
Quick question about this. In FastFlow (and other dataflow libraries), when you create parallel branches of components (equal length or of different number of components), does the result of a piece of data diverging across these branches arrive at the converging point all at the same time?
Probably not (regardless of the length of the parallel branches). Is this something that DSPatch would guarantee, even when the branches have different numbers of components? That would be interesting.
The other feature is the use of dedicated threads to create one-to- many and many-to-one connections out of only pure one-to-one connections. The FastFlow authors refer to these as "emitters" and "collectors", respectively. This simplifies the implementation and improves throughput in scenarios where a thread has multiple inputs or multiple outputs.
Referring to a thread as having multiple inputs and outputs leads me to believe that a "threads" and "components" / "processes" are directly related? Does this mean the more components you have, the more threads you require?
Yes, for convenience I've been assuming a one-to-one relationship between components and threads. This is the default in FastFlow and I think it makes sense. It isn't a necessity, though; you could also take a fixed number of threads and have them handle a variable number of components in turn. I'm not sure whether or how FastFlow implements such a thing but I'm sure it can be done in principle. -Julian