On Sat, 22 May 2004 20:20:07 +1000, Arash Partow wrote
Hi all,
I was wondering if there would be any interest in a template library based around producers and consumers.
Sure, but....
I've developed something already and have a series of working examples, I propose a library that consist of 3 basic entities, a producer a consumer and a hybrid entity called an interchange.
the entities themselves will be threaded and will be linked together
Making each element of the processing chain a thread is rather constraining. What if I want to use a leaders / followers approach to threading my chain of transformers? So when I get a callback on my socket I chain thru all the transformations in the callback thread thus avoiding several thread context switches just to process the data.
via a linking mechanism that is itself a base template but which can be extended to provide data transfer between same system processes and threads and also extended versions which provide data transfer over pipes, sockets, serial lines etc..
the idea of the producer-consumer is very simple the producer produces a type of data ie: string, a class, a struct etc.. and a consumer that is instantiated as a consumer of the type will consume the data provided by the producer and do something with it.
How does this differ from the signals and slots library? http://www.boost.org/doc/html/signals.html Couldn't we just create a slot endpoint for the various types? Can I connect 2 or more consumers to a particular interchange? For this to be useful, I should be able to.
...snip detail...
The link mechanism is a basically a type that supports the common type between a producer and its consumer, it is a one way flow of data, the consumer is signaled when data is ready for it to collect
Again, seems like signals and slots to me...
...snip...
A link has a basic get and add functionality but depending what entity is being implemented they will only see it via an interface and hence only be able to invoke 1 kind of method, a producer can only invoke "add" where-as a consumer can only invoke a get, an interchange can invoke both because it is instantiated with both a consumer and producer link interface, but the links are separate for an interchange, because it requires 2 links, one link for the incoming data and one link for the out going data. both link maybes of different types, ie:
Producer <string> ----> Interchange <string, AClass> ----> Consumer <AClass>
I need some opinions, about whether or not people will find this library useful and hence have it as part of Boost, if so could people give me their opinions about my design, its just a simple first trial run at designing it, i intended for the producers and interchange entities to be policy driven as far as production of data is concerned.
Well, I think it could be useful and wouldn't mind having it in boost, but it seems like we need other basic infrastructure like a reactor, callback timers, and sockets before there will be much advantage.
I think this library can be used in distributed computing, in network packet assembly, and also for interfacing and passing data up and down threaded software layers.
Yes, it is a useful design in 'data flow' systems. Distributed publish-subscribe type systems which have various data transformers use this type of design (or they should). Jeff