a new release of Join library. Could a review be arranged?

Hi there, I just uploaded a new version of Join library into Vault under "Concurrent Programming". Could a review be arranged? A brief description: "Join is an asynchronous, message based C++ concurrency library based on join calculus. It is applicable both to multi-threaded applications and to the orchestration of asynchronous, event-based applications. It follows Comega's design and implementation and builds with Boost facilities. It provides a high level concurrency API with asynchronous methods, synchronous methods, and chords which are "join-patterns" defining the synchronization, asynchrony, and concurrency." This update includes code cleanup and exploring using Join for dynamic flow based programming. project website: http://channel.sourceforge.net doc: http://channel.sourceforge.net/boost_join/libs/join/doc/boost_join_design.ht... download: http://sourceforge.net/project/showfiles.php?group_id=157583 Thanks Yigong

Yigong Liu wrote:
Hi there,
I just uploaded a new version of Join library into Vault under "Concurrent Programming".
Have you considered benchmarking your library against Joins (the .NET library), but also Cw and JOCaml ? It could be interesting to know how good your library performs against other library implementations as well as against language implementations. Writing the same code with multiple languages or libraries can also help people used to those to move to your Join library.

Mathias Gaunard wrote:
Have you considered benchmarking your library against Joins (the .NET library), but also Cw and JOCaml ?
It could be interesting to know how good your library performs against other library implementations as well as against language implementations.
I am planning to do some comparison with C#.Joins library. Maybe Jocaml is good next candidate if i got more spare time for it.
Writing the same code with multiple languages or libraries can also help people used to those to move to your Join library.
most of tutorials compare Cw code and Join's code for the same application. The newly added parts also compare CCR's code and Join's code for the same application; and there is a brief comparison between Erlang and Join too. Thanks for the quick feedback. Yigong

Hello, are you asking for a formal review schedule? Best regards Vicente ----- Original Message ----- From: "Yigong Liu" <yigongliu@gmail.com> To: <boost@lists.boost.org> Sent: Monday, March 31, 2008 5:38 AM Subject: [boost] a new release of Join library. Could a review be arranged?
Hi there,
I just uploaded a new version of Join library into Vault under "Concurrent Programming". Could a review be arranged?
A brief description:
"Join is an asynchronous, message based C++ concurrency library based on join calculus. It is applicable both to multi-threaded applications and to the orchestration of asynchronous, event-based applications. It follows Comega's design and implementation and builds with Boost facilities. It provides a high level concurrency API with asynchronous methods, synchronous methods, and chords which are "join-patterns" defining the synchronization, asynchrony, and concurrency."
This update includes code cleanup and exploring using Join for dynamic flow based programming.
project website: http://channel.sourceforge.net doc: http://channel.sourceforge.net/boost_join/libs/join/doc/boost_join_design.ht... download: http://sourceforge.net/project/showfiles.php?group_id=157583
Thanks Yigong _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Hello there, Yes i am asking for a formal review schedule, which is much needed for further improvement of this library. Thanks Yigong On Sun, May 11, 2008 at 3:51 PM, vicente.botet <vicente.botet@wanadoo.fr> wrote:
Hello,
are you asking for a formal review schedule?
Best regards
Vicente
----- Original Message ----- From: "Yigong Liu" <yigongliu@gmail.com> To: <boost@lists.boost.org> Sent: Monday, March 31, 2008 5:38 AM Subject: [boost] a new release of Join library. Could a review be arranged?
Hi there,
I just uploaded a new version of Join library into Vault under "Concurrent Programming". Could a review be arranged?
A brief description:
"Join is an asynchronous, message based C++ concurrency library based on join calculus. It is applicable both to multi-threaded applications and to the orchestration of asynchronous, event-based applications. It follows Comega's design and implementation and builds with Boost facilities. It provides a high level concurrency API with asynchronous methods, synchronous methods, and chords which are "join-patterns" defining the synchronization, asynchrony, and concurrency."
This update includes code cleanup and exploring using Join for dynamic flow based programming.
project website: http://channel.sourceforge.net doc:
http://channel.sourceforge.net/boost_join/libs/join/doc/boost_join_design.ht...
download: http://sourceforge.net/project/showfiles.php?group_id=157583
Thanks Yigong _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Hi Yigong, I have just see your two libraries Join and Channel on the vault, excelent work. For the moment some superficial comments and questions? 1. There is a lot of NULL constants in the code that should be replaced by 0 :) 2. In the Erlang example you state "Erlang check messages types during runtime, if Erlang server receives a unexpected message, "Other -> ..." section is invoked. In Join based server, the message types are defined at interface and checked by compiler. Because sending a message to Join based server is to invoke its async methods, so compiler can prevent invoking unsupported methods." Can we emulate the "Other -> ..." semantics with Join? What about AnyMessage and Timeout? 3. FYI, there is another join library for the language Scala (jolib). Scala provides also an actor library based on Erlang, but you know surelly that already. How jolib compares to your library? 4. select template in actor.h could be replaced by mpl::if_c template <bool flag, typename T, typename U> struct select { typedef T result; }; template <typename T, typename U> struct select<false, T, U> { typedef U result; }; 5. bitmap seams a good candidate to boost template <size_t size> struct bitmap 6. In the Comega comparaison you state "chord override Cw and Join have different rules for chord overriding." Which ones and which is the rationalle? 7. In the CCR comparaison you state "In CCR, arbiters roughly correspond to chords in Join. However arbiters take more roles than chords:" Do you mean that chords could be defined in terms of arbiters? is arbiter a lower level concept than chord? and "In CCR, ports (the message channels) are totally independent from join-patterns or arbiters, instead arbiters are attached to ports. Ports can be used independently from arbiters as plain message queues. However in Join, the async<> / synch<> methods or channels must be attached to chords before they can be used (invoked), otherwise exceptions will be thrown. This is more consistent with original join-calculus and Jocaml." Why not defining a independent port, which should be already useful (the CCR port recall me the future_stream), and let async<> / synch<> the actor and chord dependency? 8. Could you provide a more deep comparaison to C#.Joins? 9. "The Join library is implemented as three layers:", Are the interfaces of the base layer port public? They do not appear on the "Synopsis of Join Classes" 10. I would like to cuple asynchronous methods and futurers, giving the to possibility asynchronous methods to returnn some data. Do you think that this is possible? 11. The name of the files and the classes do not match too much. E.g. the port class is found in the file join_base.hpp, and the port.h files contains sync_v/async_v sync_f/async_f. Could you do something? 12. Do you plan to provide variable number of arguments to sync/async methods and to chords using the Boost.preprocesor library, and why not the C++0x varying templates feature? 13. If actor and joint are the same, why not define a typedef? 14. As the goal of such a library is to scale better than mapping concurrent components to threads, a comparaison on performance and number of actors should be included. People need this to adopt the paradigm change. To finish, do you intend to submit your Channel library to boost? Best regards ____________________ Vicente Juan Botet Escriba

Hello Vicente, Thanks for looking into Join and Channel. Your questions brought me to do some homework (put some time into studying future proposals). Sorry for the delay. Here is the reply. 1. There is a lot of NULL constants in the code that should be replaced by 0
:)
Sure
2. In the Erlang example you state "Erlang check messages types during runtime, if Erlang server receives a unexpected message, "Other -> ..." section is invoked. In Join based server, the message types are defined at interface and checked by compiler. Because sending a message to Join based server is to invoke its async methods, so compiler can prevent invoking unsupported methods."
Can we emulate the "Other -> ..." semantics with Join? What about AnyMessage and Timeout?
Erlang's message pattern matching happens at runtime while Join's matching/resolution thru function signatures happens at compile time. Compilers will prevent senders to invoke undefined async methods (or send undefined messages) to "joined" object, so "Other->..." scenario cannot happen with Join's translation of Erlang sample presented in design doc. However we do need "Other->..." when implementing a more traditional message passing design. Normally we'll have a generic message container: class message { public: int type; char *data; } class Server : public actor { synch<void()> receive; public: async<void(shared_ptr<message>)> send; Server() { chord(receive, send, &Server::proc_msg); new thread(bind(&Server::main_loop, this); } void main_loop() { for(;;) receive(); } void proc_msg(synch_o<void()> r, async_o<void(shared_ptr<message>)> s) { shared_ptr<message> msg = s.arg1; switch(msg->type) { case type1: ... handle msg type 1; break; case type2: ... handle msg type 2; break; default: cout << "unknown msgs"; break; } } }; Here the message type resolution happens during runtime. Since c/c++ doesnt have pattern matching so we have to switch based on msg->type. The "default" case is similar to Erlang's "other" case and the 1st messages of msg queue is removed no matter what type it is, which is similar to AnyMessage. Regarding to Timeout, only synch methods need it (since async methods always return immediately). There are 2 ways to implement it: 1> use a timeout channel (async method), so the above code become: class Server ... synch<void()> receive; async<void(shared_ptr<message>)> send; async<void()> timeout; Server() { chord(receive, send, &Server::proc_msg); chord(receive, timeout, &Server::handle_timeout); ... } void main_loop() { for (;;) { setup a timer with a callback which will call async method timeout() when timeout happens receive(); } } 2> add timeout directly into synch<...> template classes which requires some changes
3. FYI, there is another join library for the language Scala (jolib). Scala provides also an actor library based on Erlang, but you know surelly that already. How jolib compares to your library?
Yes i browsed the scala-join paper sometime back and really hoped that i could spent more time on it and scala itself. It is a library (so we'll miss the checking and optimization by compiler), but Scala's nice facilities (such as extensible pattern matching) make it just like built-in. It supports multiple synch methods per chord and "guards" which i experimented with in early versions of Join and dropped. Each method or event support only one argument so we need use tuple to wrap and unwrap multi arguments which is similar to the port api of Join. I need to do more study before i can go into details.
4. select template in actor.h could be replaced by mpl::if_c
template <bool flag, typename T, typename U> struct select { typedef T result; }; template <typename T, typename U> struct select<false, T, U> { typedef U result; };
As mentioned in the "Installation" section of design doc, the implementation is based/focused on library TR1 features which will be first added to next C++ standard, available to normal C++ users by default. Boost::mpl is a wonderful library but i dont use its full power.
5. bitmap seams a good candidate to boost template <size_t size> struct bitmap
if enough people are interested...
6. In the Comega comparaison you state "chord override Cw and Join have different rules for chord overriding."
Which ones and which is the rationalle? Cw/Jocaml talks about "method overriding" and to avoid so-called inheritance anomaly ( http://pauillac.inria.fr/~maranget/papers/inheritance.ps.gz), they designed really complex override rules which is difficult to use. Join lib talks about "chord override", ie all the methods of a overriden chord get new behaviour whose implementation is really simple. IMHO, inheritance anomaly is caused mostly by implementation inheritance (use inheritance to reuse/extend implementation). Since current OO rule is to avoid implementation inheritance and to use interface inheritance, i believe the issue is not as big as before.
7. In the CCR comparaison you state "In CCR, arbiters roughly correspond to chords in Join. However arbiters take more roles than chords:" Do you mean that chords could be defined in terms of arbiters? is arbiter a lower level concept than chord?
and "In CCR, ports (the message channels) are totally independent from join-patterns or arbiters, instead arbiters are attached to ports. Ports can be used independently from arbiters as plain message queues. However in Join, the async<> / synch<> methods or channels must be attached to chords before they can be used (invoked), otherwise exceptions will be thrown. This is more consistent with original join-calculus and Jocaml."
Why not defining a independent port, which should be already useful (the CCR port recall me the future_stream), and let async<> / synch<> the actor and chord dependency?
while simple arbiters (such as Arbiter.Receive and Arbiter.JoinedReceiver) roughly correspond to chords, combo arbiters such as Choice/Interleave presents a hierarchy of arbiters. i'd like to think chord is a simpler (lower level) concept. Since CCR and Join have totally different "internal engine", we cannot really define one with another. Join's primitives async/synch/chords are fundamental in the sense that it can be used to implement other concepts. Port or message queue is basically a buffer: template <typename MsgType> class Port : public actor { public: async<void(MsgType)> post; synch<MsgType()> recv; Port() { chord(recv, post, &Port::forward); } MsgType forward(synch_o<MsgType()> r, async_o<void(MsgType)> p) { return p.arg1; } }; This is a thread safe port or message queue, so it doesnt need to be part of basic primitives.
8. Could you provide a more deep comparaison to C#.Joins?
when i got more time, i could do it later.
9. "The Join library is implemented as three layers:", Are the interfaces of the base layer port public? They do not appear on the "Synopsis of Join Classes"
I first had port api and then added the functional api; and port api is self complete. One problem with port api is that you have to wrap/unwrap arguments with tuples. Originally the samples are written for both apis under func_api/ and port_api/. It became too much a burden to maintain both so port_api/ is dropped. If people are really interested in the port api, the doc can be added and samples added.
10. I would like to cuple asynchronous methods and futurers, giving the to possibility asynchronous methods to returnn some data. Do you think that this is possible?
Join's async method is really "send a message and go", you cannot really get anything back from it, including "future" variables. However futures can be implemented using async/synch/chords, which i'd like to expand on in next email. Best regards. Yigong

yigong liu-2 wrote:
Hello there,
Yes i am asking for a formal review schedule, which is much needed for further improvement of this library.
Hi, Great that you're still working on this, are you working on channel too? Before submitting a formal review request, you first need to ask if there's any interest in the library. Please follow the submission guidelines at http://www.boost.org/development/submissions.html Good luck! Johan -- View this message in context: http://www.nabble.com/a-new-release-of-Join-library.-Could-a-review-be-arran... Sent from the Boost - Dev mailing list archive at Nabble.com.

I did ask about any interest in my first email a (long) while back about channel and join and people expressed interests:) yes i am still working on channel. Thanks Yigong
Before submitting a formal review request, you first need to ask if there's any interest in the library. Please follow the submission guidelines at http://www.boost.org/development/submissions.html
participants (4)
-
Johan Torp
-
Mathias Gaunard
-
vicente.botet
-
Yigong Liu