Boost.Join - asynchronous concurrency library based on Cw and Join Calculus

Hello, Any interests in a boost asynchronous concurrency library based on Cw and Join Calculus? Boost.Join follows Cw's design and supports lock-free concurrent design with a few abstractions: 1. asynchronous methods: async<void (T1, T2, ...)> A asynchronous method is one-way, no-result, non-blocking call; essentially passing a message. A asynchronous call is guaranteed to return immediately; internally there could be a queue to buffer the arguments or message. 2. synchronous methods: synch<R (T1, T2, ...)> A synchronous call will block till result is returned. However a synchronous call is different from normal method call in that it involves multithreads and synchronization. 3. chords or "joined" method body definition: for a normal function/method, there is 1-1 exact correspondence between a function header/signature and its body. In Cw and Boost.Join, a body can be associated with a set of (synchronous and/or asynchronous) methods. Such a definition is called "chord". A particular async/synch method may appear in the header of several chords. The body of a chord can only execute when all the methods in its header have been called. In Boost.Join, a thread safe buffer can be defined as following: class buffer: public actor<> { public: async<void(string)> put; synch<string(void)> get; buffer() { chord(put, get, &buffer::chord_body); } void chord_body(async_o<void(string)> put, synch_o<string(void)> get) { get.reply(put.arg1); //or equally, get.reply(put); } }; such a buffer can be safely used in multithreaded applications: buffer b; b.put("Hello"); b.put("World"); cout << b.get() << b.get() << endl; More detailed info can be found at http://channel.sourceforge.net Over a dozen samples are provided, mostly using Boost.Join to implement common concurrency idioms such as semaphores, futures and active objects. Boost.Join is a header only library. All code and samples build and run in Linux with g++ and Windows with VC++2005express. Comments, suggestions and corrections are highly appreciated! Thanks Yigong

What does it bring that Channel does not?
Hello,
Yes, both Channel and Join are asynchronous messaging oriented. I have been working on restructuring/rewriting the internal of Channel, this is a offshoot from that work. In Channel, the join-pattern is implemented as two phase commit the same way as CCR did it. During the rewriting, for a comparison of simplicity and expressiveness, i really like to have a Join implementation in the original Cw (or Jocaml) style and here is the result. Internally the join-patterns are represented as bitmasks, many optimizations mentioned in Cw paper and implemented by Cw compiler ( http://research.microsoft.com/Comega/) are implemented by some template techniques. After using it to recode over a dozen Cw tutorial samples, i found it quite usable by itself and so i packaged it up and released it separately. In today's multicore world, people need high level tools for the orchestration of asynchronous and synchronous concurrent activities (besides distributed messaging and low level threads and synchronization), as evidenced by Microsoft continuous efforts on Join calculus (Cw, CCR(Coordination and Cocurrency Runtime), C# Joins lib) and Intel 's Threading Build Block library. Hope this small lib can be some help to our community. Also the API is "functional" oriented (we are talking about async / synch "methods') which may be attractive to people who dont like the explicit message passing API (send/recv) of Channel. With different API and different internal implementation, hope Channel and Join can find their applications. Thanks Yigong

Yigong Liu wrote:
With different API and different internal implementation, hope Channel and Join can find their applications.
Couldn't Join be implemented on top of Channel? Or does having a specific implementation allow Join to be more efficient?

What does it bring that Channel does not?
Hello, Here are a few more points to clarify the difference between Channel and Join: Channel's major purpose is to support asynchronous message passing thru "name spaces". Message senders bind to "names" in name space to send and receivers bind to "names" to receive. Name matching rules decide which senders and receivers will bind together and communicate. We could use linear / hierarchical / associative name spaces for different applications. Join intends to be a rather complete implementation of Cw asynchronous concurrency features. Its counterpart in C# is "Joins" library ( http://research.microsoft.com/~crusso/joins/index.htm). Channel's focus is to support transparent distributed message passing. Senders bind to names to send while the receivers could be threads in different processes or machines which bind to the same (matching) names to receive. Both senders and receivers dont need to be aware whether their communicating peers are local or remote. Join more focus on giving a good treatment of "local" orchestration of both async and synch concurrent activities. Join's primitives (async / synch methods and chords) are more "fundamental", in that we could use them to implement other concurrency idioms (such as semaphores, futures and active objects), In Channel, "join-pattern" is used as one of messaging coordination arbiters (choice and join) of pull dispatcher, one of many dispatching algorithms (including synchronous broadcast, round robin ...). It implements mostly the asynchronous part of Cw (thru two phase commit), in the same style as CCR ( http://channel9.msdn.com/wiki/default.aspx/Channel9.ConcurrencyRuntime). Compared with Channel, Join is a fairly small and simple library and header only. For more information, please refer to http://channel.sourceforge.net. Regards, Yigong

Thanks for the info. 1. Is/isn't it possible to merge the functionality of Join and Channel. and just have the Channel lib. ie. why 2 separate libraries? 2. Are you going to propose one or both libs for Boost review, if so when? 3. Channel could play a role in a Boost.Logging library, I think so? Thanks Shams -- "Yigong Liu" <yigongliu@gmail.com> wrote in message news:215932780705202227n5d79c7a0pa3c1a28768ed7a9f@mail.gmail.com...
What does it bring that Channel does not?
Hello,
Here are a few more points to clarify the difference between Channel and Join:
Channel's major purpose is to support asynchronous message passing thru "name spaces". Message senders bind to "names" in name space to send and receivers bind to "names" to receive. Name matching rules decide which senders and receivers will bind together and communicate. We could use linear / hierarchical / associative name spaces for different applications.
Join intends to be a rather complete implementation of Cw asynchronous concurrency features. Its counterpart in C# is "Joins" library ( http://research.microsoft.com/~crusso/joins/index.htm).
Channel's focus is to support transparent distributed message passing. Senders bind to names to send while the receivers could be threads in different processes or machines which bind to the same (matching) names to receive. Both senders and receivers dont need to be aware whether their communicating peers are local or remote.
Join more focus on giving a good treatment of "local" orchestration of both async and synch concurrent activities. Join's primitives (async / synch methods and chords) are more "fundamental", in that we could use them to implement other concurrency idioms (such as semaphores, futures and active objects),
In Channel, "join-pattern" is used as one of messaging coordination arbiters (choice and join) of pull dispatcher, one of many dispatching algorithms (including synchronous broadcast, round robin ...). It implements mostly the asynchronous part of Cw (thru two phase commit), in the same style as CCR ( http://channel9.msdn.com/wiki/default.aspx/Channel9.ConcurrencyRuntime).
Compared with Channel, Join is a fairly small and simple library and header only.
For more information, please refer to http://channel.sourceforge.net.
Regards, Yigong _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Yigong Liu said: (by the date of Sat, 19 May 2007 14:57:14 -0700)
Boost.Join follows Cw's design and supports lock-free concurrent design with a few abstractions:
You speak of Boost.Join as if it was a part of boost. However I am unable to find its documentation in: http://boost.org/libs/libraries.htm found under Libraries->Documentation on the main site http://boost.org/ 1. Is it a part of boost? 2. where is its documentation? -- Janek Kozicki |

Janek Kozicki wrote:
You speak of Boost.Join as if it was a part of boost. However I am unable to find its documentation [...]
1. Is it a part of boost?
No, and neither is Boost.Channel. I don't think it's even in the review queue.
2. where is its documentation?
Yigong Liu gave the link to the project website which contains the documentation: http://channel.sourceforge.net

1. Is it a part of boost? 2. where is its documentation?
Sorry for the confusion. I'll submit it for formal review soon after receiving the initial feedback from the community and cleanup code and documents. Thanks for looking into it. Yigong

On 5/19/07, Yigong Liu <yigongliu@gmail.com> wrote:
Hello,
Any interests in a boost asynchronous concurrency library based on Cw and Join Calculus?
Boost.Join follows Cw's design and supports lock-free concurrent design with a few abstractions:
Is it really lock free? I see you are using boost::mutex, and I see no atomic ops. Unless you are cleverly using boost::shared_ptr for some kind of lock free queue, I can't see how this is truly lock free ( http://en.wikipedia.org/wiki/Lock-free_and_wait-free_algorithms). May be you meant something else lock-free or you still plan to do a lock free implementation.
Comments, suggestions and corrections are highly appreciated!
I've only done a cursory look at it, but it seems very interesting. I've downloaded it and plan to try it soon. gpd

Is it really lock free? I see you are using boost::mutex, and I see no atomic ops. Unless you are cleverly using boost::shared_ptr for some kind of lock free queue
Hello, By "lock-free", i am referring to the fact that in Join (or Cw) based applications, there is NO explicit usage of threads and synchronization such as locks. I am not referring to particular lock-free data structures or algorithms. It is more about programming model change; from the model of shared-state (protected by locks or other synchronization primitives) to the model of orchestration of async and sync concurrent activities. Although the code of applications based on Join do not contains locks, the code generated by Join templates do contain mutex and conditional variables. In other words, Join (and Cw) "hides" "low level" threads and synchronization from applications code by providing a high level API of async / synch methods and chords which allow flexible and direct specification of concurrency and synchronization. There are drawbacks with constructing concurrent applications directly using threads and locks. Java's executors and concurrency utils and Intel's threading build blocks provide many high level constructs to address this issue. IMHO, Join (Cw) 's minimal toolset of async / synch methods and chords are more "fundamental" in that it can easily encode many other high level constructs. In Java/C#, the basic synchronization idiom is synchronized methods which hold locks from begining of method. In Join (Cw), synchronization is defined by chords (join-patterns). Chords are declarative in that they explicitly specify the set of methods need to be called so that the chord body will execute. And when chord body runs, no lock is held anymore. So any synchronization needed must be specified and satified in header. Thanks for looking into it. Yigong

Hi Yigong, Could the difference between your join-based applications and other seemingly similar applications be characterized in the following way? Two different areas of activity for me have been running (small) projects and also developing highly asynchronous (i.e. event driven) software for the telephony world. As projects get bigger its harder and harder to ignore techniques such as critical path analysis. This technique most obviously brings some order to proceedings but it also identifies instances of concurrency. Through that concurrency the finish date is earlier than it otherwise would be. Is your "joining" library a tool for implementing the results of a software concurrency analysis? Rather than something targeted at the implementation of the highly interactive entities in a telephony protocol? Cheers. ----- Original Message ----- From: "Yigong Liu" <yigongliu@gmail.com> To: <boost@lists.boost.org> Sent: Tuesday, May 22, 2007 6:41 PM Subject: Re: [boost] Boost.Join - asynchronous concurrency library based onCw and Join Calculus
Is it really lock free? I see you are using boost::mutex, and I see no atomic ops. Unless you are cleverly using boost::shared_ptr for some kind of lock free queue
Hello,
By "lock-free", i am referring to the fact that in Join (or Cw) based applications, there is NO explicit usage of threads and synchronization such as locks. I am not referring to particular lock-free data structures or algorithms. It is more about programming model change; from the model of shared-state (protected by locks or other synchronization primitives) to the model of orchestration of async and sync concurrent activities. Although the code of applications based on Join do not contains locks, the code generated

On 5/23/07, Scott Woods <scottw@qbik.com> wrote:
Is your "joining" library a tool for implementing the results of a software concurrency analysis? Rather than something targeted at the implementation of the highly interactive entities in a telephony protocol?
Hello, Join calculus , as a process calculus, has been used for formal specification and validation of concurrent and distributed systems, including protocols. Its simple and well-defined core for concurrency (async / synch methods and join-patterns or chords) are well suited for programming and have been integrated with programming languages for real concurrent applications in two different ways: 1. Join's primitives are integrated into languages by changing compilers; such as JoCaml, Cw and Join.Java. 2. Join's primitives are added as a separate library; such as CCR, C#.Joins. The "Join" library i am proposing falls into this category. It is intended for implementing real concurrent applications. Its simple model of async / synch methods and join-patterns (chords) should give us guideline for concurrent design and help avoiding pitfalls. Thanks Yigong

Is your "joining" library a tool for implementing the results of a software concurrency analysis? Rather than something targeted at the implementation of the highly interactive entities in a telephony protocol?
Hello,
Join calculus , as a process calculus, has been used for formal specification and validation of concurrent and distributed systems, including protocols.
Its simple and well-defined core for concurrency (async / synch methods and join-patterns or chords) are well suited for programming and have been integrated with programming languages for real concurrent applications in two different ways: 1. Join's primitives are integrated into languages by changing compilers; such as JoCaml, Cw and Join.Java. 2. Join's primitives are added as a separate library; such as CCR, C#.Joins. The "Join" library i am proposing falls into this category.
It is intended for implementing real concurrent applications. Its simple model of async / synch methods and join-patterns (chords) should give us guideline for concurrent design and help avoiding pitfalls.
Sounds like something I should look into. I have used SDL for some telephony projects in the past and found it useful but as far as I'm aware it does not exist on any mathematical foundation. Could join calculus compete with something like SDL? Is there a join-calculus- for-programmers reference? Cheers.

Could join calculus compete with something like SDL? Is there a join-calculus- for-programmers reference?
Hello, You can find tutorial and references of join-calculus at its website : http://join.inria.fr/ I am not familiar with using Join in formal verification and model checking. Maybe you can find more by talking to people behind join-calculus: http://moscova.inria.fr/
From my prior exposure to Promela/Spin in a telecom switch project, it seems Spin has more tools for model extraction and verification (i guess it is same with SDL). Promela/Spin is more CSP-like while Join is a variant of async Pi calculus.
Regards, yigong
participants (6)
-
Giovanni Piero Deretta
-
Janek Kozicki
-
Mathias Gaunard
-
Scott Woods
-
Shams
-
Yigong Liu