A while ago, Maciej Gajewski started an effort to use Asio to observe a ZeroMQ socket [1]. Based on his code, I put together a thin library that wraps the implementation details. Since other have requested a working example, here it is now at github: https://github.com/mavam/bmq Any feedback is highly appreciated. A basic usage example looks like this: #include <boost/asio/io_service.hpp> #include <bmq.h> int main() { boost::asio::io_service io; bmq::context ctx(1); bmq::component c(ctx, io); auto sink = c.add_sink(ZMQ_PAIR, "inproc://x"); c.subscribe(sink, [](bmq::message&& msg) { std::string str(static_cast<char*>(msg.data()), msg.size()); std::cout << str << std::endl; }); auto source = c.add_source(ZMQ_PAIR, "inproc://x"); bmq::message msg(const_cast<char*>("foo"), 3, nullptr); source->send(msg); // Execute message handler. io.poll(); return 0; } At this point, the library is really thin and only provides very rudimentary wrapping of ZeroMQ. In the future, I plan to extend it in order to facilitate applications that build on message-passing. The dataflow abstraction I envision is essentially a directed graph, where nodes represent components and edges communication links. 0mq provides the edges and the Asio handles asynchronous event processing at the component level. The motiviation is that this model lends itself well for scaling out distributed systems over large clusters. Matthias [1] http://permalink.gmane.org/gmane.network.zeromq.devel/9538
Hi Matthias, I was actually looking for ASIO support in ZeroMQ. Thank you. But does your library provide what you say? I mean does it really use ZeroMQ on edges and use ASIO on a binary component level for distributed systems? I am looking for something like that and doing this myself would be the last option due to time constraints. Many thanks in advance if you continue to work on this :) -Asif On Wed, Feb 8, 2012 at 5:15 AM, Matthias Vallentin < vallentin@icsi.berkeley.edu> wrote:
A while ago, Maciej Gajewski started an effort to use Asio to observe a ZeroMQ socket [1]. Based on his code, I put together a thin library that wraps the implementation details. Since other have requested a working example, here it is now at github:
Any feedback is highly appreciated. A basic usage example looks like this:
#include <boost/asio/io_service.hpp> #include <bmq.h>
int main() { boost::asio::io_service io; bmq::context ctx(1); bmq::component c(ctx, io);
auto sink = c.add_sink(ZMQ_PAIR, "inproc://x"); c.subscribe(sink, [](bmq::message&& msg) { std::string str(static_cast<char*>(msg.data()), msg.size()); std::cout << str << std::endl; });
auto source = c.add_source(ZMQ_PAIR, "inproc://x");
bmq::message msg(const_cast<char*>("foo"), 3, nullptr); source->send(msg);
// Execute message handler. io.poll();
return 0; }
At this point, the library is really thin and only provides very rudimentary wrapping of ZeroMQ. In the future, I plan to extend it in order to facilitate applications that build on message-passing. The dataflow abstraction I envision is essentially a directed graph, where nodes represent components and edges communication links. 0mq provides the edges and the Asio handles asynchronous event processing at the component level. The motiviation is that this model lends itself well for scaling out distributed systems over large clusters.
Matthias
[1] http://permalink.gmane.org/gmane.network.zeromq.devel/9538 _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
But does your library provide what you say? I mean does it really use ZeroMQ on edges and use ASIO on a binary component level for distributed systems?
I'm sorry if I was not clear enough. At this point, the library (bmq) is really thin and basic. It is a proof-of-principle that you can use ZeroMQ and Asio together; bmq does not provide a lot abstractions beyond what ZeroMQ already offers. That is, it associates sockets with what I call components, i.e., logical units of structuring your applications. (If it helps, think of components as SEDA stages [1].) It also takes these ZeroMQ sockets and hands them to the asynchronous io_service proactor. I will use library in my own projects, so it will probably change quite a bit once I figure out what the right level of abstraction are. In the long run, I would like to have message-passing that allows me to create component-based software which I can easily distribute over multiple machines, where bottlneck components can be executed on separate nodes, or replicated to reduce the load on the edges. Hopefully this helps to better understand what I am after. Another aspect concern parallelism, both multi-core and across several machines. Regarding the former, Both Asio and ZeroMQ use threads (io_service/context). If messages cannot be sent fast enough, maybe it makes sense to add more threads to the ZeroMQ context. If messages cannot be consumed fast enough, it may make sense to launch more threads on io_service::run(). Perhaps one can create a monitoring mechanism that finds out these things automatically. Again, all this is just a vision. Matthias [1] http://matt-welsh.blogspot.com/2010/07/retrospective-on-seda.html
I will use library in my own projects, so it will probably change quite a bit once I figure out what the right level of abstraction are. In the long run, I would like to have message-passing that allows me to create component-based software which I can easily distribute over multiple machines, where bottlneck components can be executed on separate nodes, or replicated to reduce the load on the edges. Hopefully this helps to better understand what I am after.
You might want to look at Zero for similar patterns. It isn't Boost related, but is something worth looking at. http://www.zeroc.com/icecpp.html No affiliation, just a user. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Hi, Matthias 2012/2/8 Matthias Vallentin <vallentin@icsi.berkeley.edu>:
A while ago, Maciej Gajewski started an effort to use Asio to observe a ZeroMQ socket [1]. Based on his code, I put together a thin library that wraps the implementation details. Since other have requested a working example, here it is now at github:
Any feedback is highly appreciated. A basic usage example looks like this:
#include <boost/asio/io_service.hpp> #include <bmq.h>
int main() { boost::asio::io_service io; bmq::context ctx(1); bmq::component c(ctx, io);
auto sink = c.add_sink(ZMQ_PAIR, "inproc://x"); c.subscribe(sink, [](bmq::message&& msg) { std::string str(static_cast<char*>(msg.data()), msg.size()); std::cout << str << std::endl; });
auto source = c.add_source(ZMQ_PAIR, "inproc://x");
bmq::message msg(const_cast<char*>("foo"), 3, nullptr); source->send(msg);
// Execute message handler. io.poll();
return 0; }
... I cannot find this library. Did move repository?
======================== Akira Takahashi mailto:faithandbrave@gmail.com site: https://sites.google.com/site/faithandbrave/about/en
participants (4)
-
Akira Takahashi -
asif saeed -
Matthias Vallentin -
Raymond Burkholder