[ASIO]: Best solution for Publisher/Subscriber pattern
Hi All, currently we do some research in how we can do implement the classical Publisher/Subscriber pattern using ASIO. We had one thread that produces some kind of a message and like to send it out to 1 to n subscribers via TCP, where n is < 5. Message rate will be up to 800 Hz. My first idea is that the producer thread puts the message in a blocking queue. On the other end a consumer thread waits, takes the message and send it to the subscribers via synchronous or asynchronous send. One requirement is, when one connection blocks (because their client is buggy/hangs) it's ok to drop messages for that connection, but we should deliver to the other subscribers. It's also a one way communication from publisher to subscriber. Can someone provide some hints for best practices / design patterns / comments please? If I show up with some other lib like ZeroMQ my colleagues will fry me, so it's the easiest to use boost also for that task.. Best, Ingo
Pub / sub messaging is a broad area, so there’s quite a bit of possible discussion.
We had one thread that produces some kind of a message and like to send it out to 1 to n subscribers via TCP, where n is < 5. Keep in mind the overall architecture / design. If you are mapping a TCP connection to each subscriber, from each publisher, the number of connections is (N –1) ** 2, which (obviously) grows large very fast. In other words, a connection “mesh” is only good for small numbers of participants. If you have a daemon / broker in the middle, or are using some form of multicast or broadcast, then there are also growth issues, but different kinds (they definitely scale much better). Message rate will be up to 800 Hz. My first idea is that the producer thread puts the message in a blocking queue. On the other end a consumer thread waits, takes the message and send it to the subscribers via synchronous or asynchronous send. I’m not sure why you need a blocking queue. You could have outgoing queues and fire off async writes for all outgoing connections. In particular, queue up a ref counted buffer (which you usually need for Asio anyway), fire off async writes for all connections, then the entry is automatically removed when all writes complete. One requirement is, when one connection blocks (because their client is buggy/hangs) it’s ok to drop messages for that connection, but we should deliver to the other subscribers. If you make everything async, there are no blocking issues. However, you might have an issue of an outgoing queue filling up, since the receiver is slow (buggy / hangs), and you might want to consider callbacks or policies that handle this situation. Can someone provide some hints for best practices / design patterns / comments please?
You are just scratching the surface. I wrote a small pub / sub library designed for (mostly) static configs / connections, with “mesh” connections, with static (startup) topic discovery (currently no “dynamic” topic discovery), which is working well. But it’s a limited library, and has drawbacks for general use. I just mentioned “topics” – most pub / sub messaging has the concept of topics, which provide the “glue” between the publishers and subscribers. Check out the OMG DDS specs (http://en.wikipedia.org/wiki/Data_distribution_service), and you might want to also check out a proposed Boost library named Channel (http://channel.sourceforge.net/). There’s also many other possibilities in the pub / sub space. Cliff
Ingo Maindorfer wrote:
Hi All,
currently we do some research in how we can do implement the classical Publisher/Subscriber pattern using ASIO. We had one thread that produces some kind of a message and like to send it out to 1 to n subscribers via TCP, where n is < 5. Message rate will be up to 800 Hz. My first idea is that the producer thread puts the message in a blocking queue. On the other end a consumer thread waits, takes the message and send it to the subscribers via synchronous or asynchronous send. One requirement is, when one connection blocks (because their client is buggy/hangs) it's ok to drop messages for that connection, but we should deliver to the other subscribers. It's also a one way communication from publisher to subscriber.
Can someone provide some hints for best practices / design patterns / comments please?
Hmm - I don't know if it's best practice but what worked for me was to: a) The signals library implements the publish/subscribe pattern b) use ASIO to generate async events. c) use boost circular buffer and boost thread to create a "bounded buffer" - this is an example in circular buffer. d) use the bounded buffer to send events from the asio to signals OR use signals to send messages to bounded buffers corresponding to other tasks. Note that in this scenario, nothing is blocked more than the time it takes to append/remove a message from the bounded buffer. This seems a little non obvious, but once you spend some time with these three libraries, it all comes together in a very natural way. Robert Ramey
If I show up with some other lib like ZeroMQ my colleagues will fry me, so it's the easiest to use boost also for that task..
Best,
Ingo
participants (3)
-
Cliff Green -
Ingo Maindorfer -
Robert Ramey