asio UDP async receive question - how to do this properly?
Hi, I wonder what the proper pattern is to receive messages, store them, and process them in an async manner? What I'm trying to do, and which doesn't work: - receive the packets in an async call - copy the data over - start to receive again after copying - store the data using a producer - consumer pattern, using a mutex - process the data in another thread (which is the consumer in the pattern) but, this doesn't seem to work, as a lot of packets are lost. my assumption is that because the handler function blocks on the mutex, and returns 'late', packets received in the meantime are lost forever. but: how to do this properly? I have a small sample here: http://pastebin.com/NnbK1T7G , which is based on one of the asio examples. the sample code is quite simple: - it allows sync sending of messages (strings) - it allows async reception of such messages - it will put each received message into a vector, which is guarded by a mutex - another thread will process messages from this vector on a regular basis, which processing is also guarded by a mutex. if I compile & run the sample, I get: ./async_asio b sent 20000 messages a received 1981 messages which is a profound loss of packets. as seen in the sample, after the receive handler is called, the recieved message is copied over, and socket::async_receive_from() is called ASAP. I don't see what else would need to be done to have all messages received. If I remove the processing of messages in threa_func(), the results are markedly better: ./async_asio b sent 20000 messages a received 19963 messages increasing the sleeping time at the end does no good - it seems the messages were lost already. Akos
From: "Ákos Maróy"
b sent 20000 messages a received 1981 messages
Depending on a lot of factors, this is not surprising. If you're sending the datagrams faster than you can receive and process them, this will happen. I've found that with UDP it's difficult to get 0% loss, even with a controlled environment (internal network, no Internet or WAN hops) and a consistent flow of datagrams. Check the list archives for previous discussions on this topic (there's quite a few, although the discussions might be on the Asio Users list instead of Boost Users list). If you need reliable data transfer, you need to use another protocol (usually TCP). If you need "closer to 100%" you will need to do something like metering the sending side (more so than you've done in your code), or put in simple "detect loss and re-send" functionality (although the more sophisticated the "detect loss and re-send" the closer the logic becomes to TCP). All of the pieces in the network chain take advantage of UDP being an unreliable protocol and can drop packets on a whim. This includes the OS protocol stack, the NIC driver code and firmware, and switches and routers in the network.
If I remove the processing of messages in threa_func(), the results are markedly better:
./async_asio b sent 20000 messages a received 19963 messages
This loss rate (less than 1%) is closer to what I routinely see, on controlled (internal) networks that are nowhere close to being congested. It's also (informally) similar to numbers I've seen from others that have used UDP for various needs. I've seen messages from others that have achieved better loss rates, but it took some real work (tuning OS buffers, driver parameters, etc). This loss rate works well for many, many applications. I glanced briefly through your code and didn't see any obvious problems (it looks nicely written, as well). Cliff
Cliff,
Depending on a lot of factors, this is not surprising. If you're sending the datagrams faster than you can receive and process them, this will happen.
I've found that with UDP it's difficult to get 0% loss, even with a controlled environment (internal network, no Internet or WAN hops) and a consistent flow of datagrams. Check the list archives for previous discussions on this topic (there's quite a few, although the discussions might be on the Asio Users list instead of Boost Users list).
I see. I'll re-post my question there.
If you need reliable data transfer, you need to use another protocol (usually TCP). If you need "closer to 100%" you will need to do something like metering the sending side (more so than you've done in your code), or put in simple "detect loss and re-send" functionality (although the more sophisticated the "detect loss and re-send" the closer the logic becomes to TCP).
actually, I'm just implementing such protocol. but, for that, I need to be able to recieve & process UDP packets :)
All of the pieces in the network chain take advantage of UDP being an unreliable protocol and can drop packets on a whim. This includes the OS protocol stack, the NIC driver code and firmware, and switches and routers in the network.
but, my suggests that the network layer at least is able to receive these packages, see exactly below:
If I remove the processing of messages in threa_func(), the results are markedly better:
./async_asio b sent 20000 messages a received 19963 messages
but, if I add a mutex so as to store the UDP packets, then it seems the asio async receive handler won't get all the messages anymore. it might be that the recieved packets are dropped at the network layer, as some internal buffer is exhausted. or it might be, that the asio implementation just discards these?
This loss rate (less than 1%) is closer to what I routinely see, on controlled (internal) networks that are nowhere close to being congested. It's also (informally) similar to numbers I've seen from others that have used UDP for various needs. I've seen messages from others that have achieved better loss rates, but it took some real work (tuning OS buffers, driver parameters, etc).
This loss rate works well for many, many applications.
yes, I'd like that, but this scenario excludes the very important part of storing & processing the messages themselves :)
I glanced briefly through your code and didn't see any obvious problems (it looks nicely written, as well).
thank you. Akos
From: "Ákos Maróy"
... might be on the Asio Users list instead of Boost Users list). I see. I'll re-post my question there.
I just read your posting on the Asio Users list. Hopefully others can provide some additional comments that will help.
actually, I'm just implementing such protocol. but, for that, I need to be able to recieve & process UDP packets :)
Your code is doing that, correctly, although maybe not optimally in design. On re-reading your code, I'm wondering about sending and receiving in the same (main) thread (yes, I see the io_service is being run in a separate thread) - are all of the datagrams sent before the receiving object can process them? This would (obviously) cause some issues. I'll have to review Asio docs to see how the threading works in this case (the "b" and "a" server objects are created in the same (main) thread). If this turns out to be part (or all) of the problem, move either "a" or "b" to a separate thread. Also, consider that you're doing a lot more work when receiving the datagrams, than when you're sending them: -- Creating a std::string object from the char buf (creates a std::string, copies the data) -- Pushing a std::string object on to a vector - this will cause another std::string to be created and another buffer copy, as well as some (quite signficant) std::string copies when the internal buffer of the vector is filled up, and a reallocation occurs, causing all of the existing std::string objects to be copied again To optimize the receiving logic, consider some or all of the following: -- Use a std::list instead of std::vector to queue up the incoming data (this will eliminate the reallocation copying) -- Use a reference counted string and store them in the container (Chris created a nice shared_const_buffer class in some example code that could be used - I've taken that class, enhanced it, wrote some unit tests, and am using it extensively).
it might be that the recieved packets are dropped at the network layer, as some internal buffer is exhausted. or it might be, that the asio implementation just discards these?
I doubt that Asio performs any incoming UDP datagram buffering (you can look at the Asio code, to verify). It's almost surely the network layer that is discarding the datagrams, after the receiving buffer fills up. This is always going to be the case where datagrams are coming in faster than they can be processed. It's also a problem with TCP, and a classic problem is that an app is sending data through a TCP connection faster than the receiver can drain / process it. Eventually, the receiving buffer fills up, and TCP flow control kicks in. For blocking TCP I/O, the sending app will block until the receiving app catches up. With async I/O, the sending app completion handler won't be invoked until the data flow catches up (which is another reason why I like async networking much better than blocking I/O). Cliff
Cliff,
Your code is doing that, correctly, although maybe not optimally in design.
On re-reading your code, I'm wondering about sending and receiving in the same (main) thread (yes, I see the io_service is being run in a separate thread) - are all of the datagrams sent before the receiving object can process them? This would (obviously) cause some issues. I'll have to review Asio docs to see how the threading works in this case (the "b" and "a" server objects are created in the same (main) thread). If this turns out to be part (or all) of the problem, move either "a" or "b" to a separate thread.
actually, putting the 'a' and 'b' object execution into a separate thread solved the issue. excellent!
Also, consider that you're doing a lot more work when receiving the datagrams, than when you're sending them:
-- Creating a std::string object from the char buf (creates a std::string, copies the data) -- Pushing a std::string object on to a vector - this will cause another std::string to be created and another buffer copy, as well as some (quite signficant) std::string copies when the internal buffer of the vector is filled up, and a reallocation occurs, causing all of the existing std::string objects to be copied again
To optimize the receiving logic, consider some or all of the following:
-- Use a std::list instead of std::vector to queue up the incoming data (this will eliminate the reallocation copying) -- Use a reference counted string and store them in the container (Chris created a nice shared_const_buffer class in some example code that could be used - I've taken that class, enhanced it, wrote some unit tests, and am using it extensively).
thank you for these suggestions.
I doubt that Asio performs any incoming UDP datagram buffering (you can look at the Asio code, to verify). It's almost surely the network layer that is discarding the datagrams, after the receiving buffer fills up.
I see
This is always going to be the case where datagrams are coming in faster than they can be processed.
It's also a problem with TCP, and a classic problem is that an app is sending data through a TCP connection faster than the receiver can drain / process it. Eventually, the receiving buffer fills up, and TCP flow control kicks in. For blocking TCP I/O, the sending app will block until the receiving app catches up. With async I/O, the sending app completion handler won't be invoked until the data flow catches up (which is another reason why I like async networking much better than blocking I/O).
indeed... thanks again, Akos
participants (2)
-
Cliff Green -
Ákos Maróy