[asio] tcp non blocking sync read query (bug?)
Hi, I'm sending data from one process to another using a tcp connection. Sender use scatter-gather to send the data using 3 buffers (total just over 2000 bytes). The reader uses 2 separate sync reads to read the data (using the asio free function read()) using a non blocking socket. Sometimes everything is fine, the 2 reads return the amount of data requested, but sometimes I see strange behaviour. I'm trying to work out whether I am doing something wrong, or whether I've stumbled across a bug. When it goes wrong for me, the second read always returns less data than I requested, along with the error code 'would_block'. I then loop, performing further reads (using the same buffer as before), eventually (one read, or multiple reads later) the read function returns the remainder of the data, but still with the error code 'would_block' set. However, when I check the contents of the buffer, it is not what I expected. I think, but haven't been able to confirm, that the start of the buffer is not the start of the data. 1. Is it expected that the read() free function, when the socket is non blocking, can return less data than requested, along with the error code 'would_block'? 2. If it is expected, does the buffer passed in subsequent reads have to be adjusted to read only the remaining about of data? 3. If the buffer does have to be changed to only read what is left, how does this work with scatter-gather reads using multiple buffers? As a little bit of extra info, I confirmed using tcpdump that the correct data is sent/received, although it arrives in 2 chunks (MTU size of 1500). This is on Linux using Boost 1.73.0. Cheers, Keith
On 11/06/2020 03:14, Keith Robinson wrote:
I’m sending data from one process to another using a tcp connection. Sender use scatter-gather to send the data using 3 buffers (total just over 2000 bytes). The reader uses 2 separate sync reads to read the data (using the asio free function read()) using a non blocking socket.
If you can, I recommend using asynchronous reads on a blocking socket instead. It's a little less weird that way since there isn't an "error" that isn't an error.
When it goes wrong for me, the second read always returns less data than I requested, along with the error code ‘would_block’. I then loop, performing further reads (using the same buffer as before), eventually (one read, or multiple reads later) the read function returns the remainder of the data, but still with the error code ‘would_block’ set. However, when I check the contents of the buffer, it is not what I expected. I think, but haven’t been able to confirm, that the start of the buffer is not the start of the data.
1. Is it expected that the read() free function, when the socket is non blocking, can return less data than requested, along with the error code ‘would_block’?
Yes, that just means that it has returned as much data as is currently available, and the rest has not yet been received yet. (Thus the error means it cannot return more data without blocking.) It may depend on which transfer-stop function you've specified, though; with blocking sockets (sync or async) you can tell it to not return until it has read all of the data, even if that takes multiple internal read operations, or to return as soon as it has at least some data. I'm not sure if non-blocking sockets behave differently or not in this regard, but it wouldn't surprise me. Non-blocking sockets aren't fashionable these days.
2. If it is expected, does the buffer passed in subsequent reads have to be adjusted to read only the remaining about of data?
The data that was returned from the first read is entirely valid and present in the buffer. It is your responsibility to either copy this elsewhere before your subsequent read, or pass a start-of-buffer pointer in your subsequent read starting at the point just after the already-read data, so that it doesn't get overwritten. Otherwise you will lose data. ASIO will never "repeat" data that has already been read (unless the originating end sends it multiple times, of course).
If you can, I recommend using asynchronous reads on a blocking socket instead. It's a little less weird that way since there isn't an "error" that isn't an error.
I do actually have a version of the code that uses asynchronous blocking reads, but the problem I found is how to deal correctly with timeout i.e. data not being received within a specified time limit. My current solution (using asynchronous blocking read) is a little too complex (and therefore potentially error prone) for my liking. I couldn't think of an elegant solution to timeout, that didn't involve closing the socket. I need the connection to be long lived, even after timeout. Given that, I started looking at non blocking sync read to cope with timeout. Knocking something together quickly has massively simplified the code compared to the async blocking read.
Yes, that just means that it has returned as much data as is currently available, and the rest has not yet been received yet. (Thus the error means it cannot return more data without blocking.)
I suspected as much, but the documentation wasn't entirely clear in this use case.
The data that was returned from the first read is entirely valid and present in the buffer. It is your responsibility to either copy this elsewhere before your subsequent read, or pass a start-of-buffer pointer in your subsequent read starting at the point just after the already-read data, so that it doesn't get overwritten. Otherwise you will lose data.
Further testing after sending my original email query, confirmed what you say. My problems was because I wasn't doing this right! Does make a scatter-gather style read (with multiple buffers) a little more complicated though. Not a big deal though for what I'm doing. I just had a play with trying to get what I wanted using a CompletionCondition, but my first attempt failed miserably but for now it doesn't matter. Thank you for your help. Keith
Hi,
do actually have a version of the code that uses asynchronous blocking reads, but the problem I found is how to deal correctly with timeout i.e. data not being received within a specified time limit. My current solution (using asynchronous blocking read) is a little too complex (and therefore potentially error prone) for my liking. I couldn't think of an elegant solution to timeout, that didn't involve closing the socket. I need the connection to be long lived, even after timeout.
You probably will have the same problem with sync APIO calls on async sockets. If your timer expires, you can have read a partial message received by the peer. Unless the peer violates your protocol, it will send a complete message,, and some bits are still waiting in the socket to be received later. You usually cannot start reading the next request, when you are not on a request boundary of your message stream. You either have to re-sync or discard the connefction by closing the socket. My solution to this kind of problem is to not cancel reading the message. Unless the peer violates your protocol, either the message sent by the peer will arrive (at some future time) or an error will be signalled. If the message arrives after the timeout has been reached, and no one is ready to handle incoming messages, I just discard it. I often use boost::asio::[async_]read_until() with a boost::asio::streambuf. I usually use a predicate, that signals when the entire message has been received. With the4 async variant, the last action in the event handler is to re-start boost::asio::async_read_until(). 73, Mario -- Mario Klebsch Actia I+ME GmbH Mario.klebsch@ime-actia.de Dresdenstrasse 17/18 Fon: +49 531 38 701 716 38124 Braunschweig Fax: +49 531 38 701 88 German
participants (3)
-
Gavin Lambert -
Keith Robinson -
Klebsch, Mario