Hi Gavin,
The "problem" with the protocol I need to decode (which I have no choice to change) is that the data stream possibly contains escaped characters, so there is no way to know the read data length at the socket layer - its up to the filtering_stream layers to request data from the device layer until a complete packet is decoded.
This all sounds good in theory, but in practice the filter layers attempt to read in blocks, and the read size is often larger than the read data that the device layer will supply. This lead to higher-level layers blocking in read(). I figured (perhaps incorrectly) that I could deal with this using non-blocking socket reads.
I can't really answer your specific questions about the Boost implementations
No problem, I appreciate you taking time to respond.
but in general sockets (both blocking and non-blocking) and code dealing with them are expecting that read() will only block (or return WOULD_BLOCK) if no data can be read -- if at least one byte of data is available then that is what it will return, regardless of the amount actually requested. (The read size acts only as a maximum.)
Yep, understood. In theory the filtering_stream 'filter' and 'device' layers should pass characters, EOF, and WOULD_BLOCK. Unfortunately the code snippet I posted suppresses the propagation of the WOULD_BLOCK return value from the 'device' layer to the 'filter' layer.
Serial ports in particular sometimes operate this way and sometimes don't. Under Windows, the SetCommTimeouts API function selects (via ReadIntervalTimeout and ReadTotalTimeoutConstant) whether normal serial ports will return "early" as above or whether it will wait longer to see if more data is received, and whether there's an overall timeout or if it will block forever. There may be a similar API call you need to make to the FTDI library.
Ok, this is good to know, thanks. I consider solving this particular "problem" a good way to learn Boost. Without a "problem" to solve, reading code gets old quickly :) The Boost chat client/server has similar features to my problem, i.e., it involves encoding/decoding messages. The encoding/decoding is different than my case, since the chat protocol adds a header with the message length. This simplifies the socket read code, since you can read the fixed-length header, then read the now known) message length, i.e., each call to read has a fixed length parameter. In my case, the message length is unknown, and depends on the content of the message (since data can be escaped). The data stream is generated by a field-programmable gate array (FPGA), and adding a buffer to determine the message length before sending the response would use too many resources, so the encoding protocol cannot easily be changed. I'm in the process of modifying the chat example to encode the messages with a start-of-packet [, end-of-packet ], and escape \ code (so that a [ or ] or \ in the message is encoded as \[ or \] or \\), i.e., Hello! ->(encodes to)-> [Hello!] [Hello!] ->(encodes to)-> [\[Hello!\]] The modified code will use async_read_until to parse the encoded data streams ... I'll probably have to use a regex or match procedure to deal with the fact that \] is not the end-of-packet. Once I get the packet parsing working, I'll see if I can still use the filtering_stream components I came up with for my application, but instead of having them operate on a socket or serial_port device, I'll just have them operate directly on the contents of the streambuf that I filled using async_read_until. Its possible that the async_read_until match condition could use the filtering_stream, i.e., successful decoding of a message results in a match. Cheers, Dave