Re: [Boost-users] Buffering and ASIO

Good day. I'm trying to read video data transferred by HTTP. I use the following function in order to read HTTP header: boost::asio::async_read_until(m_socket, m_headerBuffer, "\r\n\r\n", boost::bind(&CAXISParser::read_header, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
, where m_headerBuffer is asio::streambuf object. During header parsing i can get video sample length. Now I want to read all video data in one function call, for example:
boost::asio::async_read(m_socket, boost::asio::buffer(pSample->GetBody(), m_currentSampleLength), boost::bind(&CAXISParser::read_body, this, pSample, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
, where pSample is object with buffer for video data. There is problem that async_read_until captures > some data from video data, and that call's behavior is incorrect. How can i fight that problem? For example, how can i figure out number of bytes left in streambuf and copy them to my buffer, and then read rest with async_read? I'm will very appreciate for any advices.
It's not clear to me exactly what you're asking here. From what I can tell, you have created a buffer of size m_currentSampleLength, and asked ASIO to call CAXISParser::read_body when it has received this many bytes. Is that what you intended to do? Is that what boost is doing? Or is the problem something else?
-----Scott.
Hello, Scott.
Sorry for bad english, i'm will try explain.
Yep, there is problem. When i'm read HTTP header with acync_read_until, it
consume header and some bytes from video data (i think). After i'm
calculate video data length from header and trying read data with
async_read. But it works incorrect as previous call async_read_until
captures some bytes from video data and video data readed with async_read
have wrong offset.
How can i understand, the problem that async_read_until and async_read
used different buffers for reading (asio::streambuf and custom buffer),
and some video data consumed by streambuf.
Question: how to get video data captured with async_read_until and read
rest from tcp buffer?There is is code sample:void read_header(PSample pSample, const boost::system::error_code& error, size_t bytes_transferred){ // Calculate data length with regex boost::asio::const_buffer data = m_headerBuffer.data(); const char* beg = boost::asio::buffer_cast

IIUC, you're trying to mix and match calls to async_read_until (with a streambuf) and calls to async_read (with a buffer parameter), but having problems with "extra data past the length data". If you keep the same design, you will need to first extract the "extra data" out of the streambuf, place it in to the buffer you want to use, then call async_read or async_read_at with the partially filled buffer. While I haven't done anything similar, I can't think of a reason this won't work. However, I think it would be easier to use async_read_until calls only, and keep all data in the streambuf. Create a match_condition function object, track the state of the HTTP transfer in the function object, and then return a true condition when the full buffer has been received. This encapsulates all of your "buffer filling logic" into one small place and I think is the simplest code of everything I'm writing in this e-mail. If you cannot use a streambuf for the full transfer for whatever reason, then I would not use async_read_until and instead use async_read or async_read_at with a single buffer for everything. It shouldn't be hard to manage the "two states" of the transfer ("need length header", "have length header, getting rest of data"). Your handler will be called multiple times (most likely) while in the second state (and it's possible it will get called multiple times while in the first state, depending on how congested the TCP incoming buffers are). I think your design of trying to mix the two models is more complicated than it needs to be. Cliff

Good day. I'm developing an application that transfer video data and there is problem with transfer big data chunk (~600 Kb) over UDP. Data processing consumes 50% CPU and performance is not good ( i need transfer 25 chunks in sec, but really there is about 18). Below is code snippet: ....... // Get video data chunk void Receive(ISample* pSample) { // Divide chunk on RTP packets IPacketIterator* iter = m_accessor->GetIterator(pSample); SendDatagram(iter); } // Send datagram void SendDatagram(IPacketIterator* iter) { if (iter->IsValid()) { IProtocolPacket* packet = iter->Next(); std::vectorasio::const_buffer buffs(2); buffs.push_back(asio::const_buffer(packet->GetHeader(), packet->GetHeaderSize()); buffs.push_back(asio::const_buffer(packet->GetBody(), packet->GetBodySize()); m_socket.async_send_to(buffs, m_endpoint, boost::bind(&CChannel::handle_send, this, iter, placeholders::error, placeholders::bytes_transferred); } else { // Query next chunk ........ } } void handle_send(IPacketIterator* iter, const boost::system::error_code& error, size_t bytes_trans) { if (!error) SendDatagram(iter); } Anyone can help me with that problem? Best regards, Vadim

Try the following to see if the (probably small) reduction in memory allocation / deallocation helps (I really doubt it will help much - I think there's something else that is causing problems, since you're seeing some major performance problems that probably have nothing to do with the code sample you posted):
Change:
IProtocolPacket* packet = iter->Next();
std::vectorasio::const_buffer buffs(2);
buffs.push_back(asio::const_buffer(packet->GetHeader(), packet->GetHeaderSize());
buffs.push_back(asio::const_buffer(packet->GetBody(), packet->GetBodySize());
To:
IProtocolPacket* packet = iter->Next();
boost::array

Hello Vadim,
This is a piece of code from an application that inadvertently does what your application appears to be doing: receives a stream of images over a HTTP connection.
We had the same problems and you have in bold the solution we found: we call in_avail() on the read-buffer of the stream that tells us the amount of data already read but left in the buffer and then we readsome to get the actual data.
void onConnect(boost::system::error_code ErrorCode,unsigned int fps)
{
boost::asio::streambuf response;
boost::asio::read_until(*Socket, response, "\r\n");
std::istream response_stream(&response);
response_stream >> http_version;
response_stream >> status_code;
std::getline(response_stream, status_message);
if (!response_stream || http_version.substr(0, 5) != "HTTP/")
{
THROW_EXCEPTION(logic_error("Invalid HTTP response"));
}
// Read the response headers, which are terminated by a blank line.
boost::asio::read_until(*Socket, response, "\r\n\r\n");
while (std::getline(response_stream, header) && header != "\r")
{
LOG_DEBUG(log,"Got http header: "<
Good day. I'm trying to read video data transferred by HTTP. I use the following function in order to read HTTP header: [sip]
participants (3)
-
Cliff Green
-
Emil Mieilica
-
Vadim Shmelev