George M. Garner Jr. wrote:
Johnathan,
I'd appreciate it if you would comment out the declaration and implementation of xsgetn and see if you still experience problems.
It makes the problem worse. Then I get reads for a single character from std::streambuf<>::uflow():
virtual int_type uflow() { // get a character from stream, point past it return (_Traits::eq_int_type(_Traits::eof(), underflow()) ? _Traits::eof() : _Traits::to_int_type(*_Gninc())); }
underflow or uflow should only be called when the input buffer is empty. The only effect of removing xsgetn should be that the default xsgetn kicks in. The default reads up to n characters "as if by repeated calls to sbumpc()," which means it reads from the buffer, filling it as necessary using uflow.
Fortunately, it is relatively trivial to disable Iostreams buffering altogether and write a buffering_shim_filter.
You can simply set the buffer size to zero when you add a filter to a chain.
Actually not! I was much chagrinned to find that you cannot disable input buffering. Setting the input buffer and pback buffers to 0 actually results in an input buffer size of either 4 or 1, depending on whether or not I disable your STLPort workaround.
Unfortunately I'm not sure it's really a "workaround." As far as I can tell, STLPort's implementation is conforming, which means the buffer-sizing policy is necessary to handle an arbitrary conforming standard library.
:-( I do think it should be possible for someone not using STLPort to disable input buffering.
The stream buffers in a chain provide the guarantee that a single character can always be put back, so a buffer size of at least one is necessary. I guess I could relax this requirment for stream buffers which are not part of a chain, but I'd have to be convinced that maintaining a putback buffer is sometimes a performance bottleneck.
The following modified code appears to work:
Thanks for the code. Unfortunately I can't test it now, since I'm in the middle of adding support for non-blocking i/o and the codebase is in flux. Would it be fair to characterize you code as follows? 1. undeflow and xsgetn both contain speicial code to handle the unbuffered case 2. xsgetn fills the request by invoking the underlying source only if n is big; otherwise it fills the request from the buffer, calling overflow as necessary. Regarding 1, I'd like to handle the unbuffered case by writing a separate stream buffer, unbuffered_streambuf, which combines the unbuffered sections from the various indirect_streambuf virtual functions. This would offer a tiny performance advantage, because there would be no check for the unbuffered case while performing i/o, but mostly it would make the code easier to read. Unfortunately, the necessity to deal with STLPort-type implementations makes this impractical. If I could be convinced that STLPort's behavior is non-conforming, I might do this. That still leaves the question whether the "unbuffered" streambuf should have a putback buffer of size 1, as is usual. Regarding 2, your implementation of xsgetn is essentially the same as a quality default implementation, except for the optimization for very large n. In the absence of data showing the large n optimization is significant, I'd prefer to use the default implementation to spare code size. I'm interested to know whether I've understood your code correctly, whether you think always providing a putback buffer of size is unreasonable, and whether you think the optimization for large n is critical. Best Regards, Jonathan