Michał Nowotka wrote:
IOStream's regex_filter loads the whole file in memory befory applying the regex on it.
No, it doesn't.
Yes it does. Or I misread the docs and code :)
basic_regex_filter is an aggregate_filter. From IOStreams docs on aggregate_filter:
http://www.boost.org/doc/libs/1_40_0/libs/iostreams/doc/classes/aggregate.html>
The class template aggregate_filter is a DualUseFilter for use as a base class by Filters which filter an entire character sequence at once. Because a aggregate_filter must read an entire character sequence into memory before it begins to filter,[...]
</quote>
As I said before, when you compile my code and use it against very large file in which all matching lines lay near the end then the output file will be empty. It can't load whole file because streams use buffers.
That surprises me. Looking at the implementation of aggregate_filter (http://www.boost.org/doc/libs/1_40_0/boost/iostreams/filter/aggregate.hpp), I don't see any provisions for out of memory conditions, so I would have expected an attempt to read a several gigabytes file to result in a std::bad_alloc exception.