I'm looking at adopting boost::iostreams for reading/writing compressed streams. I'm testing with textual files commonly on the order of a couple of GB uncompressed, and a couple hundred MB compressed. I noticed during some initial testing that using a gzip_compressor to write a compressed stream through boost::iostreams results in significant memory usage, I'm seeing hundreds of MB per compressed stream. It looks like internally the gzip_compressor must be doing some GC-type behavior, because I see its memory usage increase to just under 1GB per stream with a gzip_compressor, and then decrease again.

Is there any way to control the memory that gzip_compressor is using? I've tried both increasing and decreasing the buffer size when gzip_compressor is constructed, but that buffer is tiny and cannot be what's driving the increased memory usage.

Thanks,

Alex