[zlib] Compression intended use
Hello, This is probably out of scope with boost's intended usage pattern, but I'll ask anyhow. I've got a package of output files that I want to bundle, tarball, gzip, or however I can manage to compress them. I see there is the zlib usage, but methinks that is more for in-memory compression. Not sure exactly how that translates to folders-/files-oriented compression I am thinking of. Thank you... Regards, Michael Powell
On 8/10/13 7:15 AM, Michael Powell wrote:
Hello,
This is probably out of scope with boost's intended usage pattern, but I'll ask anyhow.
I've got a package of output files that I want to bundle, tarball, gzip, or however I can manage to compress them.
I see there is the zlib usage, but methinks that is more for in-memory compression. Not sure exactly how that translates to folders-/files-oriented compression I am thinking of.
See the iostreams lib's zlib_[de]compressor and filtering streams and streambufs. Also there was a very recent posting with the tag [iostreams] in the subject describing a usage pattern that may be of interest. Jeff
Michael Powell
I've got a package of output files that I want to bundle, tarball, gzip, or however I can manage to compress them.
You're also welcome to use some code I put together to create a Zip64 archive on the fly: http://ajf-prog.blogspot.com/2013/07/streaming-zip64-archive.html Full implementation here (relies on zlib, obviously): https://github.com/tkil/ajf-prog/tree/master/Zip64Streamer This is used on a data recorder, which is an "embedded" target (albeit one that would have made a pretty nice desktop 10-15 years ago). To reduce complexity on the instrument, the code does not try to optimize the output form at all -- it always uses the 64-bit extensions (which are rarely needed by this instrument, but I can't guarantee that it won't have to transfer a file larger than 4GiB). As the name implies, it is also designed for streaming -- it does no seeking, and can write directly out to a socket if you like. In particular, it allows me to allocate a buffer once, fill it with compressed data, then pass that buffer off to the transmission machinery; no copying necessary. Output has been tested with Info-Zip on Linux; Windows Explorer and 7z on Windows 7. Apparently even the 10.8 Finder doesn't support Zip64 files, so it doesn't work there (but you can build/get Info-Zip for that platform.) It turns out that info-zip's "zip" code also has a library mode, and also has a streaming mode, although neither is particularly well- documented. Unfortunately, while the authors were sympathetic and helpful, they could not make any promises about thread safety. Hope this helps. Good luck, Tony
participants (3)
-
Anthony Foiani -
Jeff Flinn -
Michael Powell