[iostreams] 1.41 gzip filter regression
Hi guys, In 1.41 we are experiencing regression in boost::iostreams::gzip_decompressor. Almost 50% percents of our test *.gz files failed to decompress. Following code snippet usually throws gzip error: std::ifstream f(n, std::ios_base::in | std::ios_base::binary); if(!f) { std::cerr<<"cant' open "<<n<<'\n'; } namespace io = boost::iostreams; io::filtering_istream in; in.push(io::gzip_decompressor() ); in.push(f); std::istreambuf_iterator<char> b(in); std::istreambuf_iterator<char> e; std::cout<<n<<'\n'; std::copy(b, e, std::ostream_iterator<char>(std::cout) ); Maybe that is caused by incorrect api usage. In this case, sorry for the noise in advance. I've attached minimal example that reproduces the issue. Build showfile executable and run ./showfile myfile.odt.bz2.gz > result. You should have 'error: gzip error' on stderr. Regards
For what it's worth, I seem to have encountered the same bug with Boost 1.41. Strangely enough, our gzip unit tests work fine on OSX with 1.41, but fail on CentOS 5 with "gzip errors." Same tests work fine on all platforms with 1.37 through 1.40. I mention this in case it might have something to do with the version of zlib that ships with CentOS 5? Alexander, which platform are you using? Take care, -Mike On 12/2/09 1:59 AM, Alexander Arhipenko wrote: Hi guys, In 1.41 we are experiencing regression in boost::iostreams::gzip_decompressor. Almost 50% percents of our test *.gz files failed to decompress. Following code snippet usually throws gzip error: std::ifstream f(n, std::ios_base::in | std::ios_base::binary); if(!f) { std::cerr<<"cant' open "<<n<<'\n'; } namespace io = boost::iostreams; io::filtering_istream in; in.push(io::gzip_decompressor() ); in.push(f); std::istreambuf_iterator<char> b(in); std::istreambuf_iterator<char> e; std::cout<<n<<'\n'; std::copy(b, e, std::ostream_iterator<char>(std::cout) ); Maybe that is caused by incorrect api usage. In this case, sorry for the noise in advance. I've attached minimal example that reproduces the issue. Build showfile executable and run ./showfile myfile.odt.bz2.gz > result. You should have 'error: gzip error' on stderr. Regards _______________________________________________ Unsubscribe & other changes: [1]http://lists.boost.org/mailman/listinfo.cgi/boo st References 1. http://lists.boost.org/mailman/listinfo.cgi/boost
On Fri, Dec 4, 2009 at 2:22 AM, Michael Dickey <mike@mikedickey.com> wrote:
For what it's worth, I seem to have encountered the same bug with Boost 1.41. Strangely enough, our gzip unit tests work fine on OSX with 1.41, but fail on CentOS 5 with "gzip errors." Same tests work fine on all platforms with 1.37 through 1.40. I mention this in case it might have something to do with the version of zlib that ships with CentOS 5? Alexander, which platform are you using? Take care, -Mike On 12/2/09 1:59 AM, Alexander Arhipenko wrote:
Hi, Michael. The issue was reproduced on RHEL 5 x86_64, gcc 4.1.2. zlib version is 1.2.3. (I think it's the latest). Should note that msvc9 + zlib1.2.3 built from sources seems to be ok. Regards
Not sure if this is the same issue. My student found an issue with x64 bit CRC calcualations when the file length was sufficient that the value needed to roll over. I believe that the patch has been applied to trunk. -----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Alexander Arhipenko Sent: Friday, December 04, 2009 12:15 AM To: boost@lists.boost.org Subject: Re: [boost] [iostreams] 1.41 gzip filter regression On Fri, Dec 4, 2009 at 2:22 AM, Michael Dickey <mike@mikedickey.com> wrote:
For what it's worth, I seem to have encountered the same bug with Boost 1.41. Strangely enough, our gzip unit tests work fine on OSX with 1.41, but fail on CentOS 5 with "gzip errors." Same tests work fine on all platforms with 1.37 through 1.40. I mention this in case it might have something to do with the version of zlib that ships with CentOS 5? Alexander, which platform are you using? Take care, -Mike On 12/2/09 1:59 AM, Alexander Arhipenko wrote:
Hi, Michael. The issue was reproduced on RHEL 5 x86_64, gcc 4.1.2. zlib version is 1.2.3. (I think it's the latest). Should note that msvc9 + zlib1.2.3 built from sources seems to be ok. Regards _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
In my case, the iostreams gzip bugs have prevented me from upgrading to 1.41. It seems that in most releases I encounter gzip bugs and give me a bad feeling about the iostreams library. Probably the library doesn't have a maintainer but I see in the 1.41 release notes that new features were added. I addition to what has been reported I get this error: /home/code/boost_1_41_0/boost/iostreams/read.hpp:191: error: 'class boost::iostreams::detail::direct_adapter<boost::iostreams::basic_array_source<char>
' has no member named 'putback'
On Fri, Dec 4, 2009 at 6:20 PM, Schrom, Brian T <brian.schrom@pnl.gov> wrote:
Not sure if this is the same issue. My student found an issue with x64 bit CRC calcualations when the file length was sufficient that the value needed to roll over. I believe that the patch has been applied to trunk.
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Alexander Arhipenko Sent: Friday, December 04, 2009 12:15 AM To: boost@lists.boost.org Subject: Re: [boost] [iostreams] 1.41 gzip filter regression
On Fri, Dec 4, 2009 at 2:22 AM, Michael Dickey <mike@mikedickey.com> wrote:
For what it's worth, I seem to have encountered the same bug with Boost 1.41. Strangely enough, our gzip unit tests work fine on OSX with 1.41, but fail on CentOS 5 with "gzip errors." Same tests work fine on all platforms with 1.37 through 1.40. I mention this in case it might have something to do with the version of zlib that ships with CentOS 5? Alexander, which platform are you using? Take care, -Mike On 12/2/09 1:59 AM, Alexander Arhipenko wrote:
Hi, Michael. The issue was reproduced on RHEL 5 x86_64, gcc 4.1.2. zlib version is 1.2.3. (I think it's the latest). Should note that msvc9 + zlib1.2.3 built from sources seems to be ok.
Regards _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Sat, Dec 5, 2009 at 1:17 PM, Jose <jmalv04@gmail.com> wrote:
In my case, the iostreams gzip bugs have prevented me from upgrading to 1.41. It seems that in most releases I encounter gzip bugs and give me a bad feeling about the iostreams library. Probably the library doesn't have a maintainer but I see in the 1.41 release notes that new features were added.
I addition to what has been reported I get this error:
/home/code/boost_1_41_0/boost/iostreams/read.hpp:191: error: 'class boost::iostreams::detail::direct_adapter<boost::iostreams::basic_array_source<char>
' has no member named 'putback'
Hi, Jose
It seems that in most releases I encounter gzip bugs [snip]
What bugs are you talking about? We use iostream's gzip functionality starting from 1.35 and it was the first time I've encountered an issue, and this issue has trivial resolution (see my previous email). Regards
On Fri, Dec 4, 2009 at 7:20 PM, Schrom, Brian T <brian.schrom@pnl.gov> wrote:
Not sure if this is the same issue. My student found an issue with x64 bit CRC calcualations when the file length was sufficient that the value needed to roll over. I believe that the patch has been applied to trunk.
Hi, there. Found issue was really related to CRC check on x86_64 platform. The related ticket is #3352 (https://svn.boost.org/trac/boost/ticket/3352) and changeset that resolves the issue is https://svn.boost.org/trac/boost/changeset/57610. After applying this change to 1.41, the regression disappeared. Thanks everybody Regards
My main issue has been ticket 2415, and I'll track down the new issue to submit a ticket 3352 - gzip crc decompression error on x64 2557 - gzip with filtering_stream infinite loop 2415 - gzip_decompressor does not work with filtering_ostream 1896 - gzip_decompressor ignores multiple members 1579 - gzip does not model DualUseFilter https://svn.boost.org/trac/boost/search?q=gzip&noquickjump=1&ticket=on On Sat, Dec 5, 2009 at 12:25 PM, Alexander Arhipenko <arhipjan@gmail.com> wrote:
On Fri, Dec 4, 2009 at 7:20 PM, Schrom, Brian T <brian.schrom@pnl.gov> wrote:
Not sure if this is the same issue. My student found an issue with x64 bit CRC calcualations when the file length was sufficient that the value needed to roll over. I believe that the patch has been applied to trunk.
Hi, there. Found issue was really related to CRC check on x86_64 platform. The related ticket is #3352 (https://svn.boost.org/trac/boost/ticket/3352) and changeset that resolves the issue is https://svn.boost.org/trac/boost/changeset/57610. After applying this change to 1.41, the regression disappeared.
Thanks everybody
Regards _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
This function used to work fine in previous releases but doesn't now: std::string decompress_gzip(const std::string& in) { std::string res; bio::array_source src(in.data(),in.length()); bio::copy(bio::compose(bio::gzip_decompressor(), src), bio::back_inserter(res)); return res; } I get this compile error: home/code/boost_1_41_0/boost/iostreams/read.hpp:191: error: 'class boost::iostreams::detail::direct_adapter<boost::iostreams::basic_array_source<char>
' has no member named 'putback'
On Sat, Dec 5, 2009 at 12:34 PM, Jose <jmalv04@gmail.com> wrote:
My main issue has been ticket 2415, and I'll track down the new issue to submit a ticket
3352 - gzip crc decompression error on x64 2557 - gzip with filtering_stream infinite loop 2415 - gzip_decompressor does not work with filtering_ostream 1896 - gzip_decompressor ignores multiple members 1579 - gzip does not model DualUseFilter https://svn.boost.org/trac/boost/search?q=gzip&noquickjump=1&ticket=on
On Sat, Dec 5, 2009 at 12:25 PM, Alexander Arhipenko <arhipjan@gmail.com> wrote:
On Fri, Dec 4, 2009 at 7:20 PM, Schrom, Brian T <brian.schrom@pnl.gov> wrote:
Not sure if this is the same issue. My student found an issue with x64 bit CRC calcualations when the file length was sufficient that the value needed to roll over. I believe that the patch has been applied to trunk.
Hi, there. Found issue was really related to CRC check on x86_64 platform. The related ticket is #3352 (https://svn.boost.org/trac/boost/ticket/3352) and changeset that resolves the issue is https://svn.boost.org/trac/boost/changeset/57610. After applying this change to 1.41, the regression disappeared.
Thanks everybody
Regards _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
participants (4)
-
Alexander Arhipenko -
Jose -
Michael Dickey -
Schrom, Brian T