[iostreams] array and gzip_decompressor
Recently I came back to a source written some time ago, that was running then but it is not compiling now. the failing snippet is: typedef boost::iostreams::composite<boost::iostreams::gzip_decompressor, boost::iostreams::array_source> array_gzip_decompressor; boost::iostreams::gzip_decompressor filter; boost::iostreams::array_source source(&_storage[0], _storage.size()); array_gzip_decompressor dev(filter,source); boost::iostreams::stream_buffer<array_gzip_decompressor> ibuf(dev); the error is:
boost\iostreams\read.hpp(191) : error C2039: 'putback' : is not a member of 'boost::iostreams::detail::direct_adapter<Direct>' with [ Direct=boost::iostreams::array_source ]
It seems to me that gzip_decompressor requires peekable source, but array_source is not peekable. Is there another way I should be creating gzipped stream into memory? Or maybe the arrays could be made peekable? Thanks in advance, Nikolay Mladenov
On 2/23/2010 10:48 PM, Nikolay Mladenov wrote:
Is there another way I should be creating gzipped stream into memory? Or maybe the arrays could be made peekable?
Perhaps you could read it into a stringstream. namespace io = boost::iostreams; io::filtering_istream in; in.push(io::gzip_decompressor()); in.push(io::file_source(fileName.c_str(), std::ios_base::in | std::ios_base::binary)); std::stringstream memStream; memStream << in.rdbuf();
On 2/23/2010 10:48 PM, Nikolay Mladenov wrote:
Is there another way I should be creating gzipped stream into memory? Or maybe the arrays could be made peekable?
Perhaps you could read it into a stringstream.
namespace io = boost::iostreams;
io::filtering_istream in; in.push(io::gzip_decompressor()); in.push(io::file_source(fileName.c_str(), std::ios_base::in | std::ios_base::binary)); std::stringstream memStream; memStream << in.rdbuf();
If you don't know before-hand how big the output can be, you can write into a std::vector relatively easily. The following creates an ovectorstream object: template<typename Ch> struct basic_vector_sink { typedef Ch char_type; typedef bio::sink_tag category; typedef std::vector<char_type> vector_type; basic_vector_sink( vector_type& vector ): _vector(vector) { } std::streamsize write( const char_type *buf, std::streamsize n ) { if ( n <= 0 ) return n; size_t s = _vector.size(); _vector.resize( s + n ); memcpy( (void*)((&_vector[0])+s), (const void*)buf, n*sizeof(char_type) ); return n; } vector_type& _vector; }; typedef basic_vector_sink<char> vector_sink; typedef bio::stream<vector_sink> ovectorstream; In my mind, something like arraystream (but for vectors) would be useful in boost::iostreams. The above is a pale imitation of the full functionality offered by an arraystream (bidirectional, seekable, etc), but it does the job for me. Cheers, Chris
Thank you eg, Chris, But I have probably misled you a bit with my post. The gzipped data is actually in memory (inside the storage variable), and I wanted to have a gzip_decompressing stream from that. for no I am using this patch Index: detail/adapter/direct_adapter.hpp =================================================================== --- detail/adapter/direct_adapter.hpp (revision 59992) +++ detail/adapter/direct_adapter.hpp (working copy) @@ -116,6 +116,7 @@ std::streamsize write(const char_type* s, std::streamsize n); std::streampos seek( stream_offset, BOOST_IOS::seekdir, BOOST_IOS::openmode = BOOST_IOS::in | BOOST_IOS::out ); + bool putback(const char_type c); void close(); void close(BOOST_IOS::openmode which); #ifndef BOOST_IOSTREAMS_NO_LOCALE @@ -204,6 +205,18 @@ } template<typename Direct> +inline bool direct_adapter<Direct>::putback + (const char_type c) +{ + pointers& get = ptrs_.first(); + if( get.ptr == get.beg ) + throw bad_putback(); + get.ptr -= 1; + *get.ptr = c; + return true; +} + +template<typename Direct> inline std::streamsize direct_adapter<Direct>::write (const char_type* s, std::streamsize n) { Nikolay Mladenov
Ahh... my bad for misreading ;) Rather than patching things this way, you could always use the array_source device defined in <boost/iostreams/device/array.hpp>. A very simple wrapper can also be written around a std::vector, or you can use the array_source directly on the internal data of the std::vector. You then use a filtered_source, and push back the gzip_decompressor followed by the array_source, like the earlier example in this thread. Cheers, Chris
Thank you eg, Chris,
But I have probably misled you a bit with my post. The gzipped data is actually in memory (inside the storage variable), and I wanted to have a gzip_decompressing stream from that.
for no I am using this patch
Index: detail/adapter/direct_adapter.hpp =================================================================== --- detail/adapter/direct_adapter.hpp (revision 59992) +++ detail/adapter/direct_adapter.hpp (working copy) @@ -116,6 +116,7 @@ std::streamsize write(const char_type* s, std::streamsize n); std::streampos seek( stream_offset, BOOST_IOS::seekdir, BOOST_IOS::openmode = BOOST_IOS::in | BOOST_IOS::out ); + bool putback(const char_type c); void close(); void close(BOOST_IOS::openmode which); #ifndef BOOST_IOSTREAMS_NO_LOCALE @@ -204,6 +205,18 @@ }
template<typename Direct> +inline bool direct_adapter<Direct>::putback + (const char_type c) +{ + pointers& get = ptrs_.first(); + if( get.ptr == get.beg ) + throw bad_putback(); + get.ptr -= 1; + *get.ptr = c; + return true; +} + +template<typename Direct> inline std::streamsize direct_adapter<Direct>::write (const char_type* s, std::streamsize n) {
Nikolay Mladenov _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Mon, Mar 1, 2010 at 10:40 AM, Chris Hamilton <chamilton@cs.dal.ca> wrote:
Ahh... my bad for misreading ;)
Rather than patching things this way, you could always use the array_source device defined in <boost/iostreams/device/array.hpp>. A very simple wrapper can also be written around a std::vector, or you can use the array_source directly on the internal data of the std::vector.
You then use a filtered_source, and push back the gzip_decompressor followed by the array_source, like the earlier example in this thread.
That is exactly what is not working. And the patch is to make it work. I actually found out that there is a ticket for this problem already.
Cheers,
Chris
Thank you eg, Chris,
But I have probably misled you a bit with my post. The gzipped data is actually in memory (inside the storage variable), and I wanted to have a gzip_decompressing stream fromT that.
for no I am using this patch
Index: detail/adapter/direct_adapter.hpp =================================================================== --- detail/adapter/direct_adapter.hpp (revision 59992) +++ detail/adapter/direct_adapter.hpp (working copy) @@ -116,6 +116,7 @@ std::streamsize write(const char_type* s, std::streamsize n); std::streampos seek( stream_offset, BOOST_IOS::seekdir, BOOST_IOS::openmode = BOOST_IOS::in | BOOST_IOS::out ); + bool putback(const char_type c); void close(); void close(BOOST_IOS::openmode which); #ifndef BOOST_IOSTREAMS_NO_LOCALE @@ -204,6 +205,18 @@ }
template<typename Direct> +inline bool direct_adapter<Direct>::putback + (const char_type c) +{ + pointers& get = ptrs_.first(); + if( get.ptr == get.beg ) + throw bad_putback(); + get.ptr -= 1; + *get.ptr = c; + return true; +} + +template<typename Direct> inline std::streamsize direct_adapter<Direct>::write (const char_type* s, std::streamsize n) {
Nikolay Mladenov _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
participants (3)
-
Chris Hamilton -
eg -
Nikolay Mladenov