[gil io_new review] Reading images from in-memory sources
review of Christian Henning's extensions to the Boost Generic Image Library starts on December 1st and lasts until December 10th, 2010.
I've been looking at how to read images from in-memory data. For example, I have various places where I mmap() something that contains encoded images (e.g. a large map made up from many tiles in a single file). I can also imagine retrieving an encoded image from a database, for example. According to io.qbk: [h2 Reading And Writing In-Memory Buffers] Reading and writing in-mempory buffers are supported as well. See as follows: `` // 1. Read an image. ifstream in( "test.tif", ios::binary ); rgb8_image_t img; read_image( in, img, tiff_tag() ); // 2. Write image to in-memory buffer. stringstream out_buffer( ios_base::out | ios_base::binary ); rgb8_image_t src; write_view( out_buffer, view( src ), tiff_tag() ); // 3. Copy in-memory buffer to another. stringstream in_buffer( ios_base::in | ios_base::binary ); in_buffer << out_buffer.rdbuf(); // 4. Read in-memory buffer to gil image rgb8_image_t dst; read_image( in_buffer, dst, tag_t() ); In the cases that I described above, using a stringstream would require that all of the data is copied at least once. I guess that if I knew more about how streambufs work I would know how to avoid that. Can anyone help me to implement this: void read_mem_jpeg( some_gil_image_tpye& dest, const char* jpeg_mem_begin, const char* jpeg_mem_end) { ... } in a way that doesn't copy all of the encoded data? Looking at formats/jpeg/read.hpp, it seems that irrespective of that the data will be copied again (into 'buffer') before being read by libjpeg. On the subject of copying, while reading PNGs it looks like the data is always copied into a temporary buffer in read_rows<>() even when no format conversion is needed (or is there some specialisation that I have missed?). And when reading JPEGs, it seems to always read single lines, yet libjeg advises that you should get at least rec_outbuf_height lines on each call to avoid extra copying within the library. Regards, Phil.
Hi Phil,
In the cases that I described above, using a stringstream would require that all of the data is copied at least once. I guess that if I knew more about how streambufs work I would know how to avoid that. Can anyone help me to implement this:
void read_mem_jpeg( some_gil_image_tpye& dest, const char* jpeg_mem_begin, const char* jpeg_mem_end) { ... }
in a way that doesn't copy all of the encoded data?
Please correct if I'm wrong but what you're looking for a way to create a stringstream or similar that can be initialized with a character array containing an image? std::stringstream allows you to do that. Like this: template< typename Image > void read_mem_jpeg( Image& dest , const char* jpeg_file , const char* jpeg_file_end ) { istringstream ss( string( jpeg_file, jpeg_file_end ), std::ios::binary ); read_image( ss, dest, jpeg_tag() ); } Now, I don't know if the constructor of stringstream copies the data into another buffer. I'm not really an expert with c++ streams. I have posted a complete source code here http://pastebin.com/Ru8nvNsq
Looking at formats/jpeg/read.hpp, it seems that irrespective of that the data will be copied again (into 'buffer') before being read by libjpeg.
The 'buffer' only contains one scanline ( or row ) of your image. When reading an image with with this extension it usually is done a scanline by scanline manner.
On the subject of copying, while reading PNGs it looks like the data is always copied into a temporary buffer in read_rows<>() even when no format conversion is needed (or is there some specialisation that I have missed?). And when reading JPEGs, it seems to always read single lines, yet libjeg advises that you should get at least rec_outbuf_height lines on each call to avoid extra copying within the library.
Each format has their own reader class. One of the template parameters is the ConversionPolicy which can be either read_and_no_convert or read_and_convert. Each of these two classes have a member "read" which in the case of read_and_no_convert calls std::copy() and for read_and_convert calls std::transform(). When reading scanline by scanline the io extension has a buffer which holds one scanline only. The buffer is fed into the underlying library if there is one. In case no conversion is necessary I could have potentially use the user provided image as the buffer. About libjpeg, thanks for pointing out one potential inefficiency. As far as I can tell rec_outbuf_height is set to 1 be default. A user can change that number to 2 or 4 but the lib would need to be recompiled. The user also need to set the UPSAMPLE_MERGING_SUPPORTED compiler flag. I gladly add this feature to my todo list. Thanks, Christian
Christian Henning wrote:
Can anyone help me to implement this:
void read_mem_jpeg( some_gil_image_tpye& dest, const char* jpeg_mem_begin, const char* jpeg_mem_end) { ... }
in a way that doesn't copy all of the encoded data?
Please correct if I'm wrong but what you're looking for a way to create a stringstream or similar that can be initialized with a character array containing an image?
..without copying all of the data, yes.
std::stringstream allows you to do that. Like this:
template< typename Image > void read_mem_jpeg( Image& dest , const char* jpeg_file , const char* jpeg_file_end ) { istringstream ss( string( jpeg_file, jpeg_file_end ), std::ios::binary );
read_image( ss, dest, jpeg_tag() ); }
Now, I don't know if the constructor of stringstream copies the data into another buffer. I'm not really an expert with c++ streams.
The string ctor copies all the data, and then the istringstream ctor may or may not copy it all again (does anyone know?).
Looking at formats/jpeg/read.hpp, it seems that irrespective of that the data will be copied again (into 'buffer') before being read by libjpeg.
The 'buffer' only contains one scanline ( or row ) of your image. When reading an image with with this extension it usually is done a scanline by scanline manner.
Right. The data is all copied again. (There are two issues with copying. The first the the extra memory needed. In cases like this where you're copying line-by-line that's not much of a concern. The second is the extra time taken. This is a concern even when done line-by-line.)
On the subject of copying, while reading PNGs it looks like the data is always copied into a temporary buffer in read_rows<>() even when no format conversion is needed (or is there some specialisation that I have missed?). ?And when reading JPEGs, it seems to always read single lines, yet libjeg advises that you should get at least rec_outbuf_height lines on each call to avoid extra copying within the library.
Each format has their own reader class. One of the template parameters is the ConversionPolicy which can be either read_and_no_convert or read_and_convert. Each of these two classes have a member "read" which in the case of read_and_no_convert calls std::copy()
Right. It would be great to eliminate that copy.
and for read_and_convert calls std::transform().
When reading scanline by scanline the io extension has a buffer which holds one scanline only. The buffer is fed into the underlying library if there is one. In case no conversion is necessary I could have potentially use the user provided image as the buffer.
About libjpeg, thanks for pointing out one potential inefficiency. As far as I can tell rec_outbuf_height is set to 1 be default. A user can change that number to 2 or 4 but the lib would need to be recompiled. The user also need to set the UPSAMPLE_MERGING_SUPPORTED compiler flag.
Well, here's what it says in jpeglib.h: /* Description of actual output image that will be returned to application. * These fields are computed by jpeg_start_decompress(). * You can also use jpeg_calc_output_dimensions() to determine these values * in advance of calling jpeg_start_decompress(). */ JDIMENSION output_width; /* scaled image width */ JDIMENSION output_height; /* scaled image height */ int out_color_components; /* # of color components in out_color_space */ int output_components; /* # of color components returned */ /* output_components is 1 (a colormap index) when quantizing colors; * otherwise it equals out_color_components. */ int rec_outbuf_height; /* min recommended height of scanline buffer */ /* If the buffer passed to jpeg_read_scanlines() is less than this many rows * high, space and time will be wasted due to unnecessary data copying. * Usually rec_outbuf_height will be 1 or 2, at most 4. */ To me, that looks like the library sets the value not the user. Regards, Phil.
Hi Phil,
The string ctor copies all the data, and then the istringstream ctor may or may not copy it all again (does anyone know?).
Did you see Kenny's email regarding boost::iostreams?
About libjpeg, thanks for pointing out one potential inefficiency. As far as I can tell rec_outbuf_height is set to 1 be default. A user can change that number to 2 or 4 but the lib would need to be recompiled. The user also need to set the UPSAMPLE_MERGING_SUPPORTED compiler flag.
Well, here's what it says in jpeglib.h:
/* Description of actual output image that will be returned to application. * These fields are computed by jpeg_start_decompress(). * You can also use jpeg_calc_output_dimensions() to determine these values * in advance of calling jpeg_start_decompress(). */
JDIMENSION output_width; /* scaled image width */ JDIMENSION output_height; /* scaled image height */ int out_color_components; /* # of color components in out_color_space */ int output_components; /* # of color components returned */ /* output_components is 1 (a colormap index) when quantizing colors; * otherwise it equals out_color_components. */ int rec_outbuf_height; /* min recommended height of scanline buffer */ /* If the buffer passed to jpeg_read_scanlines() is less than this many rows * high, space and time will be wasted due to unnecessary data copying. * Usually rec_outbuf_height will be 1 or 2, at most 4. */
To me, that looks like the library sets the value not the user.
There is a function called use_merged_upsample which returns only true when UPSAMPLE_MERGING_SUPPORTED is set during compilation. Otherwise cinfo->rec_outbuf_height = 1; That's the only assignment I can find in the whole libjpeg source code. Maybe I'm missing something? Regards, Christian
"Christian Henning" <chhenning@gmail.com> wrote in message news:AANLkTik=kNJT2zBinixfgLpq3dvB7_nVDkEKw4ghr8oq@mail.gmail.com...
There is a function called use_merged_upsample which returns only true when UPSAMPLE_MERGING_SUPPORTED is set during compilation. Otherwise
cinfo->rec_outbuf_height = 1;
That's the only assignment I can find in the whole libjpeg source code. Maybe I'm missing something?
If I understood Phil correctly, he meant reading scanlines in packs like this: http://svn.boost.org/svn/boost/sandbox/gil/boost/gil/extension/io2/libjpeg_i... (search for rec_outbuf_height in function raw_convert_to_prepared_view())... -- "What Huxley teaches is that in the age of advanced technology, spiritual devastation is more likely to come from an enemy with a smiling face than from one whose countenance exudes suspicion and hate." Neil Postman
Hi there, On Tue, Dec 7, 2010 at 12:45 PM, Domagoj Saric <dsaritz@gmail.com> wrote:
"Christian Henning" <chhenning@gmail.com> wrote in message news:AANLkTik=kNJT2zBinixfgLpq3dvB7_nVDkEKw4ghr8oq@mail.gmail.com...
There is a function called use_merged_upsample which returns only true when UPSAMPLE_MERGING_SUPPORTED is set during compilation. Otherwise
cinfo->rec_outbuf_height = 1;
That's the only assignment I can find in the whole libjpeg source code. Maybe I'm missing something?
If I understood Phil correctly, he meant reading scanlines in packs like this: http://svn.boost.org/svn/boost/sandbox/gil/boost/gil/extension/io2/libjpeg_i... (search for rec_outbuf_height in function raw_convert_to_prepared_view())...
To make sure we are on the same page. You refer to rec_outbuf_height only once, inside a BOOST_ASSERT statement, correct? But Phil's point is taken and will be incorporated into the final version. Regards, Christian
"Christian Henning" <chhenning@gmail.com> wrote in message news:AANLkTinuo11NoaMQd3Ch-miXjHJVuWcA=_LL_jJFmLqD@mail.gmail.com...
If I understood Phil correctly, he meant reading scanlines in packs like this: http://svn.boost.org/svn/boost/sandbox/gil/boost/gil/extension/io2/libjpeg_i... (search for rec_outbuf_height in function raw_convert_to_prepared_view())...
To make sure we are on the same page. You refer to rec_outbuf_height only once, inside a BOOST_ASSERT statement, correct?
Yes (I explained the code in the post to Phil)...but yes, as you noted, the whole idea might make little or no difference with current/default LibJPEG builds... -- "What Huxley teaches is that in the age of advanced technology, spiritual devastation is more likely to come from an enemy with a smiling face than from one whose countenance exudes suspicion and hate." Neil Postman
Domagoj Saric wrote:
If I understood Phil correctly, he meant reading scanlines in packs like this: http://svn.boost.org/svn/boost/sandbox/gil/boost/gil/extension/io2/libjpeg_i... (search for rec_outbuf_height in function raw_convert_to_prepared_view())...
You're always reading 4 lines and asserting that rec_outbuf_height is not greater than this (as the docs say it will be). This is not unreasonably, but it's not exactly what I meant; in my code, I explicitly read exactly rec_outbuf_height lines. However, according to Christian's investigation, it looks like this is a feature that is present in the interface of libjpeg but not in the implementation; unless this is likely to change, I think it can be ignored. Phil.
"Phil Endecott" <spam_from_boost_dev@chezphil.org> wrote in message news:1291755767785@dmwebmail.dmwebmail.chezphil.org...
You're always reading 4 lines and asserting that rec_outbuf_height is not greater than this (as the docs say it will be). This is not unreasonably, but it's not exactly what I meant; in my code, I explicitly read exactly rec_outbuf_height lines.
Hmm, yes, this passage is definitely missing an implementation note :) The assert is actually to be read the other way around, it is verifying that I am providing enough scanlines for optimal reading (as recommended by the documentation)...And the code does not actually read 4 lines, rather it provides LibJPEG with the space to read up to 4 lines if it can/wants...it is up to LibJPEG to decide how much to read...There is no need to read/use the rec_outbuf_height value (unless you plan to dynamically allocate the small scanlines buffer which would be self defeating) if the documentation guarantees it will be between 1 and 4... -- "What Huxley teaches is that in the age of advanced technology, spiritual devastation is more likely to come from an enemy with a smiling face than from one whose countenance exudes suspicion and hate." Neil Postman
On 12/5/2010 3:46 PM, Phil Endecott wrote:
review of Christian Henning's extensions to the Boost Generic Image Library starts on December 1st and lasts until December 10th, 2010.
I've been looking at how to read images from in-memory data. For example, I have various places where I mmap() something that contains encoded images (e.g. a large map made up from many tiles in a single file). I can also imagine retrieving an encoded image from a database, for example.
According to io.qbk:
[h2 Reading And Writing In-Memory Buffers] Reading and writing in-mempory buffers are supported as well. See as follows:
`` // 1. Read an image. ifstream in( "test.tif", ios::binary );
rgb8_image_t img; read_image( in, img, tiff_tag() );
// 2. Write image to in-memory buffer. stringstream out_buffer( ios_base::out | ios_base::binary );
rgb8_image_t src; write_view( out_buffer, view( src ), tiff_tag() );
// 3. Copy in-memory buffer to another. stringstream in_buffer( ios_base::in | ios_base::binary ); in_buffer << out_buffer.rdbuf();
// 4. Read in-memory buffer to gil image rgb8_image_t dst; read_image( in_buffer, dst, tag_t() );
In the cases that I described above, using a stringstream would require that all of the data is copied at least once. I guess that if I knew more about how streambufs work I would know how to avoid that. Can anyone help me to implement this:
void read_mem_jpeg( some_gil_image_tpye& dest, const char* jpeg_mem_begin, const char* jpeg_mem_end) { ... }
in a way that doesn't copy all of the encoded data?
Looking at formats/jpeg/read.hpp, it seems that irrespective of that the data will be copied again (into 'buffer') before being read by libjpeg.
On the subject of copying, while reading PNGs it looks like the data is always copied into a temporary buffer in read_rows<>() even when no format conversion is needed (or is there some specialisation that I have missed?). And when reading JPEGs, it seems to always read single lines, yet libjeg advises that you should get at least rec_outbuf_height lines on each call to avoid extra copying within the library.
Regards, Phil.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I accomplish this using Boost.IOStreams...something like this works for me: using namespace boost::iostreams; stream< array_source > bufferStream( buffer, size ); read_image( bufferStream, dest, jpeg_tag() );
Hi Kenny,
I accomplish this using Boost.IOStreams...something like this works for me:
using namespace boost::iostreams; stream< array_source > bufferStream( buffer, size ); read_image( bufferStream, dest, jpeg_tag() );
Cool this is what I was looking for. No extra copying involved? Thanks, Christian
On 12/6/2010 10:10 AM, Christian Henning wrote:
Hi Kenny,
I accomplish this using Boost.IOStreams...something like this works for me:
using namespace boost::iostreams; stream< array_source> bufferStream( buffer, size ); read_image( bufferStream, dest, jpeg_tag() );
Cool this is what I was looking for. No extra copying involved?
Thanks, Christian _______________________________________________ Unsubscribe& other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
From the IOStreams documentation on array_source: "Model of Source providing read-only access to a sequence of characters in memory." doesn't sound like any copying to me :)
On 06/12/10 16:21, Kenny Riddile wrote:
On 12/6/2010 10:10 AM, Christian Henning wrote:
I accomplish this using Boost.IOStreams...something like this works for me:
using namespace boost::iostreams; stream< array_source> bufferStream( buffer, size ); read_image( bufferStream, dest, jpeg_tag() );
Cool this is what I was looking for. No extra copying involved?
From the IOStreams documentation on array_source:
"Model of Source providing read-only access to a sequence of characters in memory."
doesn't sound like any copying to me :)
I'm afraid this is your interpretation. This statement does not say anything about copying. It only states read-only access is performed. It is possible read-only access is performed against a copy of data. p.s. Cut signatures, please. Regards, -- Mateusz Loskot, http://mateusz.loskot.net Charter Member of OSGeo, http://osgeo.org Member of ACCU, http://accu.org
On 12/6/2010 11:26 AM, Mateusz Loskot wrote:
On 06/12/10 16:21, Kenny Riddile wrote:
On 12/6/2010 10:10 AM, Christian Henning wrote:
I accomplish this using Boost.IOStreams...something like this works for me:
using namespace boost::iostreams; stream< array_source> bufferStream( buffer, size ); read_image( bufferStream, dest, jpeg_tag() );
Cool this is what I was looking for. No extra copying involved?
From the IOStreams documentation on array_source:
"Model of Source providing read-only access to a sequence of characters in memory."
doesn't sound like any copying to me :)
I'm afraid this is your interpretation. This statement does not say anything about copying. It only states read-only access is performed. It is possible read-only access is performed against a copy of data.
p.s. Cut signatures, please.
Regards,
I agree that it's just my interpretation and I haven't verified that interpretation. Still I think it is likely to be a common interpretation, and one that is only incorrect if the author of the documentation was intentionally trying to be confusing and/or the library author was doing something silly.
On 06/12/10 17:28, Kenny Riddile wrote:
On 12/6/2010 11:26 AM, Mateusz Loskot wrote:
On 06/12/10 16:21, Kenny Riddile wrote:
On 12/6/2010 10:10 AM, Christian Henning wrote:
I accomplish this using Boost.IOStreams...something like this works for me:
using namespace boost::iostreams; stream< array_source> bufferStream( buffer, size ); read_image( bufferStream, dest, jpeg_tag() );
Cool this is what I was looking for. No extra copying involved?
From the IOStreams documentation on array_source:
"Model of Source providing read-only access to a sequence of characters in memory."
doesn't sound like any copying to me :)
I'm afraid this is your interpretation. This statement does not say anything about copying. It only states read-only access is performed. It is possible read-only access is performed against a copy of data.
I agree that it's just my interpretation and I haven't verified that interpretation. Still I think it is likely to be a common interpretation, and one that is only incorrect if the author of the documentation was intentionally trying to be confusing and/or the library author was doing something silly.
I am not sure about it. IMO, the situation is is similar to standard algorithms and predicates: if a semantic is not specified in details, any valid semantic is possible in terms of presented function prototype/class definition, etc. Reading about std::remove_if in copy of the n3092, I don't see a word about specific move/copy requirements of predicate, however, the algorithm is free to make number of copies of the predicate internally. Back to the Boost.IOStreams, indeed, the docs are incomplete [1] but as the the author explained on the list, the devices (Source is a device) does not have to follow "must be copy-constructibile" but they "can be non-copyable", what means that some may be copyable as well. [1] http://lists.boost.org/Archives/boost/2005/11/96479.php [2] http://lists.boost.org/Archives/boost/2005/10/95939.php Best regards, -- Mateusz Loskot, http://mateusz.loskot.net Charter Member of OSGeo, http://osgeo.org Member of ACCU, http://accu.org
On 12/6/2010 12:56 PM, Mateusz Loskot wrote:
I am not sure about it. IMO, the situation is is similar to standard algorithms and predicates: if a semantic is not specified in details, any valid semantic is possible in terms of presented function prototype/class definition, etc.
Reading about std::remove_if in copy of the n3092, I don't see a word about specific move/copy requirements of predicate, however, the algorithm is free to make number of copies of the predicate internally.
Back to the Boost.IOStreams, indeed, the docs are incomplete [1] but as the the author explained on the list, the devices (Source is a device) does not have to follow "must be copy-constructibile" but they "can be non-copyable", what means that some may be copyable as well.
[1] http://lists.boost.org/Archives/boost/2005/11/96479.php [2] http://lists.boost.org/Archives/boost/2005/10/95939.php
Best regards,
Yes, implementations of a spec are free to implement their requirements however they like (assuming no other requirements are violated). That being said, I just looked at the implementation of array_source and it appears that none of the data in the array is being copied.
"Kenny Riddile" <kfriddile@gmail.com> wrote in message news:idj944$lk7$1@dough.gmane.org...
Yes, implementations of a spec are free to implement their requirements however they like (assuming no other requirements are violated). That being said, I just looked at the implementation of array_source and it appears that none of the data in the array is being copied.
Yes, AFAICT also, the array_source implementation does not copy the input data (in fact, even if the documentation does not state it explicitly, it would IMO be common sense to assume so...as if it did actually copy the data this would constitute a performance bug by any standard)... However, redundant copying is only part of the issue here (in some use cases possibly even a less important one) as boost::iostreams::array_source still uses the std::iostreams infrastructure which means that includes all of its redundant horrors (locales, mutexes, virtual functions, RTTI, dynamic memory allocation, exceptions, transformations......)... Simply looking at/stepping through the release-mode generated assembly for merely the boost::iostreams::stream<array_source> constructor will make most heads dizzy... A proper (and zero overhead at that) way to model an in memory image would be a plain boost::iterator_range<> ... Not only because it is free of all the iostream monstrosities but because with backends like LibJPEG and LibTIFF reading an image from a boost::iterator_range<> can be accomplished without the LibXXX backend making a single seek/read callback (back to the GIL.IO 'device' that is in this case a boost::iterator_range<> or a transparent wrapper around it) because it already has all the data readily available... Additionally with the boost::iterator_range<> approach it is easy to add memory mapped file support for reading images (where the input image/file is again represented with a plain boost::iterator_range<> with all the benefits outlined above)... -- "What Huxley teaches is that in the age of advanced technology, spiritual devastation is more likely to come from an enemy with a smiling face than from one whose countenance exudes suspicion and hate." Neil Postman
participants (6)
-
Christian Henning -
Domagoj Saric -
Domagoj Saric -
Kenny Riddile -
Mateusz Loskot -
Phil Endecott