Hi Phil,
In the cases that I described above, using a stringstream would require that all of the data is copied at least once. I guess that if I knew more about how streambufs work I would know how to avoid that. Can anyone help me to implement this:
void read_mem_jpeg( some_gil_image_tpye& dest, const char* jpeg_mem_begin, const char* jpeg_mem_end) { ... }
in a way that doesn't copy all of the encoded data?
Please correct if I'm wrong but what you're looking for a way to create a stringstream or similar that can be initialized with a character array containing an image? std::stringstream allows you to do that. Like this: template< typename Image > void read_mem_jpeg( Image& dest , const char* jpeg_file , const char* jpeg_file_end ) { istringstream ss( string( jpeg_file, jpeg_file_end ), std::ios::binary ); read_image( ss, dest, jpeg_tag() ); } Now, I don't know if the constructor of stringstream copies the data into another buffer. I'm not really an expert with c++ streams. I have posted a complete source code here http://pastebin.com/Ru8nvNsq
Looking at formats/jpeg/read.hpp, it seems that irrespective of that the data will be copied again (into 'buffer') before being read by libjpeg.
The 'buffer' only contains one scanline ( or row ) of your image. When reading an image with with this extension it usually is done a scanline by scanline manner.
On the subject of copying, while reading PNGs it looks like the data is always copied into a temporary buffer in read_rows<>() even when no format conversion is needed (or is there some specialisation that I have missed?). And when reading JPEGs, it seems to always read single lines, yet libjeg advises that you should get at least rec_outbuf_height lines on each call to avoid extra copying within the library.
Each format has their own reader class. One of the template parameters is the ConversionPolicy which can be either read_and_no_convert or read_and_convert. Each of these two classes have a member "read" which in the case of read_and_no_convert calls std::copy() and for read_and_convert calls std::transform(). When reading scanline by scanline the io extension has a buffer which holds one scanline only. The buffer is fed into the underlying library if there is one. In case no conversion is necessary I could have potentially use the user provided image as the buffer. About libjpeg, thanks for pointing out one potential inefficiency. As far as I can tell rec_outbuf_height is set to 1 be default. A user can change that number to 2 or 4 but the lib would need to be recompiled. The user also need to set the UPSAMPLE_MERGING_SUPPORTED compiler flag. I gladly add this feature to my todo list. Thanks, Christian