Hi Mateusz, I'm glad you can build the documentation. I'm pretty much illiterate when it comes to bjam, etc. I have written a short readme.txt. See here: http://code.google.com/p/gil-contributions/source/browse/trunk/gil_2/readme.... If you like it I'll add it to the download. Regards, Christian On 10/4/10, Mateusz Loskot <mateusz@loskot.net> wrote:

On 01/10/10 16:25, Mateusz Loskot wrote:

...

On 01/10/10 16:14, Christian Henning wrote:

...

This is the final version that will be used for the boost review.

Could someone try to build documentation on Linux?

I tried on both Windows and Linux and on both platforms I'm getting the same error message:

G:\dev\gil\review\_svn\trunk\libs\gil\io_new\doc>bjam

error: Attempt to redeclare already existing project id '/boost/exception' location '../../../../libs/exception/build'

Christian,

It looks it was my fault.

Simply, I had incorrect location to boost project in my user-config.jam Normally, I use this:

use-project boost : G:/dev/boost/_svn/trunk ;

but for GIL review I set up separate dedicated Boost tree and forgot to update the entry to read:

use-project boost : G:/dev/gil/review/_svn/trunk ;

After updating the use-project entry, I can build documentation.

I still think the review package is missing installation instructions. Have you thought about it?

Best regards, -- Mateusz Loskot, http://mateusz.loskot.net Charter Member of OSGeo, http://osgeo.org Member of ACCU, http://accu.org _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Hi Domagoj, Domagoj Saric wrote:

Unlike io and io_new it uses objects that represent on-disk images ("formatted images" in io2-speak). This has several advantages: - you do not have to open an image twice in order to first query its properties (if you need to do so for whatever reason) and then to actually read it...which is both cumbersome and inefficient... - you do not have to open an image many times (and seek thru it over and over again) when reading an image in smaller pieces/ROIs... - an object based design allowed for a CRTP based design where most of the shared boiler plate code could have been extracted into a single base class which in turn allows for greater maintainability as well as easier extensibility - it provides access to the underlying backend to the user for maximum flexibility.

This sounds great. It's much closer to what I was trying to suggest back in March when we last had this discussion. I've had a quick look at your libjpeg interface. Two features that I often use are dct_method=JDCT_IFAST and scaling down 2x or 4x using scale_num and scale_denom. It would be interesting to know whether (a) your existing code lets me use these via a "native handle", or (b) they could be added easily. My preference would still be to have a thinner library-wrapping layer, i.e. just a C++ification of the C library interface (see e.g. http://chezphil.org/tmp/LibJpeg.hh for an incomplete example), underneath the gil interface. But what you're proposing sounds much closer to this than Christian's proposal. Regards, Phil.

"Phil Endecott" <spam_from_boost_dev@chezphil.org> wrote in message news:1286804807448@dmwebmail.dmwebmail.chezphil.org...

...

I've had a quick look at your libjpeg interface. Two features that I often use are dct_method=JDCT_IFAST and scaling down 2x or 4x using scale_num and scale_denom. It would be interesting to know whether (a) your existing code lets me use these via a "native handle", or (b) they could be added easily.

Well, w/o trying, I think you should be able to do it straight away with option a:

gil::libjpeg_image::reader_for<...>::type your_image( ... ); your_image.lib_object().dct_method = JDCT_IFAST; your_image.lib_object().scale_num = your_scale_num; your_image.copy_to.....(...);

Ok, w/ trying it did not work if you tried to copy to an image with 'dimension synchronization' (resizing the target GIL image to fit the JPEG image) because: a) the input/original/unscaled dimensions were used to resize the image/report the JPEG dimensions b) even if output/scaled dimensions were used it still would not work because the target image is resized _before_ the start of the JPEG decompression (when the output dimensions are updated)... So I just committed a fix to account for both of those issues. The key issue was to cause the output dimensions to be updated/recalculated (to call jpeg_calc_output_dimensions()) if the user changes the scale_num value. There are several ways that this can be handled: - add an internal flag to libjpeg_image that will be set every time the user calls lib_object() and will be checked (and cleared) in dimensions() (where jpeg_calc_output_dimensions() will be called if the flag is set) - instead of a plain jpeg_decompress_struct reference make lib_object() return a proxy reference class that will call jpeg_calc_output_dimensions() in its destructor - expose a new 'update_output_dimensions()' like member function through which the user can (and must if relevant changes are made using lib_object()) deterministically cause the update... ...all of these of course have different compromises between overhead, 'cleanness' and safety...currently I used the first option... -- "What Huxley teaches is that in the age of advanced technology, spiritual devastation is more likely to come from an enemy with a smiling face than from one whose countenance exudes suspicion and hate." Neil Postman

Hi Domagoj, thanks for hard work. I haven't had a chance to look at it but if everything that you say is true that's great! There is quite a bit to answer here. See below:

Hi, a while back I wrote about using alternative GIL.IO backend implementations that would use native libraries/functionality (e.g. GDI+ on Windows, QuickTime on Mac...) and provided a GDI+ implementation for the original/current/official GIL.IO. That version used a configuration macro to select backend implementation and you remarked that I should rather use a different type to differentiate backends. That was a very good point as it pointed :) in the right direction, where/how should the GIL.IO interface change: to use objects instead of free functions (as was already requested by other users) however you chose not to go that way but to only refine the existing interface...because I did not agree both with this design decision as well as the many implementation decisions (such as using std::streams for in-memory image loading) I 'sat down' to write my own proposal...

I'm a little lost here. What objects and free functions do you mean? And what way did I not choose to go? I apologize, my memory is a little spotty. Why not use std::streams and what are the alternatives? In the end I understand std::streams as providing an interface though other implementations can be used, as well.

- wic_image (WIC (WinXP SP3+)).

WIC?

xxx The design/interface difference xxx --------------------------------------------

Unlike io and io_new it uses objects that represent on-disk images ("formatted images" in io2-speak). This has several advantages:

Could you go into more details? What are "formatted images"?

- you do not have to open an image twice in order to first query its properties (if you need to do so for whatever reason) and then to actually read it...which is both cumbersome and inefficient...

Agreed. In io_new you can reuse a std::ifstream as often as you need. Same for FILE*.

- you do not have to open an image many times (and seek thru it over and over again) when reading an image in smaller pieces/ROIs...

That's the same statement as the first one, correct? ROI is Region Of Interest?

- an object based design allowed for a CRTP based design where most of the shared boiler plate code could have been extracted into a single base class which in turn allows for greater maintainability as well as easier extensibility

CRTP?

- it provides access to the underlying backend to the user for maximum flexibility. With the current design, if the GIL.IO wrapper does not provide access to a backend feature users are helpless: for example, users had to wait for months for the library maintainer to add support for TIFF directory selection by adding yet another global function or adding more (defaulted) parameters to the existing one...With io2 this would not be necessary, the user could simply say: libtiff_image my_tiff( "my_tiff.tiff" ); ::TIFFSetDirectory( &my_tiff.lib_object(), <a directory number> ); my_tiff.copy_to(....);

That's a neat feature that I like to have too.

- it allows for easy/direct selection of the preferred backend and/or using several different backends... All in all the issue of easier writing/adding and selecting backend wrappers is very important because someone might want to use FreeImage or LodePNG or GDI+ or might simply be forced into using a specific backend by a 3rd party library (like a GUI framework)...

Again, that's good to have. Are you providing a jamfile that has such flexibility?

Furthermore IO2 also provides moving ROI capabilities, meaning you can read a large image in pieces by repeatedly calling an appropriate method on the image object thereby avoiding the need to reopen the image an seek through it. It will also try to be smart and read/decode as little as possible of unwanted data when skipping through an image.

I would love to know how you seek through a image using a 3rd party lib, for instance with libjpeg. Right now I'm just reading and discarding unwanted regions. Not the most ideal solution to say the least.

IO2 also tries to be very metaprograming friendly, for example the various backends provide introspective information either directly or thru a traits class: - mpl typelist of supported GIL image/pixel formats - mpl typelist of supported on-disk-image formats - mpl typelist of supported source types (e.g. all backends support char const *, FILE & and a memory-range, while wic_image, for example, also supports wchar_t const *, HANDLE and IStream &) - its native ROI and offset types - a metafunction that can be queried whether a pixel format is natively supported - the desired allocation alignment - whether it has builtin conversion...

Sounds all good if I understand correctly. I believe io_new does some of these things, as well.

It can read from in-memory images which it models using a plain boost::iterator_range<unsigned char const *>. So if you have a static image like: unsigned char const my_static_png[] = {...} io2 will read it directly without the overhead of an intermediate std::stream object...

io_new tries to have an extensible framework though supporting such plain byte arrays can be added.

Various policies for configuring the reading process are also provided, e.g.: { typedef libpng_image::reader_for<memory_chunk_t const &>::type reader_t; typedef image<bgr8_pixel_t, false> image_t;

image_t test_image; reader_t reader( my_static_png ); reader.copy_to_image( test_image, synchronize_dimensions(), assert_formats_match() );

wic_image::writer_for<char const *>::type( "test.jpg", test_image._view, jpeg ).write_default(); }

Not sure what synchronize_dimensions and assert_formats_match stands for. In io_new you will get an assertion when you try to read an image ( without conversion ) that is incompatible with the user provided image. For instance in case a file is rgb8 a user can read it with bgr8 image.

xxx The implementation difference xxx --------------------------------------------

[snip]

notes: - the starting overhead of static linking with the CRT and of constructing and resizing a GIL image<> is 40.448 bytes

What do you mean?

- LibJPEG was additionally compiled with the NO_GETENV macro and LibPNG with the PNG_NO_CONSOLE_IO (additionaly for io2 it was built with PNG_NO_STDIO, PNG_NO_WARNINGS, PNG_NO_ERROR_TEXT macros, io and io_new would not link with those macros)

When using these compiler symbols do you experience significant speed ups?

- I don't know if I missed something or did something seriously wrong but the io_new TIFF reader seems broken in the sense that it does not do any pixel/image format conversion and thus works properly only when the destination image/view is in the same format as the image on disk....?

I'll investigate.

- unfortunately io2 currently builds only with MSVC++...

Shouldn't be too hard to make changes to compile with gcc. I don't have gcc but I remember what to do. Maybe you wanna try MinGW and code_blocks IDE.

There are many many things io2 does (or does not do) to achieve these results, some of them are: - it has special support/handling/code-paths for in-memory/"basic" views for which it decodes directly to the target view avoiding any intermediate buffers (and inherent memory allocation and copying. Even if the backend does not support the target view's format it can sometimes still decode directly to target view memory space and then do an in-place transformation

A basic view is byte array? Not sure what you mean.

- it maximally uses functionality provided by the backends, so if a backend provides conversion capabilities and the user only specified the 'synchronize_formats' policy without explicitly specifying a colour converter the backend's builtin conversion will be used

That sounds great. I have to look at how you deal with certain functionalities, like reading ROI in a jpeg image.

- it completely avoids expensive/bloating STL and/or boost constructs, it does not use a single std::vector (if required a scoped_ptr does just fine), std::string (unlike io_new which, for example, creates and destroys 8 useless std::strings before it even calls jpeg_start_decompress(), or 15 std::strings before the first call to TIFFReadScanline() and then one std::string for every subsequent TIFFReadScanline() call), shared_ptrs and of course no streams...it also avoids iterator_facade abstractions as I've seen them producing code several times larger than the loop they wrap...

As far as I remember I use references when passing strings along. Could you tell where in my code you see that useless string creation destroying problem?

- uses streamlined/minimized/out-of-the-main-code-path-as-possible error detection and handling

Not sure what you mean.

- uses custom error handling and input-output routines that avoid the use/inclusion of stdio and printf family of functions (and possibly allow the linker to remove the big error/warning message tables provided by some of the libraries), use memory mapped files for input, use MSVC capability to throw through C code (avoiding the use of setjmp)

I have some sort of error handling implemented that users can customize. Not for all supported formats but for some. You do understand that io2 has to have portable source code in order to become a boost lib? Any idea of avoiding setjmp in a portable manner?

ps. The libjpeg_image::read( ... ) interface used in the test code is a simple utility static member function provided by the base CRTP class (thus available in all backends automatically) that simply wraps the default ...reader_for<>::...copy_to(...) code...

I really don't understand.

pps. There is one more thing I developed along with/for io2 and that is the ability to fully configure the way 3rd party libraries (used by the backends) are linked and initialised (although this is currently only implemented for GDI+ and WIC). So if you have an image-centric application you will probably want the backend lib to be linked statically and initialised once on startup , if on the other hand you use GIL.IO only for things like loading a skin a application startup and then no longer need image IO functionality you'll want to load and initialise the library on-demand once, do the work and then release it...and there are all possible combinations in between (e.g. static linking can be with a .lib/.a or a .dll/.so, dynamic linking can be 'only delayed but one-time loading' or can be repeated loading, unloading and reloading of a .dll etc...)...This is configured globaly with a macro, for example: #define BOOST_GIL_EXTERNAL_LIB ( BOOST_LIB_LINK_LOADTIME_OR_STATIC, BOOST_LIB_LOADING_STATIC, BOOST_LIB_INIT_ASSUME ) According to the BOOST_GIL_EXTERNAL_LIB macro individual library backends will make assumptions as to in which state the 3rd party library is in when they are constructed and what else they need to do to fully initialise it...To ensure the 'contract' specified with the BOOST_GIL_EXTERNAL_LIB macro each backend provides a public nested type called "guard" intended for the user to instantiate it before using the backend. For example: gp_image::guard const lib_guard; ... do something ... GDI+ automatically cleaned up at end of scope...

That sounds great and certainly is a nice-to-have feature. In conclusion, I must say I'm impressed with your io2. I think there are a lot of things we both can reuse and add into our projects. Maybe a merge could be possible. One of my highest goals is to support as many image formats as possible. For instance, a lot of work went into the tiff extension since it's to most flexible. Here, all bit image types are supported, for instance rgb7_14_12 in a tiled or strip manner. Lets try to make to most useful gil.io extension possible. Regards, Christian

Hi there, good to hear from you.

Your interface uses free (global) functions (like xxx_read_and_convert_image()...) while my interface uses objects/classes and corresponding member functions...

OK, now I understand. The new interfaces don't have the xxx_ anymore. Now we have: read_image( filename, gil_view, xxx_tag() ); The reason I choose this unified interface is first of all historical. The current gil::io uses free functions as well, which are by the way not global but inside a namespace. I also provide header files which supports the old styles to avoid breaking existing code. The second reason is the ease of use. I believe using these free functions is easy for users to understand since they all have their own distinct behavior and meaning.

...

Why not use std::streams

Streams are _evil_ ... http://lists.boost.org/Archives/boost/2010/01/160911.php ...

...

and what are the alternatives?

In my book anything is an alternative to screams :D

OTOH, only providing a streams based interface as an additional option is usually just fine (as opposed to forcing a streams only based interface)

Don't wanna get into a "streaming" fight here. Point is noted. Did you know there are several fast streams implementation available on the net? They might change some arguments.

...

WIC?

http://msdn.microsoft.com/en-us/library/ee719902(VS.85).aspx

Looks interesting.

...

...

Unlike io and io_new it uses objects that represent on-disk images ("formatted images" in io2-speak). This has several advantages:

Could you go into more details? What are "formatted images"?

Well, images in different 'formats' like JPEG, PNG, GIF...obviously not a very clever name that I just came up with to somehow distinguish them from 'raw'/in-memory images (like gil::image<>)...you are more than welcome to come up with a more intuitive name ;)

How about in-memory vs. of-file? ;-)

(AFAICT) You can reuse the low level file object but not the backend library object, which is what I was talking about and what is prerequisite for, for example, efficient 'moving ROI'/sequential-in-blocks image reading...

Are you talking about FILE* only or does that include std::ifstream as well?

...

...

   libtiff_image my_tiff( "my_tiff.tiff" );    ::TIFFSetDirectory( &my_tiff.lib_object(), <a directory number> );    my_tiff.copy_to(....);

That's a neat feature that I like to have too.

But I don't see how you can have that with free functions...

Maybe I should give out access to my reader< ..., xxx_tag, ... > objects.

No jamfile yet (Boost build/bjam are definitely not one of my favourite subjects or skills for that matter :) OTOH, I'm not quite sure (with my limited bjam knowledge) why/how would different backends influence the library's jamfile (atleast as long as it is header only)..?

Check out Mateusz's blog: http://mateusz.loskot.net/2010/10/17/notes-on-boost-build-for-boost-gil/

...

I would love to know how you seek through a image using a 3rd party lib, for instance with libjpeg. Right now I'm just reading and discarding unwanted regions. Not the most ideal solution to say the least.

With LibJPEG you cannot really/literally skip unwanted data but you can make the library read the unwanted data using faster/less precise methods which is what I currently do (and admittedly use a bit of LibJPEG's internal implementation detail knowledge for that)...

Cool, I would love to know more. Can you point where in your code you do that?

...

Not sure what synchronize_dimensions and assert_formats_match stands for.

synchronize_dimensions is a policy that causes the target image to be resized to fit the source image (it is obviously not valid for views), alternatively assert_dimensions_match and ensure_dimensions_match (throws in case of mismatch) can be used... For formats there are the corresponding assert_formats_match, ensure_formats_match and synchronize_formats (with the latter performing a builtin conversion if supported by the backend otherwise it uses GIL's default colour converter) with the addition that a user specified colour converter can be passed instead of the three mentioned predefined policies...

For synchronize_dimensions I have read_and_convert_image() which takes a custom color converter as on of the parameters. In case the users doesn't want to convert he/she would you read_image() but then the io extension would check of the user supplied image is compatible to the on-file image. Compatible images means both color spaces are compatible and all channel's are pairwise compatible.

...

...

- LibJPEG was additionally compiled with the NO_GETENV macro and LibPNG with the PNG_NO_CONSOLE_IO (additionaly for io2 it was built with PNG_NO_STDIO, PNG_NO_WARNINGS, PNG_NO_ERROR_TEXT macros, io and io_new would not link with those macros)

When using these compiler symbols do you experience significant speed ups?

Not (direct) speedups but smaller binaries (when coupled with custom error and IO handling routines) because of removal of stdio code and error message tables from the binary...

Good to know.

...

...

- I don't know if I missed something or did something seriously wrong but the io_new TIFF reader seems broken in the sense that it does not do any pixel/image format conversion and thus works properly only when the destination image/view is in the same format as the image on disk....?

I'll investigate.

Have you perhaps had the chance?

Yep, working on a fix. Tiff is very flexible and reading files without conversion works fine. But when reading with conversion I get into a problem with template member overloading. Basically I have a switch statement that would overload the read_data member with the on-file pixel format as template parameter. Now this pixel parameter can be anything for gray1 to rgb23_55_12, etc. Which makes the switch statement approach not feasible. I'm inclined to only allow read_and_convert_image/view for the most common channel sizes.

...

That sounds great. I have to look at how you deal with certain functionalities, like reading ROI in a jpeg image.

Actually, I forgot to mention, with the LibX (JPEG, PNG, TIFF) backends I do not provide 'full'/2D/rectangular ROI capability but only 'vertical' ROIs (i.e. you can specify from which to which row/scanline to read but whole rows/scanlines are always read...) because the backends do not support reading partial rows/scanlines so adding full/proper ROIs for those backends would require emulation which in turn would cause code bloat/complication and (possible) redundant data copying...For now it seems to me that the user should handle 'full' ROI support in such cases (if required)...

I try to allow all sizes of ROI. It can be anything from a single pixel to a scanline or a vertical line to a rectangle. This was bit** to implement for tiff's tiled images. ;-) The basic algorithm is to read scanline by scanline and decided if there are portions that are needed or not. Same for tiles.

Place a breakpoint in the std::string constructor...it is mostly due to the change you made to the io_error() helper function (to take a std::string const & instead of char const *)...

You're absolutely right. Are you recommending a string table?

For example not performing a full check-and-throw on every library call but accumulating multiple results and throwing in one place/out of a loop...

Error reporting 'level'/complexity should ideally be configurable by the user (which is something I'd like to see on a global Boost level) in the sense that the user can choose whether errors should be reported with a simple std::exception with a single/simple hardcoded message (like "GIL IO failure") or a with a full std::ios_base::failure filled with detailed and formatted error messages provided by the backend library or with something inbetween...

Sounds interesting. I might add that after the review depending on the reviewers input.

...

Any idea of avoiding setjmp in a portable manner?

It cannot be avoided using only standard C and C++ because/if the libraries use setjmp. My code also uses/works with setjmp unless BOOST_GIL_THROW_THROUGH_C_SUPPORTED is defined (which is automatically defined for MSVC++)...

Are you suggesting your lib is not header-only?

...

In conclusion, I must say I'm impressed with your io2. I think there are a lot of things we both can reuse and add into our projects. Maybe a merge could be possible. One of my highest goals is to support as many image formats as possible. For instance, a lot of work went into the tiff extension since it's to most flexible. Here, all bit image types are supported, for instance rgb7_14_12 in a tiled or strip manner.

Well yes, your proposal goes further than just IO with for example the toolbox extension...although I had an idea, since the beginning of my dealing with GIL IO, that should probably go to the toolbox extension and that is support/adapters for image classes from various GUI frameworks that would make them conform the the GIL image concept. One more thing I forgot to mention is that I already made some steps in that direction with the GDI+ backend that make a gp_image behave directly like a gil::any_image<> (without the need to call copy_to...(...) on a gp_image object to read its contents)... The massive work you did with the wrapping of all the image/reading/writing properties that the individual backends provide can also prove to be useful... However a (relatively simple) merge does not seem possible because of the radically different interfaces (as well as implementations)...

You're more than welcome to add stuff into the toolbox extension. Let me know. Thanks again for your insightful comments, Christian

"Mateusz Loskot" <mateusz@loskot.net> wrote in message news:4CBC79E9.2040009@loskot.net...

It's also not clear to me what's wrong with use of streams in such case as in Boost.GIL where no character formatting, no use of manipulators but read/write calls (binary mode).

That's exactly what's wrong with them :) You do not use something yet 'they' still make you pay for it because of the almost exclusively runtime/dynamic configuration where different behaviours are runtime-switched using traditional branching and virtual functions... Not to mention dynamic memory allocation...and thread synchronization...and RTTI...and locales...and ad nauseam... It puzzles me that you still have to 'persuade' people into the simple fact that std::streams are an evil failure (as shyly/indirectly, and of course not in this wording, admitted even by the official technical report on C++ performance)...

...

I try to allow all sizes of ROI. It can be anything from a single pixel to a scanline or a vertical line to a rectangle. This was bit** to implement for tiff's tiled images. ;-)

IMO, that's correct approach. If scanline-based reading is not efficient, such image is a candidate for tiled tiff. This would improve decompression as well, as tiles are compressed independently.

This was not the issue (how to write an image, that's up to the user to decide)...the issue was whether, and how, to implement and provide ROI capabilities for backends that do not provide them or provide them poorly themselves. I better (re)explained the implications of different approaches in the last reply to Christian... -- "What Huxley teaches is that in the age of advanced technology, spiritual devastation is more likely to come from an enemy with a smiling face than from one whose countenance exudes suspicion and hate." Neil Postman

Hi Domagoj,

Finally, as far as I remember, in all discussions so far, of the few user arguments/'wishes' that touched the issue of the interface of GIL.IO all wished for an interface change exactly in the direction where IO2 went...

In any event, you need to make sure you offer at the very least a way to simulate the old gil::io interface to avoid breaking code.

ps. placing a function inside a namespace does not make it non-global...

It's not global either. ;-)

...

...

(AFAICT) You can reuse the low level file object but not the backend library object, which is what I was talking about and what is prerequisite for, for example, efficient 'moving ROI'/sequential-in-blocks image reading...

Are you talking about FILE* only or does that include std::ifstream as well?

Um, to rewind things a little to make sure we are talking about the same thing. I was talking about reusing the backend library objects/instances (e.g. TIFF, jpeg_decompress_struct, ...), which IO2 supports and IO_new AFAICT does not.

You can reuse a TIFF* but not reuse a jpeg_decompress_struct. It might be a good idea to add these objects to the image_read_settings< jpeg_tag > structures.

Simply reusing the low level source object (like a FILE) will make little difference if, for every ROI you read, you have to recreate the backend library object which in turn implies rereading the whole image up to the requested ROI instead of just continuing where you left off...

Is this statement true when an std::ifstream object is used?

Again, I don't see how can you do that with free functions except by allowing users to pass in low-level-object-modifier-functors to your global interface functions...which IMO would pretty much defeat your original argument of a 'simple and easy to use interface'...

More advanced use cases might need such access to low level objects. I agree with you that my current implementation lacks support for some advanced use cases.

...

Cool, I would love to know more. Can you point where in your code you do that?

In libjpeg_image.hpp search for "void skip_rows("...

Thanks, I will have a look.

...

Yep, working on a fix. Tiff is very flexible and reading files without conversion works fine. But when reading with conversion I get into a problem with template member overloading. Basically I have a switch statement that would overload the read_data member with the on-file pixel format as template parameter. Now this pixel parameter can be anything for gray1 to rgb23_55_12, etc. Which makes the switch statement approach not feasible. I'm inclined to only allow read_and_convert_image/view for the most common channel sizes.

I use Steven Watanabe's switch_ for this. And yes, when it comes to LibTIFF (and WIC) and the most generic case of runtime pixel format detection and conversion, this approach can cause generation of an uber huge amount of code of which the user may actually want to support/allow only a small subset... For this reason I planned to allow the user to specify compile-time filters for allowed pixel formats so that the code for other pixel formats does not get generated even if the backend supports them...

I went down the same alley for now. The user can only use read_and_convert_image for TIFFs for common formats like, gray1, rgb8, etc..

No, simply revert your change and use char const * where they are enough...unlike some think, a char const * is a perfectly valid C++ construct...

Yep, will do.

Also consider making friends with the disassembly window in release builds...

Any good tutorial on this somewhere? Regards, Christian

Hi Domagoj, good to hear from you! Not sure if you noticed, gil::io_new has scheduled a review starting Dec, 1st. For this I wont make changes to the source until the review starts.

...

...

Um, to rewind things a little to make sure we are talking about the same thing. I was talking about reusing the backend library objects/instances (e.g. TIFF, jpeg_decompress_struct, ...), which IO2 supports and IO_new AFAICT does not.

You can reuse a TIFF* but not reuse a jpeg_decompress_struct. It might be a good idea to add these objects to the image_read_settings< jpeg_tag > structures.

Hmm...for this not to turn out akward/non-RAII-like you'd probably need to make your image_read_settings<> into a non-trivial class with proper initialisation and destruction...IOW your design would become more object-based (more io2-like)...which is what I'm proposing all along (and the way io2 is already designed ;)

io_new is also object-oriented but it also provides, as you know, the free floating helper functions like read_image, etc. Even though, I don't state it in the documentation but you could use the underlying classes also. All image formats use the same reader class which is hidden inside the detail namespace. For instance: [taken from image_read.hpp] detail::reader< Device , FormatTag , detail::read_and_no_convert > reader( file , settings ); reader.init_image( img , reader.get_info() ); reader.apply( view( img )); This object can be reused at will. But I have a question for you. Are you sure you can easily reuse a jpeg_decompress_struct object and jump around a jpeg image without getting it confused? I mean, for instance, imagine you read the last image line first and then try to read the first line. Does that work?

...

...

Simply reusing the low level source object (like a FILE) will make little difference if, for every ROI you read, you have to recreate the backend library object which in turn implies rereading the whole image up to the requested ROI instead of just continuing where you left off...

Is this statement true when an std::ifstream object is used?

Well yes, the type of the input/source object does not matter...the problem is in the fact that the backend object (e.g. jpeg_decompress_struct) is destroyed and recreated (e.g. for each call to read_image()) which in turn requires/makes the backend library code reread the image from the beginning (to read the header, dimensions, format etc...)...

See my question from above.

...

More advanced use cases might need such access to low level objects. I agree with you that my current implementation lacks support for some advanced use cases.

The problem (in this particular case) is not (primarily) in the implementation but in the interface/design of io_new... If we can agree on this and that io2 solves these problems better would you agree to switch to (an) io2(-like) interface so we can merge and join our efforts? I can write some preliminary documentation on the base formatted_image (that forms the backbone of io2) and the requirements that each xxx_image implementation class (libpng_image, wic_image...) must fulfil according to which you can try and port for example your GIF reader and then, through this experience, say your objections/what you think needs to be changed/improved, how to accommodate/fit into/work with other parts of your work (the toolbox, the properties system...)...

I would love to work with you to create the best image reader/writer possible for the boost community. Let's see what the review brings to the table and afterwards we can start sorting out the major differences. How does that sound? Regards, Christian