[Boost-users] [iostreams] Issue with closing of filters (esp. tee_filter)

I am running into an issue with iostreams related to the closing of filters (tee_filter in particular) and I think I really need a little help. Thanks in advance to anyone patient enough to read through the following. I created a multichar_output_filter called ChecksumFilter. It outputs a crc for every 4K of data streamed into it. I am using in conjunction with a tee_filter to stream out a file of checksums in parallel with another file stream. Unfortunately, ChecksumFilter::close() is not invoked at times when I believe it should be, resulting in the checksum for the final chunk of data not getting pushed out. The following code shows the problem: typedef boost::iostreams::composite ChecksumComposite; char buffer1[BufferSize]; memset(buffer1, DefaultValue, sizeof(buffer1)); std::string fileName1(tmpnam(0)); std::string fileName2(tmpnam(0)); boost::iostreams::file_sink sink1(fileName1.c_str()); boost::iostreams::file_sink sink2(fileName2.c_str()); ChecksumFilter chksum; ChecksumComposite comp(chksum, sink2); boost::iostreams::tee_filter<ChecksumComposite> tee(comp); { boost::iostreams::filtering_streamboost::iostreams::output ostream; ostream.push(tee); ostream.push(sink1); result = boost::iostreams::write(ostream, buffer1, sizeof(buffer1)); } // ChecksumFilter::close() is not invoked here!!! ChecksumFilter itself is pretty basic -- it is simply a multichar_output_filter: class ChecksumFilter : public boost::iostreams::multichar_output_filter { public: ChecksumFilter(void) : mImpl(new details::ChecksumFilterImpl()) { // Nothing to do here } template<typename Sink> std::streamsize write(Sink& sink, const char* s, std::streamsize n) { return mImpl->write<Sink>(sink, s, n); } template<typename Sink> void close(Sink& sink) { mImpl->close<Sink>(sink); } private: boost::shared_ptrdetails::ChecksumFilterImpl mImpl; }; I don't think the details of ChecksumFilterImpl are relevant. I spent some time tracing through the closing code path. I'm sure this comes as no surprise to anyone but it's quite a twisty maze. I will be the first to admit that my understanding of the library is incomplete, but here is something that I think is a bit odd -- somewhere along the way between the code in closer.hpp and tee.hpp, the openmode gets converted from BOOST_IOS::out to BOOST_IOS::in | BOOST_IOS::out.

From closer.hpp:

template<> struct close_impl { template<typename T> static void close(T& t, BOOST_IOS::openmode which) { printf("close closable_tag 1\n"); typedef typename category_of<T>::type category; const bool in = is_convertible::value && !is_convertible::value; if (in == ((which & BOOST_IOS::in) != 0)) { t.close(); } } template static void close(T& t, Sink& snk, BOOST_IOS::openmode which) { typedef typename category_of<T>::type category; const bool in = is_convertible::value && !is_convertible::value; if (in == ((which & BOOST_IOS::in) != 0)) { non_blocking_adapter<Sink> nb(snk); t.close(nb); } } };

From tee.hpp:

template<typename Device> class tee_filter : public detail::basic_adapter<Device> { public: ... template<typename Next> void close( Next&, BOOST_IOS::openmode which = BOOST_IOS::in | BOOST_IOS::out ) { iostreams::close(this->component(), which); } ... }; The original openmode at the start of the close chain is BOOST_IOS::out, but the close_impl's two close methods discard that openmode when they call t.close() or t.close(nb). This means that tee_filter's close() sets openmode to its default value of BOOST_IOS::in | BOOST_IOS::out. This mode is passed along to iostreams::close(), which ends up back in close_impl close() again as it is attempting to close the ChecksumFilter. However, now the check "if (in == ((which & BOOST_IOS::in) != 0))" fails, and so ChecksumFilter.close() is never invoked. If I change the default value for openmode in tee_filter::close() to BOOST_IOS::out, everything works OK for me, but I am not really sure that this is the right thing to do. It appears that some kind of custom closing in closer.hpp could be done for tee_filters that would pass through the openmode, much like the handling for classes that support the two_sequence tag (sorry if I am jumbling terminology a bit here). But I am not sure my understanding of the overall code is strong enough to do the right thing here either. Can anyone help me out or put me in touch with someone who can? Much thanks! Chad Walters PS: And if you read this far, you probably understand the iostreams library more than well enough to confirm whether the possible (likely?) bug I reported earlier this week is really a bug (nobody has replied as of yet): At the bottom of concepts.hpp, the following typedefs appear: typedef multichar_filter<input> multichar_input_filter; typedef multichar_filter<input> multichar_input_wfilter; typedef multichar_filter<output> multichar_output_filter; typedef multichar_filter<output> multichar_output_wfilter; typedef multichar_filter multichar_dual_use_filter; typedef multichar_filter multichar_dual_use_wfilter; I think the wfilter versions of these typedefs should use multichar_wfilter<...>. Agreed?