[Iostreams] Support std::codecvt<char> and std::codecvt_base::noconv in boost::iostreams::code_converter

Hello, I found two problems when using boost::iostreams::code_converter with char as source (internal) type for the conversion. Firstly, I was not able to use facets parameterized with char extracted from std::locale and secondly, my test run in an endless loop if the facet returned std::codecvt_base::noconv from do_in() or do_out(). I would like to use the boost::iostreams::code_converter in a class parametrized either for char or wchar_t using the default std::codecvt. This code works for wchar_t: namespace io = boost::iostreams; class test { io::stream<io::code_converter<xxxodevice, io::default_codecvt> > output; ... }; However I cannot make in templated like this without inheriting from std::codecvt and pasing the inherited type to io::code_converter instead of std::codecvt: template <class C> class test { io::stream<io::code_converter<xxxodevice, std::codecvt<C, char, std::mbstate_t> > output; ... }; The compiler complains about the usage of std::codecvt: error C2248: 'std::codecvt<_Elem,_Byte,_Statype>::~codecvt' : cannot access protected member declared in class 'std::codecvt<_Elem,_Byte,_Statype>' io::detail::codecvt_holder<Codecvt> in codecvt_holder.hpp stores a copy of Codecvt but std::codecvt defines only protected destructor: template<typename Codecvt> struct codecvt_holder { ... Codecvt codecvt_; }; Standard std::codecvt specialized for wchar_t is solved by its explicit specialization storing the pointer: struct default_codecvt { typedef wchar_t intern_type, from_type; typedef char extern_type, to_type; typedef std::mbstate_t state_type; }; template<> struct codecvt_holder<default_codecvt> { ... const codecvt_type* codecvt_; }; Obviously it supports the most usual usage with an instance extracted from std::locale which owns it: typedef std::codecvt<wchar_t, char, std::mbstate_t> codecvt_type; ... = &std::use_facet<codecvt_type>(locale); What about facets not converting from sequence of wchat_ts but chars? If I imbue such facet to a locale I am not able to extract it and use it in io::code_converter: template <class C> class test { io::stream<io::code_converter<xxxodevice, std::codecvt<C, char, std::mbstate_t> > output; void imbue(std::locale const & locale) { output.imbue(locale); } }; I helped it by converting io::detail::default_codecvt and the specialization of io::detail::codecvt_holder for it in codecvt_holder.hpp to this: template <class C> struct basic_default_codecvt { typedef C intern_type, from_type; ... }; template <class C> struct codecvt_holder<basic_default_codecvt<C> > { typedef std::codecvt<C, char, std::mbstate_t> codecvt_type; ... }; typedef basic_default_codecvt<wchar_t> default_codecvt; Now I was able to compile my slightly modified code: template <class C> class test { io::stream<io::code_converter<xxxodevice, io::detail::basic_default_codecvt<C> > output; void imbue(std::locale const & locale) { output.imbue(locale); } }; After being able to compile it a simple output code run endlessly. I found out that the facet in the default locale did not convert but it did not return true from always_noconv() from some reason. io::code_converter can cope with it - it copies the input buffer to the output one but it does not increase the pointer to the next character in the output buffer nor does it decrease the total bytes processed in io::code_converter::write. I am not sure about the behavior of the coverter if the facet returns std::codecvt_base::noconv. Currently it copies the buffer if internal type is the same as external one (both char) otherwise it does nothing (?). Should it not copy or assert if they are diferent? At least the writing should succeed if there was no conversion done and the characters were copied. I tried to modify the code accordingly and it succeeded then. I am attaching a patch to code_converter.hpp and codecvt_holder.hpp from boost 1.35.0. I could provide a test for it too. Thank you, Ferda
participants (1)
-
Ferdinand Prantl