[serialization]Nans and infinity in wide xml archive
Hi
I am using wide xml boost serialization archive. Because of the problem with
loading of nans and infinities I tried the nonfinite facet as in the code
snippet. This works, expect
it messes with the encoding of the xml arhive which is no longer UTF-8 like
before the using the facet, and I believe it is because of the no_codecvt
flag which is passed to the archive. But without this flag the loading of
nans/infinity does not work. How to use the facet and keep the wanted UTF-8
encoding ?
float d = std::numeric_limits<float>::quiet_NaN;
{
std::wofstream oss("C:\\test.xml");
std::locale locale1(oss.getloc(), new
boost::math::nonfinite_num_put
On 7/3/17 2:23 AM, Elizabeta via Boost-users wrote:
Hi I am using wide xml boost serialization archive. Because of the problem with loading of nans and infinities I tried the nonfinite facet as in the code snippet. This works, expect it messes with the encoding of the xml arhive which is no longer UTF-8 like before the using the facet, and I believe it is because of the no_codecvt flag which is passed to the archive. But without this flag the loading of nans/infinity does not work. How to use the facet and keep the wanted UTF-8 encoding ?
float d = std::numeric_limits<float>::quiet_NaN; { std::wofstream oss("C:\\test.xml"); std::locale locale1(oss.getloc(), new boost::math::nonfinite_num_put
); oss.imbue(locale1); xml_woarchive oar(oss, no_codecvt); oar << BOOST_SERIALIZATION_NVP(d); }
{ float d; std::wifstream iss("C:\\test.xml"); std::locale locale1(iss.getloc(), new boost::math::nonfinite_num_get
); iss.imbue(locale1); xml_wiarchive iar(iss, no_codecvt); iar >> BOOST_SERIALIZATION_NVP(d); std::cout << d << std::endl; }
This is an old problem going back many years. The problem isn't really the serialization library, but rather writing and reading NaN values to a text stream. There have been solutions proposed, but there hasn't been enough motivation pursue it. These solution address the issue at the stream level and don't touch serialization. For more information it would be best to search the mailing list - going back many years. Note that this problem occurs only with stream/text i/o. Binary formats don't suffer from this - but they are, of course, do not create/use portable archives. One expedient way to address this at the serialization level is to step back and ask oneself why he is serializing NaN values in the first place. They generally contain no useful information. One could catch them on output and replace them with a custom flag or boost.optional or some ad hoc solution. Or one could fix the NaN text i/o issue at a lower level. Robert Ramey
Hi Robert
I did template specialization of basic_text_iprimitive::load for double, and
it works with visual studio 2010 compiler. Do you see something problematic
with this code
namespace boost {
namespace archive {
template<> template<>
void basic_text_iprimitive< std::basic_istream
On 7/4/17 2:48 AM, Elizabeta via Boost-users wrote:
Hi Robert I did template specialization of basic_text_iprimitive::load for double, and it works with visual studio 2010 compiler. Do you see something problematic with this code
namespace boost { namespace archive { template<> template<> void basic_text_iprimitive< std::basic_istream
> >::load<double>(double& t) { if (!is.fail()) { std::wstring s; getline(is, s, L'<'); is.putback(L'<'); int index = s.find(L"IND"); if (index != -1) { t = std::numeric_limits<double>::quiet_NaN(); return; }
index = s.find(L"NAN"); if (index != -1) { t = std::numeric_limits<double>::quiet_NaN(); return; }
index = s.find(L"INF"); if (index != -1) { t = std::numeric_limits<double>::infinity(); return; }
t = std::stod(s); return; }
boost::serialization::throw_exception( archive_exception(archive_exception::input_stream_error) ); } } }
Hmmm - very clever. I have to say the specialization of just one member function is something that would never have occurred to me. I would wonder though. I'm not sure that the Nan literals are standard. I also don't trust them to be portable. Also to me it's a problem with stream i/o rather than serialization itself. So my instinct would have been to maybe make a custom manipulator or something like that. But still, you've got a solution which looks like it will work well for your needs so good luck with this. Robert Ramey
-----Original Message----- From: Boost-users [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Robert Ramey via Boost-users Sent: 04 July 2017 16:05 To: Elizabeta via Boost-users Cc: Robert Ramey Subject: Re: [Boost-users] [serialization]Nans and infinity in wide xml archive
On 7/4/17 2:48 AM, Elizabeta via Boost-users wrote:
Hi Robert I did template specialization of basic_text_iprimitive::load for double, and it works with visual studio 2010 compiler. Do you see something problematic with this code
namespace boost { namespace archive { template<> template<> void basic_text_iprimitive< std::basic_istream
> >::load<double>(double& t) { if (!is.fail()) { std::wstring s; getline(is, s, L'<'); is.putback(L'<'); int index = s.find(L"IND"); if (index != -1) { t = std::numeric_limits<double>::quiet_NaN(); return; }
index = s.find(L"NAN"); if (index != -1) { t = std::numeric_limits<double>::quiet_NaN(); return; }
index = s.find(L"INF"); if (index != -1) { t = std::numeric_limits<double>::infinity(); return; }
t = std::stod(s); return; }
boost::serialization::throw_exception( archive_exception(archive_exception::input_stream_error) ); } } }
Hmmm - very clever. I have to say the specialization of just one member function is something that would never have occurred to me.
I would wonder though. I'm not sure that the Nan literals are standard. I also don't trust them to be portable. Also to me it's a problem with stream i/o rather than serialization itself. So my instinct would have been to maybe make a custom manipulator or something like that. But still, you've got a solution which looks like it will work well for your needs so good luck with this.
http://www.boost.org/doc/libs/1_64_0/libs/math/doc/html/math_toolkit/fp_face... has some info on this: "C99 standard for output of infinity and NaN The C99 standard does specify how infinity and NaN are formatted by printf and similar output functions, and parsed by scanf and similar input functions. The following string representations are used: Table 2.1. C99 Representation of Infinity and NaN number string Positive infinity "inf" or "infinity" Positive NaN "nan" or "nan(...)" Negative infinity "-inf" or "-infinity" Negative NaN "-nan" or "-nan(...)" So following C99 provides a sensible 'standard' way of handling input and output of nonfinites in C++, and this implementation follows most of these formats." And Elizabeta's solution is close to this. Using lower case might become more 'Standard'. I don't know of any progress in a 'Standard' representation in C++14, 17 .... HTH Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830
On 7/4/17 10:01 AM, Paul A. Bristow via Boost-users wrote:
I would wonder though. I'm not sure that the Nan literals are standard. I also don't trust them to be portable. Also to me it's a problem with stream i/o rather than serialization itself. So my instinct would have been to maybe make a custom manipulator or something like that. But still, you've got a solution which looks like it will work well for your needs so good luck with this.
http://www.boost.org/doc/libs/1_64_0/libs/math/doc/html/math_toolkit/fp_face...
has some info on this:
"C99 standard for output of infinity and NaN
The C99 standard does specify how infinity and NaN are formatted by printf and similar output functions, and parsed by scanf and similar input functions.
The following string representations are used:
Table 2.1. C99 Representation of Infinity and NaN
number
string Positive infinity "inf" or "infinity" Positive NaN "nan" or "nan(...)" Negative infinity "-inf" or "-infinity" Negative NaN "-nan" or "-nan(...)"
So following C99 provides a sensible 'standard' way of handling input and output of nonfinites in C++, and this implementation follows most of these formats."
And Elizabeta's solution is close to this. Using lower case might become more 'Standard'.
I don't know of any progress in a 'Standard' representation in C++14, 17 ....
HTH
This is all very interesting! I know some time ago when this problem first came up (I'm always getting the blame for this!) There was some work in addressing it. I didn't take part because I was sort of burned out on working on the serialization library. But I would think it would be a good thing if this could be "resolved" via a combination of documentation and code so provide a canonical C++ solution which would be compatible with just C++ stream i/o and hence automatically apply to the the serialization library. Robert Ramey
participants (3)
-
Elizabeta
-
Paul A. Bristow
-
Robert Ramey