[serialization] [units] class_id and class_name to describe quantity dimension in serilization archive

Hi, I am serializing a bunch of object that contains boost.units quantities. Boost.Units provides serialization for quantities already. By default, the output file looks ok but I would like to add a class name to the object description. Current, serializing quantity<si::length> q produces: <q class_id="0" tracking_level="1" version="0" object_id="_0"> <value>5</value> </q> Which doesn't give any information about the quantity at hand, not even for the human reading it. So, I tried to use BOOST_CLASS_EXPORT_GUID to register a name for the serialized class, but it is ignored. Ideally I would like to have something like this, with a class_name description for example: <q class_id="0" class_name = "quantity_si_length" tracking_level="1" version="0" object_id="_0"> <value>5</value> </q> Is something like this possible? Below is the working code: #include <fstream> #include <boost/serialization/export.hpp> #include <boost/archive/text_oarchive.hpp> #include <boost/archive/xml_oarchive.hpp> #include <boost/units/systems/si/io.hpp> using namespace boost::units; BOOST_CLASS_EXPORT_GUID(quantity<si::length>, "quantity_si_length") //ignored int main(int argc, char *argv[]) { std::ofstream ofs("filename.xml"); boost::archive::xml_oarchive oa(ofs); quantity<si::length> q = 5.*si::meter; oa << BOOST_SERIALIZATION_NVP(q); return 0; }

The BOOST_SERIALIZATION_NVP macro is shorthand for the boost::serialization::make_nvp(string name,object) template function. Easiest thing would be to call it directly: oa << boost::serialization::make_nvp("quantity_si_length",q); Sent from my iPhone On Sep 23, 2011, at 8:50 PM, alfC <alfredo.correa@gmail.com> wrote:
Hi, I am serializing a bunch of object that contains boost.units quantities. Boost.Units provides serialization for quantities already. By default, the output file looks ok but I would like to add a class name to the object description.
Current, serializing quantity<si::length> q produces:
<q class_id="0" tracking_level="1" version="0" object_id="_0"> <value>5</value> </q>
Which doesn't give any information about the quantity at hand, not even for the human reading it. So, I tried to use BOOST_CLASS_EXPORT_GUID to register a name for the serialized class, but it is ignored. Ideally I would like to have something like this, with a class_name description for example:
<q class_id="0" class_name = "quantity_si_length" tracking_level="1" version="0" object_id="_0"> <value>5</value> </q>
Is something like this possible? Below is the working code:
#include <fstream> #include <boost/serialization/export.hpp> #include <boost/archive/text_oarchive.hpp> #include <boost/archive/xml_oarchive.hpp> #include <boost/units/systems/si/io.hpp> using namespace boost::units; BOOST_CLASS_EXPORT_GUID(quantity<si::length>, "quantity_si_length") //ignored
int main(int argc, char *argv[]) { std::ofstream ofs("filename.xml"); boost::archive::xml_oarchive oa(ofs);
quantity<si::length> q = 5.*si::meter;
oa << BOOST_SERIALIZATION_NVP(q);
return 0; }
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Hi, On Friday, September 23, 2011 7:10:48 PM UTC-7, Bob Dean wrote:
The BOOST_SERIALIZATION_NVP macro is shorthand for the boost::serialization::make_nvp(string name,object) template function. Easiest thing would be to call it directly:
oa << boost::serialization::make_nvp("quantity_si_length",q);
sorry if my question was not understood. It is fine (and necessary to have the element named "q"), what I want is that in the class description some information about the type itself, instead of class_id = 0. Such to produce: <*q* class_id="0" class_name = "*quantity_si_length*" tracking_level="1" version="0" object_id="_0"> <value>5</value> </*q*> note that I am fine with the "q" name (either using BOOST_..._NV or make_nvp), I want some (automatic) description of the type not the element. Thank you, Alfredo On Sep 23, 2011, at 8:50 PM, alfC <alfredo...@gmail.com> wrote:
Hi, I am serializing a bunch of object that contains boost.units quantities. Boost.Units provides serialization for quantities already. By default, the output file looks ok but I would like to add a class name to the object description.
Current, serializing quantity<si::length> q produces:
<q class_id="0" tracking_level="1" version="0" object_id="_0"> <value>5</value> </q>
Which doesn't give any information about the quantity at hand, not even for the human reading it. So, I tried to use BOOST_CLASS_EXPORT_GUID to register a name for the serialized class, but it is ignored. Ideally I would like to have something like this, with a class_name description for example:
<q class_id="0" class_name = "quantity_si_length" tracking_level="1" version="0" object_id="_0"> <value>5</value> </q>
Is something like this possible? Below is the working code:
#include <fstream> #include <boost/serialization/export.hpp> #include <boost/archive/text_oarchive.hpp> #include <boost/archive/xml_oarchive.hpp> #include <boost/units/systems/si/io.hpp> using namespace boost::units; BOOST_CLASS_EXPORT_GUID(quantity<si::length>, "quantity_si_length") //ignored
int main(int argc, char *argv[]) { std::ofstream ofs("filename.xml"); boost::archive::xml_oarchive oa(ofs);
quantity<si::length> q = 5.*si::meter;
oa << BOOST_SERIALIZATION_NVP(q);
return 0; }
_______________________________________________ Boost-users mailing list Boost...@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Boost-users mailing list Boost...@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

alfC wrote:
Hi, I am serializing a bunch of object that contains boost.units quantities. Boost.Units provides serialization for quantities already. By default, the output file looks ok but I would like to add a class name to the object description.
Current, serializing quantity<si::length> q produces:
<q class_id="0" tracking_level="1" version="0" object_id="_0"> <value>5</value> </q>
Which doesn't give any information about the quantity at hand, not even for the human reading it. So, I tried to use BOOST_CLASS_EXPORT_GUID to register a name for the serialized class, but it is ignored. Ideally I would like to have something like this, with a class_name description for example:
<q class_id="0" class_name = "quantity_si_length" tracking_level="1" version="0" object_id="_0"> <value>5</value> </q>
Is something like this possible? Below is the working code:
#include <fstream> #include <boost/serialization/export.hpp> #include <boost/archive/text_oarchive.hpp> #include <boost/archive/xml_oarchive.hpp> #include <boost/units/systems/si/io.hpp> using namespace boost::units; BOOST_CLASS_EXPORT_GUID(quantity<si::length>, "quantity_si_length") //ignored
int main(int argc, char *argv[]) { std::ofstream ofs("filename.xml"); boost::archive::xml_oarchive oa(ofs);
quantity<si::length> q = 5.*si::meter;
oa << BOOST_SERIALIZATION_NVP(q);
return 0; }
This is a feature of the xml_?archive implementation. In general, the archive implementations include the minimum information required to re-constitute the original data - and no more. Robert Ramey

Hi Robert, On Sat, Sep 24, 2011 at 10:08 AM, Robert Ramey <ramey@rrsd.com> wrote:
Current, serializing quantity<si::length> q produces:
<q class_id="0" tracking_level="1" version="0" object_id="_0"> <value>5</value> </q>
Which doesn't give any information about the quantity at hand, not even for the human reading it. So, I tried to use BOOST_CLASS_EXPORT_GUID to register a name for the serialized class, but it is ignored. This is a feature of the xml_?archive implementation. In general, the archive implementations include the minimum information required to re-constitute the original data - and no more.
Ah! that may also explain why in all the examples I see of BOOST_CLASS_EXPORT, it only worked (saved a string with the name of the class) for derived classes saves from their base class. Good. Having settled on that, I think it would be cool to have the *option* to save the class name (or other tag information) with the object. Just a suggestion for Boost.Serialization.XML I think of the quantity example because all quantities saved to xml_archive look exactly the same regardless of whether they are the same dimension or not, so it exists the possibility that quantity<si::length> is saved, but then read as quantity<si::time>. Now about a workaround from the point of view of Boost.Units, suppose I want to ensure that additional information (spurious for reconstruction but useful as a check and human reader) with the current implementation. Currently the serialization of boost.unit.quantity is: // boost/units/io.hpp template<class Archive,class Unit,class Y> inline void serialize(Archive& ar,boost::units::quantity<Unit,Y>& q,const unsigned int /*version*/){ ar & boost::serialization::make_nvp("value", q.value); //basically this in the code } if changed to template<class Archive,class Unit,class Y> inline void serialize(Archive& ar,boost::units::quantity<Unit,Y>& q,const unsigned int /*version*/){ ar & boost::serialization::make_nvp("value", q.value); //basically this in the code #ifdef BOOST_UNIT_SERIALIZATION_ADD_INFO std::string unit_info = boost::to_string(Unit()); ar & boost::serialization::make_nvp("unit", unit_info ); #ifdef BOOST_UNIT_SERIALIZATION_CHECK_INFO assert( unit_info == boost::to_string(Unit()); // or throw SOMETHING (see below)? #endif #endif } then I expect the output to be: <q class_id="0" tracking_level="1" version="0" object_id="_0"> <value>5</value> <unit>m</unit> </q> Which provides information to the human and and also can be used do a basic check at reading the archive. The logic would be that nobody ever will want to read a saved quantity of a given system and dimension as another quantity in anothe system or dimension. In this case this is a suggestion for the Boost.Units serialization implementation. If double check is not desired the human can still benefit, as it can if the tag had the following info <q unit = "m" class_id="0" tracking_level="1" version="0" object_id="_0"> BTW, Robert, what would be the canonical exception to throw from the assert line in the proposed code (which I plan to implement internally)? something that can by useful for the error handling of boost.serialization? (I now understand that boost.serialization was not designed with this in mind, but I need to do something like this to add this little piece of information to the xml.) Thanks, Alfredo

Hi
On 25/09/2011 00:08, Alfredo Correa wrote:
Good. Having settled on that, I think it would be cool to have the *option* to save the class name (or other tag information) with the object. Just a suggestion for Boost.Serialization.XML
I would suggest that nothing should be changed in the core Boost/Serialization XML lib, even if what you propose is fancy and even if it is an option (who would choose the "name of the class" to be saved ? How to make sure that the convention for class naming is the same while reading and writing ? I guess it addresses some new problems...). Rather than acking the BS lib, what you asked for could be probably easily implemented by some specific I/O manager dedicated for your specific need during I/O operations. When I serialize some heterogeneous sets of objects in any kind of archives (text/XML/eos::portable_binary), I first store/load a "serialization tag" (std::string) that identifies the type/class of the next object to be stored|loaded. It is thus my responsability to manage a set of known tags associated to the classes I need to serialize. So it turns out that any serializable data is confined in some kind of virtual container, together with some kind of class ID: <archive> serial_tag #0 data_record #0 serial_tag #1 data_record #1 ... serial_tag #N data_record #N </archive> There is also no problem to add meta-informations if needed <archive> serial_tag #0 meta_info #0 data_record #0 ... </archive> Of course, it implies some smarter management of the I/O operations and some hard-coded factory (in a basic implementation). But it is very convenient for debugging and compatible with all kind of archives. This mechanism is clearly a brute force approach and maybe some people consider this as inelegant. Anyway I have used this in production code for years and it is very robust. Note that it is different from the BOOST_CLASS_EXPORT mechanism which implies all your objects must be serialized by a base pointer and inherited from a unique abstract class/interface.
I think of the quantity example because all quantities saved to xml_archive look exactly the same regardless of whether they are the same dimension or not, so it exists the possibility that quantity<si::length> is saved, but then read as quantity<si::time>.
Now about a workaround from the point of view of Boost.Units, suppose I want to ensure that additional information (spurious for reconstruction but useful as a check and human reader) with the current implementation. Currently the serialization of boost.unit.quantity is:
// boost/units/io.hpp template<class Archive,class Unit,class Y> inline void serialize(Archive& ar,boost::units::quantity<Unit,Y>& q,const unsigned int /*version*/){ ar& boost::serialization::make_nvp("value", q.value); //basically this in the code }
Could it be possible for you to use a wrapper class with the additional meta-info you need rather than to rely on some intrusive modification of the core functionality of the lib ? struct quantity_wrapper<...> { boost::units::quantity<...> q; std::string more_info; void serialize(...); }; cheers frc -- -- François Mauger Groupe "Interactions Fondamentales et Nature du Neutrino" NEMO-3/SuperNEMO Collaboration LPC Caen-CNRS/IN2P3-UCBN-ENSICAEN Département de Physique -- Université de Caen Basse-Normandie Adresse/address: Laboratoire de Physique Corpusculaire de Caen (UMR 6534) ENSICAEN 6, Boulevard du Marechal Juin 14050 CAEN Cedex FRANCE Courriel/e-mail: mauger@lpccaen.in2p3.fr Tél./phone: 02 31 45 25 12 / (+33) 2 31 45 25 12 Fax: 02 31 45 25 49 / (+33) 2 31 45 25 49

Hi François,
I would suggest that nothing should be changed in the core Boost/Serialization XML lib,
Sure. I am just seen what is the consensus on this. The suggestion was only based in the fact that the implementation of boost.serialization.XML was based on that XML is more human readable than text files. Now I will continue answering your questions.
even if what you propose is fancy and even if it is an option (who would choose the "name of the class" to be saved ?
In the same way "name of the class" is chosen arbitrarily for derived-from-base class during serialization. Furthermore it can be ignored on reading.
How to make sure that the convention for class naming is the same while reading and writing ? I guess it addresses some new problems...).
In the same way the convention for class naming is the same for derived-from-base classes. Again, it can be ignored on reading.
Rather than acking the BS lib, what you asked for could be probably easily implemented by some specific I/O manager dedicated for your specific need during I/O operations.
the "hacking" I suggested (i.e. the explicit code) does not hack Boost.Serialization but it hacks Boost.Units.serialization, and it is within the framework of Boost.Serialization. (the standalone serialization/load/save function can be anything the user wants... I guess, although I was not sure if it should throw or not)
When I serialize some heterogeneous sets of objects in any kind of archives (text/XML/eos::portable_binary), I first store/load a "serialization tag" (std::string) that identifies the type/class of the next object to be stored|loaded. It is thus my responsability to manage a set of known tags associated to the classes I need to serialize.
Do you mean something like this? void serialize(Archive & ar, Object obj, unsigned){ ar & typeof(obj.a).name(); // just one way to store type information, check enforce can be added in the same way I did in my quantity example ar & obj.a; ar & "type of b"; // just one way to store type information ar & obj.b; } It is actually a good idea, Thanks!. (It is indeed similar to what I did by storing the information but only inside the tag for boost.quantity.)
So it turns out that any serializable data is confined in some kind of virtual container, together with some kind of class ID: <archive> serial_tag #0 data_record #0 (snip) Of course, it implies some smarter management of the I/O operations and some hard-coded factory (in a basic implementation). But it is very convenient for debugging and compatible with all kind of archives.
I got lost with the "hard-coded factory" but I get the idea.
This mechanism is clearly a brute force approach and maybe some people consider this as inelegant. Anyway I have used this in production code for years and it is very robust.
Note that it is different from the BOOST_CLASS_EXPORT mechanism which implies all your objects must be serialized by a base pointer and inherited from a unique abstract class/interface.
Yes, I noticed the difference. In fact my suggestion was based in BOOST_CLASS_EXPORT but for cases where 1) the class name is not really necesary, e.g. there is not derived pointer 2) object serialized is not a pointer even. I liked the fact that the type information in the currently implemention appears in the xml tag, seemed elegant.
Could it be possible for you to use a wrapper class with the additional meta-info you need rather than to rely on some intrusive modification of the core functionality of the lib ?
I didn't investigate the use of wrappers but seems logical if the core library can not be changed. I found the description of wrapper http://www.boost.org/doc/libs/1_47_0/libs/serialization/doc/wrappers.html quite obfuscated
struct quantity_wrapper<...> { boost::units::quantity<...> q; std::string more_info; void serialize(...); };
I see, that is the wrapper, what next? do I replace quantity_wrapper in all the classes where quantity is a member? class Object{ quantity_wrapper< dim1 > a; // was quantity< dim1 > quantity_wrapper< dim2 > b; // was quantity< di21 > }; or do I serialize my class creating temporary quantity objects, don't I need to use a wrapper of reference instead? struct quantity_wrapper< Unit>{ boost::units::quantity< Unit >& q; <--- reference ??? std::string more_info; quantity_wrapper< Unit >(quantity< Unit > q_) : q(q_), more_info(to_string(Unit())){} void serialize(...); }; and then serialize(Archive ar, Object o, unsigned){ ar & quantity_wrapper<dim1>(o.a); // a is quantity<dim1> ar & quantity_wrapper<dim2 >(o.b); // b is quantity< dim2 > } do I need to specialize is_wrapper<quantity_wrapper< ... > >::value is true? Thank you François, your post was very helpful to clarify the ideas. I need to workout the details now. Alfredo

Alfredo Correa wrote:
Hi François,
Rather than acking the BS lib, what you asked for could be probably easily implemented by some specific I/O manager dedicated for your specific need during I/O operations.
the "hacking" I suggested (i.e. the explicit code) does not hack Boost.Serialization but it hacks Boost.Units.serialization, and it is within the framework of Boost.Serialization. (the standalone serialization/load/save function can be anything the user wants... I guess, although I was not sure if it should throw or not)
FYI, the serialization library doesn't impose any requirments as to how any particular type should be serialized, other than the the requirement that the save/load be symetric. So if you want to make you're own implementation of the serialization of the units library it's just fine. Just put it in to a separate header ../correat/serialization/units.hpp and you're in business.
When I serialize some heterogeneous sets of objects in any kind of archives (text/XML/eos::portable_binary), I first store/load a "serialization tag" (std::string) that identifies the type/class of the next object to be stored|loaded. It is thus my responsability to manage a set of known tags associated to the classes I need to serialize.
hmm - I just use variant with all the possible types and just serialize that - which is already in boost. Also serialization of a pointer through the base class addresses this issue in other contexts.
Do you mean something like this?
void serialize(Archive & ar, Object obj, unsigned){ ar & typeof(obj.a).name(); // just one way to store type information, check enforce can be added in the same way I did in my quantity example ar & obj.a; ar & "type of b"; // just one way to store type information ar & obj.b; }
My preference is more something like ar & boost::variant<....>(a) ... But not to handle a pointer to an "a" would require usage of "set_object_address" call (hack) to work. So basically, your problem is too many options that you have to look at. Robert Ramey
participants (5)
-
alfC
-
Alfredo Correa
-
Bob Dean
-
François Mauger
-
Robert Ramey