Re: [Boost-users] [serialization] I can't figure out what I'm doing wrong with xml_oarchive - please help
Hello Robert, Thank you for the quick response. I'm glad you liked the title ... A) I did replace all BOOST_CLASS_EXPORT with BOOST_CLASS_EXPORT_GUID, but the problem with xml persists. It is really difficult for me to figure out what is wrong with xml, as both txt and bin versions are ok. Actually what worries me, is the possibility that the problem with xml somehow will show up in other formats too ... B) I don't use the version numbers at the moment but I added it just for future use. I actually have a couple of questions here. 1) BOOST_CLASS_VERSION has to be in the header file? Can I place it in the cpp file? 2) Please correct me If I'm wrong: I don't use BOOST_CLASS_VERSION so that all classes have version number assignment set to 0. I also don't use BOOST_SERIALIZATION_SPLIT_MEMBER for now. Later on, I add some members, so that the class version must change. I simply add BOOST_CLASS_VERSION(..., 1) and split serialize to save/load at that time. Previously saved data files are not affected as long as version info is handled correctly by my code. Is that correct? C) I avoid using templated code in CTCDataBase for as long as the core data that must be serialized evolve, in order to avoid frequently modification of the most included header file TC_database.h. That meens almost recompile all, and that takes time. Thank you for the suggestion any way. I'll keep it in mind. To be honest, I did try to use export feature of templates bug no luck ... D) I re-attach a couple of header files to this post and I kindly ask you to take a look at the implementation and tracking level for template class fem::CPropRepository implementation_level< fem::CPropRepository<T, SZ> > and tracking_level< fem::CPropRepository<T, SZ> >, and one utilization of it at the other header. Should it be so? Do I need it? And also is it possible that using this CPropRepository stuff (especially with MFC CString) breaks the xml code? E) After adding all headers of the serialize library compilation takes a lot more time. Are there any guidelines by you on that? I hope I'm not encroaching your time... Thank you again, Emmanuil Tsagarakis
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Robert Ramey Sent: Friday, May 05, 2006 8:36 AM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [boost-users] [serialization] I can't figure outwhatI'm doing wrong with xml_oarchive - please help
LOL - I liked the title.
I've taken a very quick look at your code and files.
a) I'm a little suspicious of including a ":" character as the GUID for a class. I think that might be an issue. Try replacing
BOOST_CLASS_EXPORT(fem::CTCSoil);
with
BOOST_CLASS_EXPORT_GUID(fem::CTCSoil, "fem_CTCSoil");
You'll have to double check the exact syntax.
b) You've set version numbers to 1 but I don't see the version numbers used in the deserializations. Version number assignment is only necessary if you're going to use the version numbers and they are not equal to 0.
c) A minor suggestion - not important here but interesting anyway. In fem_boost_serialize.cpp Use templates to shorten the code.
template<class Archive> void CTCDataBase::boost_serialize_open(const char* filename) { try { std::ifstream ifs(filename); assert(ifs.good()); Archive ia(ifs);
ia >> BOOST_SERIALIZATION_NVP(m_lstSoils); ia >> BOOST_SERIALIZATION_NVP(m_lstBrickWalls); ia >> BOOST_SERIALIZATION_NVP(CTCFooting::s_bspSoil); ia >> BOOST_SERIALIZATION_NVP(CTCFrame::s_bspBrickWall); } catch (const boost::archive::archive_exception& e) { e; } }
and template<class Archive> void CTCDataBase::boost_serialize_saveconst char* filename) { try { std::ofstream ofs(filename); assert(ofs.good()); Archive oa(ofs);
oa << BOOST_SERIALIZATION_NVP(m_lstSoils); oa << BOOST_SERIALIZATION_NVP(m_lstBrickWalls); oa << BOOST_SERIALIZATION_NVP(CTCFooting::s_bspSoil); oa << BOOST_SERIALIZATION_NVP(CTCFrame::s_bspBrickWall); } catch (const boost::archive::archive_exception& e) { e; } }
void main() { fill_with_build_in(); boost_serialize_save<boost::archive::text_oarchive>("boost _serialize.txt"); boost_serialize_save<boost::archive::binary_oarchive>("boo st_serialize.bin"); boost_serialize_save<boost::archive::xml_oarchive>("boost_ serialize.xml");
// some time later ... boost_serialize_open<boost::archive::text_oarchive>("boost _serialize.txt"); boost_serialize_open<boost::archive::binary_oarchive>("boo st_serialize.bin"); boost_serialize_open<boost::archive::xml_oarchive>("boost_ serialize.xml"); }
Robert Ramey
Hi to all,
This is my very first post to boost-users! I've been using a couple of boost libraries for a while (shared_ptr & bind mostly). Write now we are in the process of transforming a project based on MFC collections, (with objects derived from CObject), to STL, and doing some search for a library that will allow object
MFC does, the obvious solution was boost::serialize. As I'm a newbie in mpl style code the first impression was terrifying! But after reading the documentation and experimenting a lot, I did manage to start using boost::serialize. In fact there is no need to be familiar with mpl. The library is excellent - an exemplary work, thank you Robert Ramey!
Now my problem is with xml files. I can write a couple of classes both in txt and binary files. But I can't write the very same data in xml format. I believe I followed all guidelines but I had no luck ... :(
The scenario in pseudo code is this:
////////////////////////////////////////////////////////// ///////////
/ // file fem.h #include <boost/archive/...> #include <boost/serialize/...>
namespace fem {
class CTCBase { ... }; typedef boost::shared_ptr<CTCBase> CBSPBase typedef std::list<CBSPBase> CBSPBaseList
class CTCSoil { ... }; typedef boost::shared_ptr<CTCSoil> CBSPSoil typedef std::list<CBSPSoil> CBSPSoilList
class CTCBrickWall { ... }; typedef boost::shared_ptr<CTCBrickWall> CBSPBrickWall typedef std::list<CBSPBrickWall> CBSPBrickWallList
// list of soils CBSPSoilList s_lstSoils; // list of Brick Walls CBSPBrickWallList s_lstBrickWalls; // initial soil for each new foundation beam (also contained in above list) CBSPSoil s_bspSoil; // initial brick wall on each beam (also contained in above list) CBSPBrickWall s_bspBrick; }
BOOST_CLASS_VERSION(fem::CTCBase, 1); BOOST_CLASS_VERSION(fem::CTCSoil, 1); BOOST_CLASS_VERSION(fem::CTCBrickWall, 1);
////////////////////////////////////////////////////////// ///////////
/ // file fem.cpp
#include "fem.h" #include <boost/serialization/export.hpp>
BOOST_IS_ABSTRACT(fem::CTCBaseObj); BOOST_CLASS_EXPORT(fem::CTCSoil); BOOST_CLASS_EXPORT(fem::CTCBrickWall);
void fill_with_build_in() { ... }
void main() { fill_with_build_in(); boost_serialize_save_txt("boost_serialize.txt"); boost_serialize_save_bin("boost_serialize.bin"); boost_serialize_save_xml("boost_serialize.xml");
// some time later ... boost_serialize_open_txt("boost_serialize.txt"); boost_serialize_open_bin("boost_serialize.bin"); boost_serialize_open_xml("boost_serialize.xml"); }
The xml save & open functions are in file fem_boost_serialize.cpp
I attached the 3 files produced this way. I can also read back txt & binary versions. I'm wondering if somebody - hopefully Robert Ramey! - can tell me just by examining the output files, where should I look for the problem. Or do I have to debug the code line by line and see what wend wrong? It seems something is wrong with the very first shared_ptr serialization .
I include some - not stand alone I'm afraid - source code files with this message just in case someone might take a look at
tell that during the transition period from MFC to STL (possibly after that too), our objects derive from MFC CObject. Also at the moment, we still use MFC CString and not std::string. I had to copy CString to std::string for the serialization to work
TOL mtsagara wrote: persistence the way them. I can properly. Could
there be a conflict somewhere there? Could it be the order in hpp files are included?
Finally I use MS Visual C++ .net 2003 and boost v1.33.1.
Thank you in advanced,
Emmanuil Tsagarakis
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Manos Tsagarakis wrote:
Hello Robert,
Thank you for the quick response.
I'm glad you liked the title ...
A) I did replace all BOOST_CLASS_EXPORT with BOOST_CLASS_EXPORT_GUID, but the problem with xml persists. It is really difficult for me to figure out what is wrong with xml, as both txt and bin versions are ok. Actually what worries me, is the possibility that the problem with xml somehow will show up in other formats too ...
I not sure that the exported name can have a ':' in it without tripping up XML If that's not it I'm not sure what it is.
B) I don't use the version numbers at the moment but I added it just for future use. I actually have a couple of questions here. 1) BOOST_CLASS_VERSION has to be in the header file? Can I place it in the cpp file? 2) Please correct me If I'm wrong: I don't use BOOST_CLASS_VERSION so that all classes have version number assignment set to 0. I also don't use BOOST_SERIALIZATION_SPLIT_MEMBER for now. Later on, I add some members, so that the class version must change. I simply add BOOST_CLASS_VERSION(..., 1) and split serialize to save/load at that time. Previously saved data files are not affected as long as version info is handled correctly by my code. Is that correct?
Correct - BTW you don't necessarily have to split but in many cases it would be the easiest thing to do..
C) I avoid using templated code in CTCDataBase for as long as the core data that must be serialized evolve, in order to avoid frequently modification of the most included header file TC_database.h. That meens almost recompile all, and that takes time. Thank you for the suggestion any way. I'll keep it in mind. To be honest, I did try to use export feature of templates bug no luck ...
The way to do it is to leave the template definition in a header and an explicity instanticiation in a *.cpp file. That is how the serialization library diminishes compilation.
D) I re-attach a couple of header files to this post and I kindly ask you to take a look at the implementation and tracking level for template class fem::CPropRepository implementation_level< fem::CPropRepository<T, SZ> > and tracking_level< fem::CPropRepository<T, SZ> >, and one utilization of it at the other header. Should it be so? Do I need it? And also is it possible that using this CPropRepository stuff (especially with MFC CString) breaks the xml code?
A fundemental problem here: //either use macro definitions: //BOOST_CLASS_IMPLEMENTATION(nvp<T>, boost::serialization::level_type::object_serializable) //BOOST_CLASS_TRACKING(nvp<T>, boost::serialization::track_never) the arguments can't be templates but must be expanded types. This is described in the manual - I forget where. If you need this functionality, you can't use the macro. You have to copy the original definition and make the substitution by hand. If you look at the macro definition and "expand it by hand" you'll see that it won't result in valid code.
E) After adding all headers of the serialize library compilation takes a lot more time. Are there any guidelines by you on that?
Here's what I recommend. Look at the demo/example/test demo_pimpl. This shows how one can separate the instantiated serialization code into a separate *.cpp module. This will mean that its only compiled when the serialization changes. In fact the best way is to add all the *.cpp modules for all archives in a library. That way they are almost never compiled and executables only contain the stuff actually used. Robert Ramey
Hello Robert, Thank you again for your help ... I did follow your suggestions especially with explicitly instantiation in a *.cpp file. Your example demo/example/test demo_pimpl helped a lot. On the other hand, I still can't save in xml format. I don't get any exception. Do I have to have any specific xml parser installed or not? Anyway I think I'm staying to text format for now... The most severe problem at the moment is that I keep experiencing in multiple files a compiler error (MSVC 7.1): fatal error C1055: compiler limit : out of keys It turns out that the cause, is the macro defined at the top of each cpp file: #ifdef _DEBUG #define new DEBUG_NEW #endif And also some macros like ASSERT or even CRT assert causes the above error. According to ms help: "The source file contains too many symbols. The compiler ran out of hash keys for the symbol table. Possible solutions Split the source file into smaller files. Eliminate unnecessary header files. Reuse temporary and global variables instead of creating new ones." But the error occurs even in a file with a single function! Cancelling the above macros bypasses all such errors. Any ideas on that? Diagnostic messages that DEBUG_NEW gives at program exit are useful for locating memory leaks ... But I guess that as long as I don't use raw pointers and stick to boost::shared_ptr I don't need the above macros ... One more question, that's probably irrelevant to the library. So far with MFC I'm used to use macros like _T("...") or _TEXT("...") for string literals and character constants ... Aren't such macros needed with std::string? The following are copied from Microsoft's <tchar.h>: // Generic text macros to be used with string literals and character constants. // Will also allow symbolic constants that resolve to same. #ifdef _UNICODE // ++++++++++++++++++++ UNICODE ++++++++++++++++++++ #define __T(x) L ## x #else /* ndef _UNICODE */ // ++++++++++++++++++++ SBCS and MBCS ++++++++++++++++++++ #define __T(x) x #endif #define _T(x) __T(x) #define _TEXT(x) __T(x) Finally again with question (D) below: the real question is: do I need the functionality of BOOST_CLASS_IMPLEMENTATION and/or BOOST_CLASS_TRACKING for a class like this one? template<class T, int SZ> class CPropRepository { public: //uniform typedefs used in STL algorithms and containers typedef T value_type; typedef T* iterator; typedef const T * const_iterator; typedef T& reference; typedef const T& const_reference; ... private: value_type _t[SZ]; friend class boost::serialization::access; template<class Archive> void serialize(Archive & ar, const unsigned int /*version*/) { ar & BOOST_SERIALIZATION_NVP(_t); } }; Used like this: class A { CPropRepository<int, 2> m_intProps; friend class boost::serialization::access; template<class Archive> void serialize(Archive & ar, const unsigned int /*version*/) { ar & BOOST_SERIALIZATION_NVP(m_intProps); } public: ... }; Thank you again, Emmanuil Tsagarakis Manos Tsagarakis wrote:
Hello Robert,
Thank you for the quick response.
I'm glad you liked the title ...
A) I did replace all BOOST_CLASS_EXPORT with BOOST_CLASS_EXPORT_GUID, but the problem with xml persists. It is really difficult for me to figure out what is wrong with xml, as both txt and bin versions are ok. Actually what worries me, is the possibility that the problem with xml somehow will show up in other formats too ...
I not sure that the exported name can have a ':' in it without tripping up XML If that's not it I'm not sure what it is.
B) I don't use the version numbers at the moment but I added it just for future use. I actually have a couple of questions here. 1) BOOST_CLASS_VERSION has to be in the header file? Can I place it in the cpp file? 2) Please correct me If I'm wrong: I don't use BOOST_CLASS_VERSION so that all classes have version number assignment set to 0. I also don't use BOOST_SERIALIZATION_SPLIT_MEMBER for now. Later on, I add some members, so that the class version must change. I simply add BOOST_CLASS_VERSION(..., 1) and split serialize to save/load at that time. Previously saved data files are not affected as long as version info is handled correctly by my code. Is that correct?
Correct - BTW you don't necessarily have to split but in many cases it would be the easiest thing to do..
C) I avoid using templated code in CTCDataBase for as long as the core data that must be serialized evolve, in order to avoid frequently modification of the most included header file TC_database.h. That meens almost recompile all, and that takes time. Thank you for the suggestion any way. I'll keep it in mind. To be honest, I did try to use export feature of templates bug no luck ...
The way to do it is to leave the template definition in a header and an explicity instanticiation in a *.cpp file. That is how the serialization library diminishes compilation.
D) I re-attach a couple of header files to this post and I kindly ask you to take a look at the implementation and tracking level for template class fem::CPropRepository implementation_level< fem::CPropRepository<T, SZ> > and tracking_level< fem::CPropRepository<T, SZ> >, and one utilization of it at the other header. Should it be so? Do I need it? And also is it possible that using this CPropRepository stuff (especially with MFC CString) breaks the xml code?
A fundemental problem here: //either use macro definitions: //BOOST_CLASS_IMPLEMENTATION(nvp<T>, boost::serialization::level_type::object_serializable) //BOOST_CLASS_TRACKING(nvp<T>, boost::serialization::track_never) the arguments can't be templates but must be expanded types. This is described in the manual - I forget where. If you need this functionality, you can't use the macro. You have to copy the original definition and make the substitution by hand. If you look at the macro definition and "expand it by hand" you'll see that it won't result in valid code.
E) After adding all headers of the serialize library compilation takes a lot more time. Are there any guidelines by you on that?
Here's what I recommend. Look at the demo/example/test demo_pimpl. This shows how one can separate the instantiated serialization code into a separate *.cpp module. This will mean that its only compiled when the serialization changes. In fact the best way is to add all the *.cpp modules for all archives in a library. That way they are almost never compiled and executables only contain the stuff actually used. Robert Ramey _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
TOL: Emmanuil Tsagarakis wrote:
Hello Robert,
Thank you again for your help ...
I did follow your suggestions especially with explicitly instantiation in a *.cpp file. Your example demo/example/test demo_pimpl helped a lot. On the other hand, I still can't save in xml format. I don't get any exception. Do I have to have any specific xml parser installed or not?
Anyway I think I'm staying to text format for now...
The most severe problem at the moment is that I keep experiencing in multiple files a compiler error (MSVC 7.1):
fatal error C1055: compiler limit : out of keys
It turns out that the cause, is the macro defined at the top of each cpp file:
#ifdef _DEBUG #define new DEBUG_NEW #endif
And also some macros like ASSERT or even CRT assert causes the above error.
According to ms help: "The source file contains too many symbols. The compiler ran out of hash keys for the symbol table. Possible solutions Split the source file into smaller files. Eliminate unnecessary header files. Reuse temporary and global variables instead of creating new ones."
But the error occurs even in a file with a single function!
Cancelling the above macros bypasses all such errors. Any ideas on that? Diagnostic messages that DEBUG_NEW gives at program exit are useful for locating memory leaks ... But I guess that as long as I don't use raw pointers and stick to boost::shared_ptr I don't need the above macros ...
I think a better strategy is to separte you source code in more modules. You can have one header class.hpp an several implementation modules class1.cpp, class2.cpp etc. The DEBUG_NEW can be excluded from those modules which don't use MFC. Personally I would be wary of mixing MFC and boost without taking the above precautions. It will make your projects compile and build much faster since not all the modules need include windows.h and all the mfc stuff.
One more question, that's probably irrelevant to the library. So far with MFC I'm used to use macros like _T("...") or _TEXT("...") for string literals and character constants ... Aren't such macros needed with std::string? The following are copied from Microsoft's <tchar.h>:
// Generic text macros to be used with string literals and character constants. // Will also allow symbolic constants that resolve to same. #ifdef _UNICODE // ++++++++++++++++++++ UNICODE ++++++++++++++++++++ #define __T(x) L ## x #else /* ndef _UNICODE */ // ++++++++++++++++++++ SBCS and MBCS ++++++++++++++++++++ #define __T(x) x #endif #define _T(x) __T(x) #define _TEXT(x) __T(x)
you could use the above - I think it would work but you might have to include some extra sauce to generate stringw objects rather then CString, etc
Finally again with question (D) below: the real question is: do I need the functionality of BOOST_CLASS_IMPLEMENTATION and/or BOOST_CLASS_TRACKING for a class like this one?
Generally should should need the above only in special cases. I would expect very few users - maybe 1 percent to ever need either of these.
participants (3)
-
Manos Tsagarakis -
Robert Ramey -
TOL: Emmanuil Tsagarakis