[serialzation] Portable binary archives
Hi, I was just wondering if there is any chance of having portable binary archives soon again, or to which version of boost I have to revert to use those in the vault. I am just asking as I want to javatize some of my work for another project, and I would like to use boost binary archives for portable serialization (portability of the data is really vital for this project). Best Daniel
boost-users-bounces@lists.boost.org wrote:
I was just wondering if there is any chance of having portable binary archives soon again, or to which version of boost I have to revert to use those in the vault. I am just asking as I want to javatize some of my work for another project, and I would like to use boost binary archives for portable serialization (portability of the data is really vital for this project).
Dear Daniel! Here is what I can tell you: 1. using boost version 1.33.1 you're on the safe side. this version has been thoroughly tested and the archives perform well in our setting (ppc-32 and x86-32) 2. using boost version 1.34.1 should do the same although we have a not yet solved rupture in downwards compatibility. if you need to read in files generated by portable_binary_oarchive using 1.33.1 you are likely to fail. 3. boost version 1.35 has not been tested so far. however i have a version of the archives that compiles and _seems_ to work... if you're willing to test it i can send you the code by email. 4. Remember, the portable binary archives are not from the author of boost::serialization and will not be supported by him. So if you plan to rely on it you will be on your own. However, you're welcome to use it :-) and I shall try to help you as much as I can. 5. If speed is less important for your application (than for ours) consider using text archives instead. those are designed for portability. Regards, -- Christian Pfligersdorffer Software Engineering http://www.eos.info
Hi Christian, Daniel and all, I'm also interested in portable binary archive format. Unfortunately, I have no time to extend the example provided for integers within 1.33. Do you mean that in the 1.35 unreleased work you have at hand, you can handle floats (32bits) and doubles (64bits) using underlying IEEE standard and aware of endianess? regards frc
5. If speed is less important for your application (than for ours) consider using text archives instead. those are designed for portability.
note that for very critical applications (computations very sensitive to precision) for the last digit in text-serialized doubles is unsafe. -- Francois Mauger Laboratoire de Physique Corpusculaire de Caen et Universite de Caen ENSICAEN - 6, Boulevard du Marechal Juin, 14050 CAEN Cedex, FRANCE e-mail: mauger@lpccaen.in2p3.fr tel.: (0/+33) 2 31 45 25 12 fax: (0/+33) 2 31 45 25 49
Francois Mauger wrote:
I'm also interested in portable binary archive format. Unfortunately, I have no time to extend the example provided for integers within 1.33.
Dear Francois! There's no need for you doing so since that's exactly what we've already done. Go and grab it from the serialization folder in the boost vault: www.boost-consulting.com/vault/
Do you mean that in the 1.35 unreleased work you have at hand, you can handle floats (32bits) and doubles (64bits) using underlying IEEE standard and aware of endianess?
Yes, by using the 'Integer' and 'Floating Point Utilities' libraries, also available from the vault. Really, everything was already there, it just had to be put together by someone... In November Robert Ramey announced he would release a commercially sponsored portable binary archive with one of the next versions: http://article.gmane.org/gmane.comp.lib.boost.user/31851 - I do not know what came out of that but I thought this might be of interest. Regards, -- Christian Pfligersdorffer Software Engineering http://www.eos.info
Hi again Christian, I just downloaded the stuff and run a test prog, it seems to be exactly what I need. I'm now creating a wrapper package to handle the whole stuff and make it easy to enter my package and build system. Of course I need to do more tests under different Linux flavors using 32 and 64 bits architectures (and also MacOS X) but it already looks great! Many thanks for this work (and underlying stuff by other people). frc -- Francois Mauger Laboratoire de Physique Corpusculaire de Caen et Universite de Caen ENSICAEN - 6, Boulevard du Marechal Juin, 14050 CAEN Cedex, FRANCE e-mail: mauger@lpccaen.in2p3.fr tel.: (0/+33) 2 31 45 25 12 fax: (0/+33) 2 31 45 25 49
The original design was made as a demonstration of how to derive from an existing archive class to make a new one. Unfortunately, this example was broken by changes to binary_?archive on which it was based. So I made a new correct version of portable binary archive and its in the trunk. It has been tested with a large variety of compilers. The tests also included writing archives on one machine and reading back on a different architecture. This my best shot for portable binary archive so far. Many thanks to Cadence Design (Israel) for supporting this work and permiting it to be included in Boost. What's missing is the following: a) add floating point types. We now we have a portable floating point utilities library that will be helpful. There have been a couple of good suggestions for a portable floating point format but I haven't seen any implemented. There are number of questions and decisions here - use XDR primitves, limit to IEEE, etc. which many will want to chime in on. b) I need to remove the "case study" which describes portable binary archive and replace it with a more instructive one - like using derivation to add your own special attribute to an xml_archive or something like that. Actually - these might be interesting GSoC projects. They are actually small enough to be completed in a summer. (well b) is anyway) Robert Ramey François Mauger wrote:
Hi again Christian,
I just downloaded the stuff and run a test prog, it seems to be exactly what I need. I'm now creating a wrapper package to handle the whole stuff and make it easy to enter my package and build system.
Of course I need to do more tests under different Linux flavors using 32 and 64 bits architectures (and also MacOS X) but it already looks great!
Many thanks for this work (and underlying stuff by other people).
frc
g++ -I/afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include -I/usr/local/boost/include/boost-1_34_1 -I/usr/local/boost/include/boost-1_34_1 -fPIC prg_test.cxx -o prg_test -L/usr/local/boost/lib -lboost_serialization-gcc34 -L/usr/local/boost/lib -lboost_filesystem-gcc34 In file included from prg_test.cxx:15: /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pba/portable_binary_oarchive.hpp:65: error: wrong number of template arguments (1, should be 3) /usr/local/boost/include/boost-1_34_1/boost/archive/binary_oarchive_impl.hpp:28: error: provided for `template
class boost::archive::binary_oarchive_impl' /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pba/portable_binary_oarchive.hpp:73: error: wrong number of template arguments (1, should be 3) /usr/local/boost/include/boost-1_34_1/boost/archive/binary_oarchive_impl.hpp:28: error: provided for `template class boost::archive::binary_oarchive_impl' /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pba/portable_binary_oarchive.hpp:73: error: ISO C++ forbids declaration of `archive_base_t' with no type ...
Hi again, I'm checking PBA stuff under my other main production system (Scientific Linux [~RedHat] under 64 bits processors and gcc 3.4). I use boost 1_34 and get this message while building a sample program that works perfectly under boost 1.33 (with Ubuntu/32bits box/gcc4). Did I miss some important notice? Here is the nasty message: then follows very similar error messages... <<< I looks as if there is difference between boost release. thanks for a hint. frc -- Francois Mauger Laboratoire de Physique Corpusculaire de Caen et Universite de Caen ENSICAEN - 6, Boulevard du Marechal Juin, 14050 CAEN Cedex, FRANCE e-mail: mauger@lpccaen.in2p3.fr tel.: (0/+33) 2 31 45 25 12 fax: (0/+33) 2 31 45 25 49
I believe that the version of boost release is broken. That is why I put the new version in the trunk. Robert Ramey François Mauger wrote:
Hi again,
I'm checking PBA stuff under my other main production system (Scientific Linux [~RedHat] under 64 bits processors and gcc 3.4). I use boost 1_34 and get this message while building a sample program that works perfectly under boost 1.33 (with Ubuntu/32bits box/gcc4).
Did I miss some important notice?
g++ -I/afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include -I/usr/local/boost/include/boost-1_34_1 -I/usr/local/boost/include/boost-1_34_1 -fPIC prg_test.cxx -o
Here is the nasty message: prg_test -L/usr/local/boost/lib -lboost_serialization-gcc34 -L/usr/local/boost/lib -lboost_filesystem-gcc34 In file included from prg_test.cxx:15: /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pba/portable_binary_oarchive.hpp:65: error: wrong number of template arguments (1, should be 3) /usr/local/boost/include/boost-1_34_1/boost/archive/binary_oarchive_impl.hpp:28: error: provided for `template
class boost::archive::binary_oarchive_impl' /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pba/portable_binary_oarchive.hpp:73: error: wrong number of template arguments (1, should be 3) /usr/local/boost/include/boost-1_34_1/boost/archive/binary_oarchive_impl.hpp:28: error: provided for `template class boost::archive::binary_oarchive_impl' /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pba/portable_binary_oarchive.hpp:73: error: ISO C++ forbids declaration of `archive_base_t' with no type ... then follows very similar error messages... <<< I looks as if there is difference between boost release.
thanks for a hint.
frc
Hi Robert, Christian and all
On Tue, April 8, 2008 7:11 pm, Robert Ramey wrote: I believe that the version of boost release is broken. That is why I put the new version in the trunk.
do you mean that boost 1.34 is broken independently of my attempt to use "portable binary archive"? ...or just for this very special case? I have to know for I have 1 Tb of data to serialize using boost on my computing center with 1.34 officially installed by the staff and I also need to transfer these data from 1.33 and/or 1.34 target workstations. thanks for advice frc --
Robert Ramey
François Mauger wrote:
Hi again,
I'm checking PBA stuff under my other main production system (Scientific Linux [~RedHat] under 64 bits processors and gcc 3.4). I use boost 1_34 and get this message while building a sample program that works perfectly under boost 1.33 (with Ubuntu/32bits box/gcc4).
Did I miss some important notice?
Here is the nasty message:
g++ -I/afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include -I/usr/local/boost/include/boost-1_34_1 -I/usr/local/boost/include/boost-1_34_1 -fPIC prg_test.cxx -o prg_test -L/usr/local/boost/lib -lboost_serialization-gcc34 -L/usr/local/boost/lib -lboost_filesystem-gcc34 In file included from prg_test.cxx:15: /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pba /portable_binary_oarchive.hpp:65: error: wrong number of template arguments (1, should be 3) /usr/local/boost/include/boost-1_34_1/boost/archive/binary_oarchive_impl .hpp:28: error: provided for `template
class boost::archive::binary_oarchive_impl' /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pb a/portable_binary_oarchive.hpp:73: error: wrong number of template arguments (1, should be 3) /usr/local/boost/include/boost-1_34_1/boost/archive/binary_oarchive_impl .hpp:28: error: provided for `template class boost::archive::binary_oarchive_impl' /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pb a/portable_binary_oarchive.hpp:73: error: ISO C++ forbids declaration of `archive_base_t' with no type ... then follows very similar error messages... <<< I looks as if there is difference between boost release.
thanks for a hint.
frc
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Francois Mauger Laboratoire de Physique Corpusculaire de Caen et Universite de Caen ENSICAEN - 6, Boulevard du Marechal Juin, 14050 CAEN Cedex, FRANCE e-mail: mauger@lpccaen.in2p3.fr tel.: (0/+33) 2 31 45 25 12 fax: (0/+33) 2 31 45 25 49
The version of portable binary archive in the boost 1.35 release is broken. a) it will create an archive b) that archive will not be portable. c) it will work however on the same machine. d) but it cant be loaded on a machine of differing endianness Robert Ramey François Mauger wrote:
Hi Robert, Christian and all
On Tue, April 8, 2008 7:11 pm, Robert Ramey wrote: I believe that the version of boost release is broken. That is why I put the new version in the trunk.
do you mean that boost 1.34 is broken independently of my attempt to use "portable binary archive"? ...or just for this very special case? I have to know for I have 1 Tb of data to serialize using boost on my computing center with 1.34 officially installed by the staff and I also need to transfer these data from 1.33 and/or 1.34 target workstations.
thanks for advice
frc --
Robert Ramey
François Mauger wrote:
Hi again,
I'm checking PBA stuff under my other main production system (Scientific Linux [~RedHat] under 64 bits processors and gcc 3.4). I use boost 1_34 and get this message while building a sample program that works perfectly under boost 1.33 (with Ubuntu/32bits box/gcc4).
Did I miss some important notice?
Here is the nasty message:
g++ -I/afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include -I/usr/local/boost/include/boost-1_34_1 -I/usr/local/boost/include/boost-1_34_1 -fPIC prg_test.cxx -o prg_test -L/usr/local/boost/lib -lboost_serialization-gcc34 -L/usr/local/boost/lib -lboost_filesystem-gcc34 In file included from prg_test.cxx:15: /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pba /portable_binary_oarchive.hpp:65: error: wrong number of template arguments (1, should be 3) /usr/local/boost/include/boost-1_34_1/boost/archive/binary_oarchive_impl .hpp:28: error: provided for `template
class boost::archive::binary_oarchive_impl' /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pb a/portable_binary_oarchive.hpp:73: error: wrong number of template arguments (1, should be 3) /usr/local/boost/include/boost-1_34_1/boost/archive/binary_oarchive_impl .hpp:28: error: provided for `template class boost::archive::binary_oarchive_impl' /afs/in2p3.fr/home/m/mauger/sw/pba_help/pba_help_trunk/include/boost/pb a/portable_binary_oarchive.hpp:73: error: ISO C++ forbids declaration of `archive_base_t' with no type ... then follows very similar error messages... <<< I looks as if there is difference between boost release.
thanks for a hint.
frc
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
boost-users-bounces@lists.boost.org wrote:
Hi again,
I'm checking PBA stuff under my other main production system (Scientific Linux [~RedHat] under 64 bits processors and gcc 3.4). I use boost 1_34 and get this message while building a sample program that works perfectly under boost 1.33 (with Ubuntu/32bits box/gcc4).
This might also be a compiler issue. You should upgrade to gcc 4.x, as the portable binary archives use lots of template code, enable_if and the like. The older gcc versions may no like that too much... Good luck, Christian
Hi
I'm checking PBA stuff under my other main production system (Scientific Linux [~RedHat] under 64 bits processors and gcc 3.4). I use boost 1_34 and get this message while building a sample program that works perfectly under boost 1.33 (with Ubuntu/32bits box/gcc4).
This might also be a compiler issue. You should upgrade to gcc 4.x, as the portable binary archives use lots of template code, enable_if and the like. The older gcc versions may no like that too much...
ok, does it mean that the staff should recompile boost using gcc4 too? thanks frc -- Francois Mauger Laboratoire de Physique Corpusculaire de Caen et Universite de Caen ENSICAEN - 6, Boulevard du Marechal Juin, 14050 CAEN Cedex, FRANCE e-mail: mauger@lpccaen.in2p3.fr tel.: (0/+33) 2 31 45 25 12 fax: (0/+33) 2 31 45 25 49
Francois Mauger wrote:
This might also be a compiler issue. You should upgrade to gcc 4.x, as the portable binary archives use lots of template code, enable_if and the like. The older gcc versions may no like that too much...
ok, does it mean that the staff should recompile boost using gcc4 too?
Yes, definitely! Christian
participants (4)
-
Daniel Oberhoff -
François Mauger -
Pfligersdorffer, Christian -
Robert Ramey