[serialization] [ublas] - ublas::matrix, ublas::vector serialization patch

hi serialization of ublas matrix and vector is missing in boost 1.33, therefore I've written missing functions: /usr/include/boost/serialization/ublas_matrix.hpp /usr/include/boost/serialization/ublas_vector.hpp files are in the attachment. I'm afraid that the idea of converting matrix to std::vector<std::vector<U> > is not optimal, perhaps someone can make it better? Please include those files in next boost version. File are very short, so review should not take too long. I bet that some people out there need them. best regards -- Janek Kozicki |

Janek Kozicki wrote:
hi
serialization of ublas matrix and vector is missing in boost 1.33, therefore I've written missing functions:
/usr/include/boost/serialization/ublas_matrix.hpp /usr/include/boost/serialization/ublas_vector.hpp
files are in the attachment.
I'm afraid that the idea of converting matrix to std::vector<std::vector<U> > is not optimal, perhaps someone can make it better?
Please include those files in next boost version. File are very short, so review should not take too long. I bet that some people out there need them.
best regards
Here is the code I use to serialize ublas vector: #include <sstream> #include <boost/archive/binary_oarchive.hpp> #include <boost/archive/binary_iarchive.hpp> #include <boost/serialization/split_free.hpp> #include <boost/serialization/nvp.hpp> #include <boost/serialization/level.hpp> #include <boost/serialization/tracking.hpp> namespace boost { namespace serialization { template<class T> struct implementation_level<std::complex<T> > { typedef mpl::integral_c_tag tag; // typedef mpl::int_<primitive_type> type; typedef mpl::int_<object_serializable> type; BOOST_STATIC_CONSTANT( int, value = implementation_level::type::value ); }; template<class T> struct tracking_level<std::complex<T> > { typedef mpl::integral_c_tag tag; typedef mpl::int_<track_never> type; BOOST_STATIC_CONSTANT( int, value = tracking_level::type::value ); }; } } // template<typename Archive, typename T> // inline void save (Archive &ar, const std::complex<T>& z, const unsigned int) { // // ar << boost::serialization::make_nvp ("real", const_cast<const T&>(real (z))); // // ar << boost::serialization::make_nvp ("imag", const_cast<const T&>(imag (z))); // ar << boost::serialization::make_nvp ("real", real (z)); // ar << boost::serialization::make_nvp ("imag", imag (z)); // } // template<typename Archive, typename T> // inline void load (Archive &ar, std::complex<T>& z, const unsigned int) { // // T i, r; // // ar >> boost::serialization::make_nvp ("real", r); // // ar >> boost::serialization::make_nvp ("imag", i); // // z = std::complex<T> (r, i); // ar >> boost::serialization::make_nvp ("real", real(z)); // ar >> boost::serialization::make_nvp ("imag", imag(z)); // } // namespace boost { namespace serialization { // template<class Archive, class T> // inline void serialize (Archive &ar, std::complex<T>& z, const unsigned int file_version) { // boost::serialization::split_free (ar, z, file_version); // } // } // } namespace boost { namespace serialization { template<class Archive, class T> inline void serialize (Archive &ar, std::complex<T>& z, const unsigned int file_version) { ar & boost::serialization::make_nvp ("real", real(z)); ar & boost::serialization::make_nvp ("imag", imag(z)); // ar & real(z); // ar & imag(z); } } } namespace boost { namespace serialization { template<class T> struct implementation_level<ublas::vector<T> > { typedef mpl::integral_c_tag tag; // typedef mpl::int_<primitive_type> type; typedef mpl::int_<object_serializable> type; BOOST_STATIC_CONSTANT( int, value = implementation_level::type::value ); }; } } template<class Archive, class U> inline void save (Archive &ar, const ublas::vector<U> &v, const unsigned int) { unsigned int count = v.size(); ar << BOOST_SERIALIZATION_NVP (count); typename ublas::vector<U>::const_iterator it = v.begin(); while (count-- > 0) { ar << boost::serialization::make_nvp ("item", *it++); } } template<class Archive, class U> inline void load (Archive &ar, ublas::vector<U> &v, const unsigned int) { unsigned int count; ar >> BOOST_SERIALIZATION_NVP (count); v.resize (count); typename ublas::vector<U>::iterator it = v.begin(); while (count-- > 0) { ar >> boost::serialization::make_nvp ("item", *it++); } } namespace boost { namespace serialization { template<class Archive, class U> inline void serialize (Archive &ar, ublas::vector<U>& v, const unsigned int file_version) { boost::serialization::split_free (ar, v, file_version); } } } namespace python = boost::python; template <typename MatrixType> struct vector_pickle_suite : pickle_suite { static tuple getinitargs(const MatrixType &m) { return make_tuple(m.size()); } static python::object getstate (const MatrixType& v) { std::ostringstream os; boost::archive::binary_oarchive oa(os); oa << v; return python::str (os.str()); } static void setstate(MatrixType& v, python::object entries) { python::str s = python::extract<python::str> (entries)(); std::string st = python::extract<std::string> (s)(); std::istringstream is (st); boost::archive::binary_iarchive ia (is); ia >> v; } // static // object // getstate(const MatrixType &m) // { // typename MatrixType::const_iterator // first = m.begin(), // last = m.end(); // list result; // while (first != last) // result.append(*first++); // return result; // } // static void // setstate(MatrixType &m, object entries) // { // typename MatrixType::iterator // first = m.begin(); // unsigned len = extract<unsigned>(entries.attr("__len__")()); // for (unsigned i = 0; i < len; i++) // *first++ = (typename MatrixType::value_type) // extract<typename MatrixType::value_type>(entries[i]); // } };

This looks pretty good to me. I would prefer to see the following: a) the stream/archive lines removed from the top b) and #inlcude <.. ublas.hpp> c) a name change to boost/sublas/serialization.hpp d) tests, etc. This way boost/ublas/serialization.hpp could be included in any program which uses ublas. Then ublas data could be serialized to any archive type. Robert Ramey Neal Becker wrote:
Here is the code I use to serialize ublas vector:
#include <sstream> #include <boost/archive/binary_oarchive.hpp> #include <boost/archive/binary_iarchive.hpp>
#include <boost/serialization/split_free.hpp> #include <boost/serialization/nvp.hpp> #include <boost/serialization/level.hpp> #include <boost/serialization/tracking.hpp>
...

Can I get a little help please? I don't understand what's wrong with this, can anyone help? I have ublas_ser.hpp and a start at a test test_ublas_ser.cpp, but I get this error: [...] /usr/include/boost/archive/detail/oserializer.hpp:107: error: incomplete type ‘boost::serialization::extended_type_info_null<boost::numeric::ublas::vector<int, boost::numeric::ublas::unbounded_array<int, std::allocator<int> > > >’ used in nested name specifier /usr/include/boost/archive/detail/oserializer.hpp: In member function ‘bool boost::archive::detail::oserializer<Archive, T>::is_polymorphic() const [with Archive = boost::archive::binary_oarchive, T = boost::numeric::ublas::vector<int, boost::numeric::ublas::unbounded_array<int, std::allocator<int> > >]’: #include <boost/serialization/split_free.hpp> #include <boost/serialization/nvp.hpp> #include <boost/serialization/level.hpp> #include <boost/serialization/tracking.hpp> #include <boost/mpl/int.hpp> #include <complex> #include <boost/numeric/ublas/vector.hpp> namespace boost { namespace serialization { template<class T> struct implementation_level<std::complex<T> > { typedef mpl::integral_c_tag tag; // typedef mpl::int_<primitive_type> type; typedef mpl::int_<object_serializable> type; BOOST_STATIC_CONSTANT( int, value = implementation_level::type::value ); }; template<class T> struct tracking_level<std::complex<T> > { typedef mpl::integral_c_tag tag; typedef mpl::int_<track_never> type; BOOST_STATIC_CONSTANT( int, value = tracking_level::type::value ); }; } } namespace boost { namespace serialization { template<class Archive, class T> inline void serialize (Archive &ar, std::complex<T>& z, const unsigned int file_version) { ar & boost::serialization::make_nvp ("real", real(z)); ar & boost::serialization::make_nvp ("imag", imag(z)); // ar & real(z); // ar & imag(z); } } } namespace boost { namespace serialization { template<class T> struct implementation_level<boost::numeric::ublas::vector<T> > { typedef mpl::integral_c_tag tag; // typedef mpl::int_<primitive_type> type; typedef mpl::int_<object_serializable> type; BOOST_STATIC_CONSTANT( int, value = implementation_level::type::value ); }; } } template<class Archive, class U> inline void save (Archive &ar, const boost::numeric::ublas::vector<U> &v, const unsigned int) { unsigned int count = v.size(); ar << BOOST_SERIALIZATION_NVP (count); typename boost::numeric::ublas::vector<U>::const_iterator it = v.begin(); while (count-- > 0) { ar << boost::serialization::make_nvp ("item", *it++); } } template<class Archive, class U> inline void load (Archive &ar, boost::numeric::ublas::vector<U> &v, const unsigned int) { unsigned int count; ar >> BOOST_SERIALIZATION_NVP (count); v.resize (count); typename boost::numeric::ublas::vector<U>::iterator it = v.begin(); while (count-- > 0) { ar >> boost::serialization::make_nvp ("item", *it++); } } namespace boost { namespace serialization { template<class Archive, class U> inline void serialize (Archive &ar, boost::numeric::ublas::vector<U>& v, const unsigned int file_version) { boost::serialization::split_free (ar, v, file_version); } } } #include "ublas_ser.hpp" #include <boost/test/minimal.hpp> #include <boost/archive/binary_oarchive.hpp> #include <boost/archive/binary_iarchive.hpp> #include <sstream> #include <string> namespace ublas = boost::numeric::ublas; template<typename V> std::string Saveit (V const& v) { std::ostringstream os; boost::archive::binary_oarchive oa (os); oa << v; return os.str(); } int test_main( int, char *[] ) { using namespace boost::serialization; ublas::vector<int> a (10); for (int i = 0; i < a.size(); ++i) a[i] = i; std::ostringstream os; boost::archive::binary_oarchive oa (os); oa << const_cast<const ublas::vector<int>&> (a); // std::string s = Saveit (a); }

I see this is the same error that some others encountered. It makes serialization in 1.33 broken with gcc-4.1 (which is about to release). I was able to compile with boost-cvs and patches that were just posted here. Here is a basic ublas vector serializer and 1 test.

This includes more complete testing and some nice utilities to compare numeric sequence for closeness. Maybe useful on there own.

I would recommend the following: a) add tests. Use the current tests in the serialization library as a guide b) add your new tests to the Jamfile your copy of the serialization/test directory. c) run the tests on the compilers you have at your disposal. d) upload the whole package along with any descriptive documentation to the boost files section. Robert Ramey Janek Kozicki wrote:
hi
serialization of ublas matrix and vector is missing in boost 1.33, therefore I've written missing functions:
/usr/include/boost/serialization/ublas_matrix.hpp /usr/include/boost/serialization/ublas_vector.hpp
files are in the attachment.
I'm afraid that the idea of converting matrix to std::vector<std::vector<U> > is not optimal, perhaps someone can make it better?
Please include those files in next boost version. File are very short, so review should not take too long. I bet that some people out there need them.
best regards -- Janek Kozicki |
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
participants (3)
-
Janek Kozicki
-
Neal Becker
-
Robert Ramey