[boost.python] use of multi_array in python

I'm having some problems with the use of multi_array's with python. What I'm trying to do is to export some functions that create multi_array's and pass these multi_array's to other functions that will extract other array's from them and display them. Lets say I have this as an example. //ArrayRef2D.h typedef for a 2D multi_array of doubles #ifndef ARRAYREF2D_H_ #define ARRAYREF2D_H_ #include <boost/multi_array.hpp> typedef boost::const_multi_array_ref < double, 2 > const_array_type2D_ref_d_t; typedef boost::multi_array < double, 2 > array_type2D_d_t; #endif /*ARRAYREF2D_H_*/ //Fstd.h 2D multi_array generator #ifndef FSTD_H_ #define FSTD_H_ #include "ArrayRef2D.h" #include <string> using namespace std; class Fstd { public: Fstd(string filename) : filename_(filename){} array_type2D_d_t get2Dfield() { array_type2D_d_t matrix(boost::extents[2][2]); matrix[0][0] = 0; matrix[0][1] = -1; matrix[1][0] = 20; matrix[1][1] = -30; return matrix; } private: string filename_; }; #endif /*FSTD_H_*/ //MatrixAlgorithm2D.h provides methods for displaying and manipulating matrices #ifndef MATRIXALGORITHM2D_H_ #define MATRIXALGORITHM2D_H_ #include "ArrayRef2D.h" #include <cassert> #include <cmath> #include <iostream> using namespace std; class MatrixAlgorithm2D { public: static array_type2D_d_t windChill(const_array_type2D_ref_d_t TT, const_array_type2D_ref_d_t UV) { assert((TT.shape()[0] == UV.shape()[0]) && (TT.shape()[1] == UV.shape()[1])); array_type2D_d_t RE(boost::extents[TT.shape()[0]][TT.shape()[1]]); for (unsigned int x = 0; x < TT.shape()[0]; x++) { for (unsigned int y = 0; y < TT.shape()[1]; y++) { if ((TT[x][y] <= 0.0) && (UV[x][y] >= 5.0)) { RE[x][y] = ( 13.1200 + 0.6215 * (TT[x][y]) + (0.3965 * (TT[x][y]) - 11.3700) * (pow((UV[x][y]),0.16))); } else { RE[x][y] = (TT[x][y]); } } } return RE; } static void display_d(const_array_type2D_ref_d_t a) { for (unsigned int x = 0; x < a.shape()[0]; x++) { for (unsigned int y = 0; y < a.shape()[1]; y++) { cout << a[x][y]; if(y < a.shape()[1] - 1) { cout << ", "; } } cout<<endl; } } }; #endif /*MATRIXALGORITHM2D_H_*/ //matrix.cpp boost.python interface file generated with pyste // Boost Includes ============================================================== #include "boost/python.hpp" #include "boost/cstdint.hpp" // Includes ==================================================================== #include "Fstd.h" #include "MatrixAlgorithm2D.h" // Using ======================================================================= using namespace boost::python; // Module ====================================================================== BOOST_PYTHON_MODULE(matrix) { class_< Fstd >("Fstd", init< const Fstd& >()) .def(init< std::string >()) .def("get2Dfield", &Fstd::get2Dfield) ; class_< MatrixAlgorithm2D >("MatrixAlgorithm2D", init< >()) .def(init< const MatrixAlgorithm2D& >()) .def("windChill", &MatrixAlgorithm2D::windChill) .def("display_d", &MatrixAlgorithm2D::display_d) .staticmethod("windChill") .staticmethod("display_d") ; } I generated my so file with bjam and then Python 2.3.5 (#2, Sep 4 2005, 22:01:42) [GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import matrix dir(matrix) ['Fstd', 'MatrixAlgorithm2D', '__doc__', '__file__', '__name__'] m = matrix.Fstd("HELLO") l = m.get2Dfield() Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: No to_python (by-value) converter found for C++ type: boost::multi_array<double, 2, std::allocator<double> >
So you see I can't use my code, what am I doing wrong? I would have liked to declare 2 arrays with get2Dfield() and pass these array to my windChill function to get another 2d matrix which I could then display with my display_d function. -- Sébastien Fortier

Sebastien Fortier wrote:
I'm having some problems with the use of multi_array's with python. What I'm trying to do is to export some functions that create multi_array's and pass these multi_array's to other functions that will extract other array's from them and display them. [...] So you see I can't use my code, what am I doing wrong?
You didn't export multi_array<double, 2> to Python. If you just want to pass the multi_array's around without manipulating them, it's enough to export it without any functions: class_<multi_array<double, 2> >("multi_array_2d"); If you want a more complete binding, there's binding code for boost::multi_array queued for inclusion on the CVS. It's currently on a branch: Header: http://tinyurl.com/hm7pa Test: http://tinyurl.com/z7fzn http://tinyurl.com/hthg2 HTH, -- Daniel Wallin

Thanks a bunch Daniel! I think I've almost got it, I'll keep at it. its now asking me for const_multi_array_ref, I guess I will add the declaration to the interface as you mentionned... Daniel Wallin wrote:
Sebastien Fortier wrote:
I'm having some problems with the use of multi_array's with python. What I'm trying to do is to export some functions that create multi_array's and pass these multi_array's to other functions that will extract other array's from them and display them.
[...]
So you see I can't use my code, what am I doing wrong?
You didn't export multi_array<double, 2> to Python. If you just want to pass the multi_array's around without manipulating them, it's enough to export it without any functions:
class_<multi_array<double, 2> >("multi_array_2d");
If you want a more complete binding, there's binding code for boost::multi_array queued for inclusion on the CVS. It's currently on a branch:
Header: http://tinyurl.com/hm7pa
Test: http://tinyurl.com/z7fzn http://tinyurl.com/hthg2
HTH,
-- Sébastien Fortier

Thank you very much Daniel! I had another question, is it possible to use multi_array and const_multi_array_ref together in python. What I mean is, if I create a multi_array from python through my c++ functions and I want to pass this array to another c++ function that uses a const_multi_array_ref how do I manage this in the interface file? is this possible? I added class_<const_multi_array_ref<double, 2>
("const_multi_array_2d_ref"); to my interface file but the types seem to be incompatible. I would prefer to use the const_multi_array_ref with the multi_arrays to avoid copies...
I had to rewrite my functions to use only multi_array's to make the code I had attached work. Daniel Wallin wrote:
Sebastien Fortier wrote:
I'm having some problems with the use of multi_array's with python. What I'm trying to do is to export some functions that create multi_array's and pass these multi_array's to other functions that will extract other array's from them and display them.
[...]
So you see I can't use my code, what am I doing wrong?
You didn't export multi_array<double, 2> to Python. If you just want to pass the multi_array's around without manipulating them, it's enough to export it without any functions:
class_<multi_array<double, 2> >("multi_array_2d");
If you want a more complete binding, there's binding code for boost::multi_array queued for inclusion on the CVS. It's currently on a branch:
Header: http://tinyurl.com/hm7pa
Test: http://tinyurl.com/z7fzn http://tinyurl.com/hthg2
HTH,
-- Sébastien Fortier

Sebastien Fortier wrote:
Thank you very much Daniel!
I had another question, is it possible to use multi_array and const_multi_array_ref together in python. What I mean is, if I create a multi_array from python through my c++ functions and I want to pass this array to another c++ function that uses a const_multi_array_ref how do I manage this in the interface file? is this possible?
I added class_<const_multi_array_ref<double, 2>
("const_multi_array_2d_ref"); to my interface file but the types seem to be incompatible.
You need to let Boost.Python know that the types are convertible: implicitly_convertible< multi_array<double, 2> , multi_array_ref<double, 2>
();
implicitly_convertible< multi_array<double, 2> , const_multi_array_ref<double, 2>
();
Should do it. See http://www.boost.org/libs/python/doc/v2/implicit.html.
I would prefer to use the const_multi_array_ref with the multi_arrays to avoid copies...
If you already have multi_array's, why don't you just pass them by reference? -- Daniel Wallin

Thank you again for the quick reply! I will try what you have suggested. One last question, If I have a header file to export with its implementation file, how do I go about linking the object file with the library that is created with bjam? When I start bjam to compile the shared library it does not link nor compile the implementation file with the shares library... Ex.: // Boost Includes ============================================================== #include "boost/multi_array.hpp" #include "boost/python.hpp" #include "boost/cstdint.hpp" // Includes ==================================================================== #include "Fstd.h" #include "MatrixAlgorithm2D.h" // Using ======================================================================= using namespace boost::python; // Module ====================================================================== BOOST_PYTHON_MODULE(matrix) { class_< Fstd >("Fstd", init< const Fstd& >()) .def(init< std::string >()) .def("getTT", &Fstd::getTT) .def("getUV", &Fstd::getUV) ; class_< MatrixAlgorithm2D >("MatrixAlgorithm2D", init< >()) .def(init< const MatrixAlgorithm2D& >()) .def("windChill", &MatrixAlgorithm2D::windChill) .def("display_d", &MatrixAlgorithm2D::display_d) .staticmethod("windChill") .staticmethod("display_d") ; class_<boost::multi_array<double, 2> >("multi_array_2d"); class_<boost::const_multi_array_ref<double, 2>
("const_multi_array_ref_2d");
implicitly_convertible<multi_array<double, 2>, const_multi_array_ref<double, 2> >(); } I have MatrixAlgorithm2D.h, Fstd.h and Fstd.cpp, when I execure bjam the shared library is created but not the object file for Fstd.cpp, so when I try to use this from Python, it fails... Daniel Wallin wrote:
Sebastien Fortier wrote:
Thank you very much Daniel!
I had another question, is it possible to use multi_array and const_multi_array_ref together in python. What I mean is, if I create a multi_array from python through my c++ functions and I want to pass this array to another c++ function that uses a const_multi_array_ref how do I manage this in the interface file? is this possible?
I added class_<const_multi_array_ref<double, 2>
("const_multi_array_2d_ref"); to my interface file but the types seem to be incompatible.
You need to let Boost.Python know that the types are convertible:
implicitly_convertible< multi_array<double, 2> , multi_array_ref<double, 2>
();
implicitly_convertible< multi_array<double, 2> , const_multi_array_ref<double, 2>
();
Should do it. See http://www.boost.org/libs/python/doc/v2/implicit.html.
I would prefer to use the const_multi_array_ref with the multi_arrays to avoid copies...
If you already have multi_array's, why don't you just pass them by reference?
-- Sébastien Fortier

I've tried what you have suggested but it does not work, here is the error .../boost_1_33_1/boost/python/object/value_holder.hpp:135: error: no matching function for call to `boost::const_multi_array_ref<double, 2, const double*>::const_multi_array_ref()' .../boost_1_33_1/boost/multi_array/multi_array_ref.hpp:45: error: candidates are: boost::const_multi_array_ref<double, 2, const double*>::const_multi_array_ref(const boost::const_multi_array_ref<double, 2, const double*>&) .../boost_1_33_1/boost/multi_array/multi_array_ref.hpp:325: error: boost::const_multi_array_ref<T, NumDims, TPtr>::const_multi_array_ref(TPtr, const boost::general_storage_order<NumDims>&, typename boost::detail::multi_array::multi_array_impl_base<T, NumDims>::index*, typename boost::detail::multi_array::multi_array_impl_base<T, NumDims>::size_type*) [with T = double, unsigned int NumDims = 2, TPtr = const double*] .../boost_1_33_1/boost/multi_array/multi_array_ref.hpp:120: error: boost::const_multi_array_ref<T, NumDims, TPtr>::const_multi_array_ref(TPtr, const boost::detail::multi_array::extent_gen<NumDims>&, const boost::general_storage_order<NumDims>&) [with T = double, unsigned int NumDims = 2, TPtr = const double*] .../boost_1_33_1/boost/multi_array/multi_array_ref.hpp:111: error: boost::const_multi_array_ref<T, NumDims, TPtr>::const_multi_array_ref(TPtr, const boost::detail::multi_array::extent_gen<NumDims>&) [with T = double, unsigned int NumDims = 2, TPtr = const double*] my interface file is // Boost Includes ============================================================== #include "boost/multi_array.hpp" #include "boost/python/class.hpp" #include "boost/python/implicit.hpp" #include "boost/python/module.hpp" #include "boost/python.hpp" #include "boost/cstdint.hpp" // Includes ==================================================================== #include "Fstd.h" #include "MatrixAlgorithm2D.h" // Using ======================================================================= using namespace boost::python; // Module ====================================================================== BOOST_PYTHON_MODULE(matrix) { class_< Fstd >("Fstd", init< const Fstd& >()) .def(init< std::string >()) .def("getTT", &Fstd::getTT) .def("getUV", &Fstd::getUV) ; class_< MatrixAlgorithm2D >("MatrixAlgorithm2D", init< >()) .def(init< const MatrixAlgorithm2D& >()) .def("windChill", &MatrixAlgorithm2D::windChill) .def("display_d", &MatrixAlgorithm2D::display_d) .staticmethod("windChill") .staticmethod("display_d") ; class_<boost::multi_array<double, 2> >("multi_array_2d"); class_<boost::const_multi_array_ref<double, 2>
("const_multi_array_ref_2d"); implicitly_convertible<boost::multi_array<double, 2>, boost::multi_array_ref<double, 2> >(); implicitly_convertible<boost::multi_array<double, 2>, boost::const_multi_array_ref<double, 2> >(); }
Daniel Wallin wrote:
Sebastien Fortier wrote:
Thank you very much Daniel!
I had another question, is it possible to use multi_array and const_multi_array_ref together in python. What I mean is, if I create a multi_array from python through my c++ functions and I want to pass this array to another c++ function that uses a const_multi_array_ref how do I manage this in the interface file? is this possible?
I added class_<const_multi_array_ref<double, 2>
("const_multi_array_2d_ref"); to my interface file but the types seem to be incompatible.
You need to let Boost.Python know that the types are convertible:
implicitly_convertible< multi_array<double, 2> , multi_array_ref<double, 2>
();
implicitly_convertible< multi_array<double, 2> , const_multi_array_ref<double, 2>
();
Should do it. See http://www.boost.org/libs/python/doc/v2/implicit.html.
I would prefer to use the const_multi_array_ref with the multi_arrays to avoid copies...
If you already have multi_array's, why don't you just pass them by reference?
-- Sébastien Fortier

Sebastien Fortier wrote:
I've tried what you have suggested but it does not work, here is the error [...] class_<boost::multi_array<double, 2> >("multi_array_2d"); class_<boost::const_multi_array_ref<double, 2>
("const_multi_array_ref_2d"); implicitly_convertible<boost::multi_array<double, 2>, boost::multi_array_ref<double, 2> >(); implicitly_convertible<boost::multi_array<double, 2>, boost::const_multi_array_ref<double, 2> >(); }
multi_array_ref<> isn't default constructible. You need to let Boost.Python know that: class_<boost::const_multi_array_ref<double, 2> > >( "const_multi_array_ref_2d", no_init ); class_<boost::multi_array_ref<double, 2> > >( "multi_array_ref_2d", no_init ); -- Daniel Wallin

Thanks again! These answers will probably get me out of alot of jams... Daniel Wallin wrote: [...]
multi_array_ref<> isn't default constructible. You need to let Boost.Python know that:
class_<boost::const_multi_array_ref<double, 2> > >( "const_multi_array_ref_2d", no_init );
class_<boost::multi_array_ref<double, 2> > >( "multi_array_ref_2d", no_init );
-- Sébastien Fortier Analyste-Programmeur Centre Météorologique Canadien 2121, voie de service nord, Trans-Canadienne Dorval, QC H9P 1J3 Tel: (514) 421-5049 Fax: (514) 421-4657 Courrier électronique: _ Sebastien.Fortier@ec.gc.ca_
participants (2)
-
Daniel Wallin
-
Sebastien Fortier