Re: [Boost-users] multi_array with large dimensions crashes

The code work now, thanks very much. But the programme crashes with a run-time error when the dimensions of the array are huge 500x500x500.

In your examples, you were using "float" variable, which requires 4 bytes to store. If you want a vector of 500*500*500=125000000 floats, that requires 500000000 bytes, i.e., 0.5 GB (or 0.466 GiB), of contiguous space in memory.

This is probably due to the limitation of the 32 bit addrress space.

For 32 bits, the total amount of addressable space is 4 GB. So no, that is not directly the problem.

Is there any other work around. I heard of creating arrays with strides and allocating them in different blocks of memory rather than in contiguos memory?

I fail to see how strides may help. However, requesting memory in a non-contiguous way may certainly help. The easiest way to do so would be to use a vector at the first level. typedef std::vector< boost::multi_array< data_type, 2 > > vector_of_2D_multi_array; When using this kind of object, remember that this is not a 3 dimensional multi_array but instead a vector of two dimensional ones. If you cannot reach 500x500x500 with this setting, try to see if your OS imposes limitations to the amount of memory that can be allocated to a single process. Finally, if you aim for even larger sizes, the amount of system memory will eventually become a problem. In such cases, you may want to consider distributed computing... Hope it helps, Pierre-André On Thu, Aug 25, 2011 at 5:04 AM, Jothy wrote:

Hi Pierre,

Sorry for the late reply.

The code work now, thanks very much. But the programme crashes with a run-time error when the dimensions of the array are huge 500x500x500. This is probably due to the limitation of the 32 bit addrress space. Is there any other work around. I heard of creating arrays with strides and allocating them in different blocks of memory rather than in contiguos memory?

Thanks again

Jothy

On Wed, Aug 24, 2011 at 6:25 PM, Pierre-Andre Noel < noel.pierre.andre@gmail.com> wrote:

...

Yes, 500, 300 and 400 were the example dimensions you used in your earlier message.

Did you got it to work?

Pierre-André

PS: This email was not sent to the whole list.

On Wed, Aug 24, 2011 at 11:55 AM, Jothy wrote:

...

You mean 500,300,400 are the required dimesnions and xDim,yDim,zDim are site_type parameters.

So the code should be like below I guess for an array of [x][y][z]

array_ref_type::size_type xDim(x), yDim(y), zDim(z);

array_storage_type Storage(xDim*yDim*zDim); array_ref_type Array(&Storage.front(),boost::extents[xDim][yDim][zDim]);

Thanks

Jothy

On Wed, Aug 24, 2011 at 3:51 PM, Pierre-Andre Noel < noel.pierre.andre@gmail.com> wrote:

...

...

...

I am not familiar with multi_array, but it seems that you are creating a >1GB object on the stack. This will almost always crash. Try creating it on the heap instead. Be prepared for std::bad_alloc.

...

But how to do this? I am new to c++.

I suggest you to either 1. read on the operators "new" and "delete" or 2. use a std::vector to allocate the memory then use a boost::multi_array_ref to refer to it. Here's an example:

#include <vector> #include

int main() { typedef float data_type; typedef std::vector array_storage_type; typedef boost::multi_array_ref array_ref_type;

array_ref_type::size_type xDim(500), yDim(300), zDim(400); array_storage_type Storage(xDim*yDim*zDim); array_ref_type Array(&Storage.front(),boost::extents[xDim][yDim][zDim]); // You may now use Array as usual. }

Hope it helps,

Pierre-André Noël

On Wed, Aug 24, 2011 at 5:20 AM, Jothy wrote:

...

Thanks for pointing this out.

But how to do this? I am new to c++.

Jothy

On Tue, Aug 23, 2011 at 10:42 PM, Juraj Ivančić < juraj.ivancic@gmail.com> wrote:

...

On 23.8.2011. 12:18, Jothy wrote:

> Hi all, > > I have just started to use boost libs. I am trying to create a large > float array (mandatory for my purpose). it works fine till certain > limit, but crashes above that. I think it about allocating memory in > the > stack. > > My code is like this > > typedefboost::multi_array<**float,3>array_type; > typedefarray_type::indexindex; > array_typeArray(boost::**extents[xDim][yDim][zDim]);//**500,300,400 > for instance > > > Can some one help me to resolve this? >

I am not familiar with multi_array, but it seems that you are creating a >1GB object on the stack. This will almost always crash. Try creating it on the heap instead. Be prepared for std::bad_alloc.

HTH

______________________________**_________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/**mailman/listinfo.cgi/boost-**usershttp://lists.boost.org/mailman/listinfo.cgi/boost-users

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users