Try out a multi-dimensional adapter class template?
I added a new class template for my library at https://github.com/CTMacUser/ArrayMD. It's called "multiarray," and it's an adapter class like stack and queue. But the new interface is only for accessing elements. Besides the element and container types, the template header for the class takes the number of dimensions. The class uses at, operator [] and operator () to access an element. It has methods to read/write the shape of the array and the priority of each index. There are no methods to change the number of elements stored; you have to either use a fixed-size container or sub-class the multi-array to add methods to change the size. (The container sub-object is protected, like the Standard adapters.) I split the main code into two base class templates, because the code for computing the offset from an index tuple and the code for referencing an element were nearly distinct. This really screws up my attempt at Doxygen comments. Any ideas? Or is my case too complex for what Doxygen's author thought could be handled. Daryle W.
Daryle Walker wrote:
I added a new class template for my library at https://github.com/CTMacUser/ArrayMD. It's called "multiarray," and it's an adapter class like stack and queue. But the new interface is only for accessing elements. Besides the element and container types, the template header for the class takes the number of dimensions. The class uses at, operator [] and operator () to access an element. It has methods to read/write the shape of the array and the priority of each index. There are no methods to change the number of elements stored; you have to either use a fixed-size container or sub-class the multi-array to add methods to change the size. (The container sub-object is protected, like the Standard adapters.) I split the main code into two base class templates, because the code for computing the offset from an index tuple and the code for referencing an element were nearly distinct. This really screws up my attempt at Doxygen comments. Any ideas? Or is my case too complex for what Doxygen's author thought could be handled.
I've looked at the document above. It would seem to me that this is similar or equivalent to boost.multi-array with extents set at compile time rather than at runtime. Is my understanding of this correct? If not how is it wrong. I've used multi-array on various occasions and have been struck by how well it's done. (and better than average documented). Have you considered just augmentting multi-array with some constructors or "re-sizer" which use extents defined as template parameters? It seem that if you could do this, one would be able to leverage all that is done already. Even if this is not possible, it would be nice to have an interface as close as possible to mult-array to leverage on one's understanding of both libraries. Robert Ramey
Daryle W.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
From: ramey@rrsd.com Date: Tue, 8 Oct 2013 08:50:54 -0800
Daryle Walker wrote:
I added a new class template for my library at https://github.com/CTMacUser/ArrayMD. It's called "multiarray," and it's an adapter class like stack and queue. But the new interface is only for accessing elements. Besides the element and container types, the template header for the class takes the number of dimensions. The class uses at, operator [] and operator () to access an element. It has methods to read/write the shape of the array and the priority of each index. There are no methods to change the number of elements stored; you have to either use a fixed-size container or sub-class the multi-array to add methods to change the size. (The container sub-object is protected, like the Standard adapters.) I split the main code into two base class templates, because the code for computing the offset from an index tuple and the code for referencing an element were nearly distinct. This really screws up my attempt at Doxygen comments. Any ideas? Or is my case too complex for what Doxygen's author thought could be handled.
I've looked at the document above. It would seem to me that this is similar or equivalent to boost.multi-array with extents set at compile time rather than at runtime. Is my understanding of this correct? If not how is it wrong.
Both templates get the number of dimensions as a compile-time parameter and the actual extent values at run-time. The Boost version can take the extents and priorities in the constructor, while my version has to change those attributes in separate call(s). But it seems that construction time is the only opportunity for the Boost version to use a different index priority; it can't be changed later, unlike mine. The Boost version is a full container, including allocator support. There are also mutable and immutable view classes. My version is an adapter, and punts storage considerations into what container type is used. Since my version holds the container as a protected member, view classes are nonsensical. The Boost version was designed during a time of broken C++98 support, while mine was designed for compliant C++11 support. The new version can take advantage of newer features. My version uses Standard classes to take indices and to get & set extents and priorities. The Boost version uses a lot of auxiliary classes. The Boost design is spread over many headers; my version is compact enough to be a done-in-one header (but it does less, too). Neither version provides methods to directly change the number of elements. The Boost version can change its element count through its resize method. My version can't at all, except through derived classes. But my version decouples the elements stored and what's needed. You only crash when referencing an element beyond the container's bounds. It was designed with std::array as a base container. It should be efficient when the desired size stays close to the array's fixed size. My version indexes only elements; sub-arrays are not supported. It wouldn't be efficient, since elements would not be contiguous in general. Create a new array object of the smaller size and use (c)apply to copy over what you want. Fancy indexing, outside of priorities, is not supported either. (Since there's no sub-arrays nor views, the chain-of-operator[] syntax is not supported.) My original intention was to make the dimension count a run-time variable. This would necessitate dynamic containers for the extents and priorities, probably a smart-pointer in array mode. But reasoning the copying & moving semantics was too hard. It's easy with a fixed extent size, and I guess it'll be the most common case. (I separated the indexing and containment into separate base classes. I could make a new indexing base for dynamic dimensions while keeping the same containment base.)
I've used multi-array on various occasions and have been struck> by how well it's done. (and better than average documented). Have you considered just augmentting multi-array with some constructors or "re-sizer" which use extents defined as template parameters? It seem that if you could do this, one would be able to leverage all that is done already. Even if this is not possible, it would be nice to have an interface as close as possible to mult-array to leverage on one's understanding of both libraries.
They have different interfaces. Neither of them can define the extent values at compile-time. Another class in that GitHub repository, array_md, does do that. However, it's a variadic extension of std::array, and so does not have any of the storage or indexing options that Boost.Multi-Array has. My multiarray adapter was started from so many complaints that array_md doesn't support anything besides row-major order, making it less desirable for all the math & science junkies that wanted column-major (or scrambled-priority) storage. Daryle W.
Daryle Walker wrote:
From: ramey@rrsd.com Date: Tue, 8 Oct 2013 08:50:54 -0800
Daryle Walker wrote:
I added a new class template for my library at https://github.com/CTMacUser/ArrayMD. It's called "multiarray," and it's an adapter class like stack and queue. But the new interface is only for accessing elements. Besides the element and container types, the template header for the class takes the number of dimensions. The class uses at, operator [] and operator () to access an element. It has methods to read/write the shape of the array and the priority of each index. There are no methods to change the number of elements stored; you have to either use a fixed-size container or sub-class the multi-array to add methods to change the size. (The container sub-object is protected, like the Standard adapters.) I split the main code into two base class templates, because the code for computing the offset from an index tuple and the code for referencing an element were nearly distinct. This really screws up my attempt at Doxygen comments. Any ideas? Or is my case too complex for what Doxygen's author thought could be handled.
I've looked at the document above. It would seem to me that this is similar or equivalent to boost.multi-array with extents set at compile time rather than at runtime. Is my understanding of this correct? If not how is it wrong.
Both templates get the number of dimensions as a compile-time parameter and the actual extent values at run-time. The Boost version can take the extents and priorities in the constructor, while my version has to change those attributes in separate call(s). But it seems that construction time is the only opportunity for the Boost version to use a different index priority; it can't be changed later, unlike mine.
Hmm- I'm not sure what "index priority" refers to.
The Boost version is a full container, including allocator support. There are also mutable and immutable view classes. My version is an adapter, and punts storage considerations into what container type is used. Since my version holds the container as a protected member, view classes are nonsensical.
views are an essential feature of multi-array to me.
The Boost version was designed during a time of broken C++98 support, while mine was designed for compliant C++11 support. The new version can take advantage of newer features. My version uses Standard classes to take indices and to get & set extents and priorities. The Boost version uses a lot of auxiliary classes. The Boost design is spread over many headers; my version is compact enough to be a done-in-one header (but it does less, too).
I'm sure that many libraries could be more easily writen with more up-to-date tools. But I'm not convinced that that is a reason for doing so. Personally I'm not really concerned about the implementation as long as it's efficient.
Neither version provides methods to directly change the number of elements. The Boost version can change its element count through its resize method.
Hmm - These statements seem to conflict - It seems to me that multi-array can have all the extents adjusted at run time. I can see that this facility could imply extra coding and loss of runtime efficiency. That was the motivation for my original question - it seemed to me there might be a motivation to have the same thing with more stuff fixed at compile time to make it faster at the cost of runtime configurability.
My version can't at all, except through derived classes. But my version decouples the elements stored and what's needed. You only crash when referencing an element beyond the container's bounds. It was designed with std::array as a base container. It should be efficient when the desired size stays close to the array's fixed size.
Hmm - doesn't one fix the array size to the "desired size" at compile time? Aren't they equal. I'm not getting this.
My version indexes only elements; sub-arrays are not supported. It wouldn't be efficient, since elements would not be contiguous in general.
To me this is an absolutly essential feature and one reason I love this library.
Create a new array object of the smaller size and use (c)apply to copy over what you want.
You have to walk all the elements to create the new contiguous array. It's not clear how you would do this in your library. The current one does it all automatically taking care of strides and all that. Your scheme just ignores the utility of this feature.
Fancy indexing, outside of priorities, is not supported either. (Since there's no sub-arrays nor views, the chain-of-operator[] syntax is not supported.)
Another very, very useful feature of the current multi-array.
I've used multi-array on various occasions and have been struck> by how well it's done. (and better than average documented). Have you considered just augmentting multi-array with some constructors or "re-sizer" which use extents defined as template parameters? It seem that if you could do this, one would be able to leverage all that is done already. Even if this is not possible, it would be nice to have an interface as close as possible to mult-array to leverage on one's understanding of both libraries.
They have different interfaces. Neither of them can define the extent values at compile-time.
That is the feature that I had thought your library might be adding.
Another class in that GitHub repository, array_md, does do that. However, it's a variadic extension of std::array, and so does not have any of the storage or indexing options that Boost.Multi-Array has. My multiarray adapter was started from so many complaints that array_md doesn't support anything besides row-major order, making it less desirable for all the math & science junkies that wanted column-major (or scrambled-priority) storage.
All this is supported by the current multi-array. Why not just focus your efforts in enhancing multi-array to include compile time setting of array extents. Note that the multi-array library includes multi_array_ref which is designed to provide the multi-array interface to an array which has already been allocated. Natually in this case it doesn't do any allocation on it's own. It seemst to me that the mult-array has everything we need and that very little is to be gained by trying to replace it. Robert Ramey Daryle W.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
From: ramey@rrsd.com Date: Tue, 8 Oct 2013 21:25:44 -0800
Daryle Walker wrote:
From: ramey@rrsd.com Date: Tue, 8 Oct 2013 08:50:54 -0800
Daryle Walker wrote:
I added a new class template for my library at https://github.com/CTMacUser/ArrayMD. It's called "multiarray," and it's an adapter class like stack and queue. But the new interface is only for accessing elements. Besides the element and container types, the template header for the class takes the number of dimensions. The class uses at, operator [] and operator () to access an element. It has methods to read/write the shape of the array and the priority of each index. There are no methods to change the number of elements stored; you have to either use a fixed-size container or sub-class the multi-array to add methods to change the size. (The container sub-object is protected, like the Standard adapters.) I split the main code into two base class templates, because the code for computing the offset from an index tuple and the code for referencing an element were nearly distinct. This really screws up my attempt at Doxygen comments. Any ideas? Or is my case too complex for what Doxygen's author thought could be handled.
I've looked at the document above. It would seem to me that this is similar or equivalent to boost.multi-array with extents set at compile time rather than at runtime. Is my understanding of this correct? If not how is it wrong.
Both templates get the number of dimensions as a compile-time parameter and the actual extent values at run-time. The Boost version can take the extents and priorities in the constructor, while my version has to change those attributes in separate call(s). But it seems that construction time is the only opportunity for the Boost version to use a different index priority; it can't be changed later, unlike mine.
Hmm- I'm not sure what "index priority" refers to.
That's row-major vs. column-major vs. scrambled. Let's look at a 3x4 array:
1 2 3 4
5 6 7 8
9 10 11 12
In row-major order, the elements are stored as {1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 11, 12} in memory. Given coordinates
Neither version provides methods to directly change the number of elements. The Boost version can change its element count through its resize method.
Hmm - These statements seem to conflict - It seems to me that multi-array can have all the extents adjusted at run time. I can see that this facility could imply extra coding and loss of runtime efficiency. That was the motivation for my original question - it seemed to me there might be a motivation to have the same thing with more stuff fixed at compile time to make it faster at the cost of runtime configurability.
Yeah, it didn't look right after I wrote it. If you want fixed extents, then you would have a fixed-sized container, and Boost.MultiArray is a dynamic one. If you want said fixed-sized container, then use something like Boost.Array or my array_md type, and use an adapter like my multiarray or the Boost (const_)multi_array_ref types if you don't like the default storage order (or other choices for the Boost adapters).
My version can't at all, except through derived classes. But my version decouples the elements stored and what's needed. You only crash when referencing an element beyond the container's bounds. It was designed with std::array as a base container. It should be efficient when the desired size stays close to the array's fixed size.
Hmm - doesn't one fix the array size to the "desired size" at compile time? Aren't they equal. I'm not getting this.
My multiarray adapter holds two sizes. One is the desired size, which is the product of the last submitted set of extents. The other is the actual size, which calls "size()" of the inner protected container. They don't have to match. If they don't match, then you get either unused container elements which can't be addressed or missing ones you shouldn't address. The constructors set the desired size to be the current size (or 1 if the container starts off empty), and the priority to be row-major order. The default constructor works right for std::array, but using a std::vector or deque will start off with an empty container. You can use the other constructors to start the container with elements.
My version indexes only elements; sub-arrays are not supported. It wouldn't be efficient, since elements would not be contiguous in general.
To me this is an absolutly essential feature and one reason I love this library.
C spoiled us with their multidimensional arrays actually being nested linear arrays. So subsets was really easy (as long as you followed row-major order). I think it's more of a secondary action of the Multi-Array concept, rather than primary (like element indexing).
Create a new array object of the smaller size and use (c)apply to copy over what you want.
Which is probably what Boost.Multiarray does in the background of its slicing methods (that return a copy).
You have to walk all the elements to create the new contiguous array. It's not clear how you would do this in your library. The current one does it all automatically taking care of strides and all that. Your scheme just ignores the utility of this feature.
The automatically-defined copy constructor and assignment work if all the template parameters are the same. But for other combinations, do you want to follow in-memory order, or match index-tuple to (mapped) index-tuple? That's why I didn't pick a default. If you're doing copying, you should call (c)apply for the smaller container. Make sure to pass the non-applying container as a lambda argument. [SNIP]
Another class in that GitHub repository, array_md, does do that. However, it's a variadic extension of std::array, and so does not have any of the storage or indexing options that Boost.Multi-Array has. My multiarray adapter was started from so many complaints that array_md doesn't support anything besides row-major order, making it less desirable for all the math & science junkies that wanted column-major (or scrambled-priority) storage.
All this is supported by the current multi-array. Why not just focus your efforts in enhancing multi-array to include compile time setting of array extents. Note that the multi-array library includes multi_array_ref which is designed to provide the multi-array interface to an array which has already been allocated. Natually in this case it doesn't do any allocation on it's own.
It seemst to me that the mult-array has everything we need and that very little is to be gained by trying to replace it.
array_md is designed to be an upgrade of boost::array, not compete with Boost.MultiArray. My multiarray adapter is a first draft at something new for me. They're both made to be possible additions to the Standard, which wouldn't have Boost.Multiarray, i.e. they have to be stand-alone. Daryle W.
Daryle Walker wrote:
From: ramey@rrsd.com Date: Tue, 8 Oct 2013 21:25:44 -0800
Daryle Walker wrote:
From: ramey@rrsd.com Date: Tue, 8 Oct 2013 08:50:54 -0800
Daryle Walker wrote:
[SNIP]
My version indexes only elements; sub-arrays are not supported. It wouldn't be efficient, since elements would not be contiguous in general.
To me this is an absolutly essential feature and one reason I love this library.
C spoiled us with their multidimensional arrays actually being nested linear arrays. So subsets was really easy (as long as you followed row-major order). I think it's more of a secondary action of the Multi-Array concept, rather than primary (like element indexing).
Create a new array object of the smaller size and use (c)apply to copy over what you want.
Which is probably what Boost.Multiarray does in the background of its slicing methods (that return a copy).
I don't think so. I think it just re-maps the indices on the fly. This is more or less what I would have to do by hand if I want to copy out a slice for subsequent faster access. It will be very, very, hard to convince me that this is not an essential feature.
You have to walk all the elements to create the new contiguous array. It's not clear how you would do this in your library. The current one does it all automatically taking care of strides and all that. Your scheme just ignores the utility of this feature.
The automatically-defined copy constructor and assignment work if all the template parameters are the same. But for other combinations, do you want to follow in-memory order, or match index-tuple to (mapped) index-tuple? That's why I didn't pick a default.
Basically, the memory order may be configurable, but it should (mostly) be irrelevant to the usage of the library. This would affect timings of creating mapped slices - but wouldn't really affect the usage of an array.
If you're doing copying, you should call (c)apply for the smaller container. Make sure to pass the non-applying container as a lambda argument.
well, I'd have to think about what this means. It is not necessary to do anything special with the multi-array library. Just create a subview and walk the elements of the subview in the desired order. Or use a standard copy using the interator interface. This is going to be very hard to beat as a transparent, efficient concept.
[SNIP]
It seemst to me that the mult-array has everything we need and that very little is to be gained by trying to replace it.
array_md is designed to be an upgrade of boost::array, not compete with Boost.MultiArray. My multiarray adapter is a first draft at something new for me. They're both made to be possible additions to the Standard, which wouldn't have Boost.Multiarray, i.e. they have to be stand-alone.
well maybe I'm just confused by the calling it "Multiarray". Since it's an enhance to array which is a facade on plain old c arrays, and plane old c arrays can have more than one dimension, maybe it should be just considered an ehance ment to std::array. if one is thinking about adding to the standard (waaaaaay premature in my opinion), then maybe it's boost.multi-array which should be added to the standard. I'm still thinking that boost multi-array is not appreciated to the extent that it should be. I would like to see it enhanced to accomodate compile time extents. Of course even that would have a ripple effect into the multi-array concepts so maybe it's too hard to do. Robert Ramey
Daryle W.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 10/9/2013 12:41 PM, Robert Ramey wrote:
From: ramey@rrsd.com ... I'm still thinking that boost multi-array is not appreciated to the extent
Daryle Walker wrote: that it should be. I would like to see it enhanced to accomodate
I wish this thread had come up 3-4months ago. Unlike you I couldn't grok the docs to use multi-array as a back end to a Qt table where I needed to support run time row/col insertion/deletion. I was under time constraints and found it much easier to build my own facilities. I'm not sure whether it was due to docs or design. Jeff
Jeff Flinn wrote:
On 10/9/2013 12:41 PM, Robert Ramey wrote:
Daryle Walker wrote:
From: ramey@rrsd.com ... I'm still thinking that boost multi-array is not appreciated to the extent that it should be. I would like to see it enhanced to accomodate
I wish this thread had come up 3-4months ago. Unlike you I couldn't grok the docs to use multi-array as a back end to a Qt table where I needed to support run time row/col insertion/deletion. I was under time constraints and found it much easier to build my own facilities. I'm not sure whether it was due to docs or design.
Well, I did come to understand it pretty well. But I really have to say I spent a LOT of time poring over the documentation, experimenting etc. The oddest thing to me is that now it seems easy to me and I can't figure out what I spent the time on - we're talking a few days. I had the same experience (only worse) with units library. Actually I have this experience with almost every boost library I use. This brings up the whole question of documentation of boost libraries and in fact documentation of computer libraries in general. I've spent a lot of time thinking about this lately. I feel we've been too lax in accepting libraries after the review whose documentation isn't up to some minimum standard (not that we have a standard). Robert Ramey
On 10/9/13, Robert Ramey
I'm still thinking that boost multi-array is not appreciated to the extent that it should be. I would like to see it enhanced to accomodate compile time extents. Of course even that would have a ripple effect into the multi-array concepts so maybe it's too hard to do.
On 10/9/13, Robert Ramey
I'm still thinking that boost multi-array is not appreciated to the extent that it should be. I would like to see it enhanced to accomodate compile time extents. Of course even that would have a ripple effect into the multi-array concepts so maybe it's too hard to do.
The maps directory in the Sandbox includes an array like that. It's a fixed size array whose memory can be statically or dynamically allocated, dependent on allocator type. The combination of both is probably a mistake, since a dynamically allocated N-dimensional array requires N memory allocations, whereas a heap allocated static array requires only one. In the static case it generates an intrinsic member array from the set of compile time bounds passed as a template parameter, written before variadic templates became available. It is somewhat similar to Daryle's array_md, and includes a view class to generate subarray views of larger arrays. The directory also contains a resizable array that needs a name change, it's called pointer, multi-array was taken, and like multi-array it's fixed size at any given time. An N-dimensional array of type pointer makes N memory allocations when passed a set of bounds and whose elements can be accessed using the normal indexing operator notation. Accessing views is possible using either the class's type definitions or a view class designed for that purpose. Also include are fixed size scalar, vector and matrix expression templates in case of interest. I have since asked this list for interest in a statically allocated array that makes use of variadic templates. It is currently in the review queue and is located at, https://github.com/BrianJSmith/Array If you require a compile time array you could give that a try. Generating views can be achieved using the class's type definitions, however, no specific view class is available. I have updated the pointer class to use initializer lists during construction, resizing and element access, it is not publicly available, although if there's any interest I can make it so. The N memory allocations clearly make it slower to construct than a similar multi-array, however, with element access it generally has better performance. The view class for it needs some rewriting and perhaps an iterator class. The begin and end methods return the address of the first and one passed the last data elements, although, that is also true of the view class whose elements can be the addresses of elements in the array the view represents. Regards Brian -- www.maidsafe.net
participants (5)
-
Brian Smith -
Daniel James -
Daryle Walker -
Jeff Flinn -
Robert Ramey