Multi_array: A "proper" iterator over a N>1 Multi_array?

I'm using multi_array types as containers for some things I'm working on, and the number of dimensions can get quite large - often over 7, sometimes as high as 11. I'd like to do things such as initialize these multi_arrays, access all elements within them, and other such 'reasonable' operations. Thing is, the only iterator interface to multi_array returns a reference to the nested containers. Thus, to iterate over all sub- elements, I'm forced to have 7-11 nested for-loops to iterate over the entire thing. Is there an existent means of simply iterating over the entire set of data in the multi_array? Has anyone implemented such a thing, or is such a thing something to be desired within boost::multi_array? It seems like such a iterator would be especially useful if also implemented/supported within subgroups, allowing the user to define sub-ranges of the multi_array, then iterate over each element therein, without having to worry about dimensionality, et cetera. Any thoughts, ideas, or pointers to how to easily (without tons of code) iterate over an 11-dimensioned object is appreciated. In conclusion then, I've got two problems. One is "How can I simplify my each-element access to a high-dimensional multi_array", and the second is "Should there be a direct-to-element iterator in multi_array, and does the documentation adequately address the issue of whether or not an iterator points to elements directly or to a sub- graph?" Thanks for your time and troubles, - Greg Link

I guess what you want is origin(). Iterate from origin() to origin+num_elements Antonio On 4/25/06, Greg Link wrote:
I'm using multi_array types as containers for some things I'm working on, and the number of dimensions can get quite large - often over 7, sometimes as high as 11. I'd like to do things such as initialize these multi_arrays, access all elements within them, and other such 'reasonable' operations.
Thing is, the only iterator interface to multi_array returns a reference to the nested containers. Thus, to iterate over all sub- elements, I'm forced to have 7-11 nested for-loops to iterate over the entire thing.
Is there an existent means of simply iterating over the entire set of data in the multi_array? Has anyone implemented such a thing, or is such a thing something to be desired within boost::multi_array? It seems like such a iterator would be especially useful if also implemented/supported within subgroups, allowing the user to define sub-ranges of the multi_array, then iterate over each element therein, without having to worry about dimensionality, et cetera.
Any thoughts, ideas, or pointers to how to easily (without tons of code) iterate over an 11-dimensioned object is appreciated.
In conclusion then, I've got two problems. One is "How can I simplify my each-element access to a high-dimensional multi_array", and the second is "Should there be a direct-to-element iterator in multi_array, and does the documentation adequately address the issue of whether or not an iterator points to elements directly or to a sub- graph?"
Thanks for your time and troubles,
- Greg Link
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

On Tue, 25 Apr 2006 20:15:37 -0300, Antonio Piccolboni
I guess what you want is origin(). Iterate from origin() to origin+num_elements
I think you meant data(). I also think that does not work if you have a multiarray that is a subset of another, because the elements are not contiguous. I also would like to iterate easily over all elements of a multiarray. Maybe a foreach-like function is easier to write than a general iterator. Bruno

I would agree with you there - a 'foreach' would do what I need as well, but as I'm not 100% familiar with the way libraries such as boost are implemented, I worry that making your own control structure is much more difficult than making an accessor/modifier pair. I can't even imagine the syntax needed to convert the following into a theoretical boost-defined 'foreach' double previous = 0; double accumulator = 0; for(iterator_t myIterator = m_array.data(); myIterator != m_array.end (); myIterator++) { accumulator += (*myIterator); (*myIterator) *= previous accumulator = (previous > 7) ? (do_function(accumulator)) : (0); } Sure, it's a contrived example, but I just don't know how a primarily header library such as boost (or any library for that matter) would be able to define a foreach(m_array) { } in a syntactically and compiler-friendly manner. That doesn't mean, however, that I'm still not interested in the possibility of using .data() and iterating, I'm just concerned that if the data in the container isn't sequential, I'm violating some law. I know (from testing) that making an iterator myIterator = m_array.begin(); and incrementing it only goes as high as the size of the highest dimension before it hits m_array.end(), so I'm concerned that doing myIterator = m_array.data(); will end at the same point (or just walk off into nowheresville). Don't have my test code in front of me until tomorrow, where I'll test for sure. - Greg On Apr 25, 2006, at 9:02 PM, Bruno Martínez wrote:
On Tue, 25 Apr 2006 20:15:37 -0300, Antonio Piccolboni
wrote: I guess what you want is origin(). Iterate from origin() to origin+num_elements
I think you meant data(). I also think that does not work if you have a multiarray that is a subset of another, because the elements are not contiguous.
I also would like to iterate easily over all elements of a multiarray. Maybe a foreach-like function is easier to write than a general iterator.
Bruno
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

On Tue, 25 Apr 2006 23:11:26 -0300, Greg Link wrote:
I would agree with you there - a 'foreach' would do what I need as well, but as I'm not 100% familiar with the way libraries such as boost are implemented, I worry that making your own control structure is much more difficult than making an accessor/modifier pair. I can't even imagine the syntax needed to convert the following into a theoretical boost-defined 'foreach'
I didn't mean new sintax, but a std::for_each work-alike.
double previous = 0; double accumulator = 0; for(iterator_t myIterator = m_array.data(); myIterator != m_array.end (); myIterator++) { accumulator += (*myIterator); (*myIterator) *= previous accumulator = (previous > 7) ? (do_function(accumulator)) : (0); }
You can do that with Boost.Lambda:
#include

Upon reflection I realized that
a) My code was broken
b) origin() returns an element* and not any smarter iterator, so that layout
is important
c) I would propose as a solution a call that returns an iterator pointing to
the first element of a multi array and when incremented goes through all the
elements one by one all the way to the end
something like
for (iter=marray.element_begin(); iter !=marray.element_end(); iter ++) {
*iter+=1;
}
would increment all the elements of the multi_array marray by 1. This should
work for views and independent of number of dimensions and layout. I think
it would be easily implementable as a skip iterator where the increment
operator increments a *element by a number that's the product of the
appropriate strides. Dereferencing semantics is the same as element*. Now
everything rests of what "appropriate" means, but I guess whoever
implemented how to increment any index in a view knows how to pull this off
as well -- that is Ron, any comments? I think this would be a powerful
extension. Once this is in place there would be no need for specialized
algorithms -- for_each, transform and even binary_search would do exactly
what people want them to do, like in
for_each(marray.element_begin(); marray.element_end(), _1++)
equivalent to the loop above. Another possibility that reminds me of the
statistical language R is that we could have a call as_vector that flattens
an multi array into a 1D view. Then begin_element above would be equivalent
to begin() on as_vector(marray).
Antonio
On 4/25/06, Bruno Martínez
On Tue, 25 Apr 2006 23:11:26 -0300, Greg Link wrote:
I would agree with you there - a 'foreach' would do what I need as well, but as I'm not 100% familiar with the way libraries such as boost are implemented, I worry that making your own control structure is much more difficult than making an accessor/modifier pair. I can't even imagine the syntax needed to convert the following into a theoretical boost-defined 'foreach'
I didn't mean new sintax, but a std::for_each work-alike.
double previous = 0; double accumulator = 0; for(iterator_t myIterator = m_array.data(); myIterator != m_array.end (); myIterator++) { accumulator += (*myIterator); (*myIterator) *= previous accumulator = (previous > 7) ? (do_function(accumulator)) : (0); }
You can do that with Boost.Lambda:
#include
#include #include #include <algorithm> int main() { double previous = 0; double accumulator = 0; double d[3] = {1, 2, 3};
using boost::lambda::_1; using boost::lambda::if_then_else_return; using boost::lambda::var; using boost::lambda::bind; std::for_each(&d[0], &d[3], ( accumulator += _1, _1 *= previous, var(accumulator) = if_then_else_return(var(previous) > 7, bind(&do_function, accumulator), 0))); }
I spend some time today writing a std::for_each-like function for multiarrays. This works correctly for subviews, but isn't as convenient as an iterator. You can't iterate through two multiarrays at the same time, for example. I only tested with MSVC8, so there may be some typenames missing. Here it is:
template
struct for_each_impl { F f; for_each_impl(F f) : f(f) {} void operator()(MA& ma) { std::for_each(ma.begin(), ma.end(), for_each_impl
(f)); } };
template
struct for_each_impl { F f; for_each_impl(F f) : f(f) {} void operator()(Ref r) { f(r); } };
template
void for_each(MA& ma, F f) { for_each_impl impl(f); impl(ma); } And a test:
typedef boost::multi_array
array_type; typedef array_type::index index; array_type myarray(boost::extents[3][4][2]); typedef array_type::index_range range; array_type::array_view<3>::type myview = myarray[ boost::indices[range(0,2)][range(1,3)][range(0,4,2)] ]; for_each(myview, _1 = 56); Hope that helps.
Bruno
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (3)
-
Antonio Piccolboni
-
Bruno Martínez
-
Greg Link