generict container for structure of arrays
Over the past few months, I've worked on a generalized structure-of-arrays for use in various scientific applications, and wanted to know if any Boostees had an interest in that sort of container. The library's main goals are 1) Optimized storage by allowing dynamic addition/removal of fields. 2) General performance help (cache coherency, vector processing, etc.) 3) Enabling the use of many types in a single SOA. 4) Provide an interface similar other STL containers 5) Type safety (of course) The library is similar to a tuple of vectors, except all vectors are forced to have the same size, and the members of the tuple aren't fixed. Internally, the data is stored as vectors for each field, but there's also an interface for getting a slice of the data. Chase Bradford
----- Original Message ----- From: "Bradford, Chase" <CHASE.BRADFORD@saic.com> To: <boost@lists.boost.org> Sent: Saturday, July 12, 2008 1:20 AM Subject: [boost] generict container for structure of arrays
Over the past few months, I've worked on a generalized structure-of-arrays for use in
various scientific applications, and wanted to know if any Boostees had an interest
in that sort of container.
The library's main goals are
1) Optimized storage by allowing dynamic addition/removal of fields.
2) General performance help (cache coherency, vector processing, etc.)
3) Enabling the use of many types in a single SOA.
4) Provide an interface similar other STL containers
5) Type safety (of course)
The library is similar to a tuple of vectors, except all vectors are forced to have the
same size, and the members of the tuple aren't fixed. Internally, the data is stored
as vectors for each field, but there's also an interface for getting a slice of the data.
Hi, could you give more precisse information. I'm unable to understand what is the goal of your library. Best, Vicente
Seeing a sample would be fine. On 7/24/08, vicente.botet <vicente.botet@wanadoo.fr> wrote:
----- Original Message ----- From: "Bradford, Chase" < CHASE.BRADFORD@saic.com> To: <boost@lists.boost.org> Sent: Saturday, July 12, 2008 1:20 AM Subject: [boost] generict container for structure of arrays
Over the past few months, I've worked on a generalized
structure-of-arrays for use in
various scientific applications, and wanted to know if any Boostees had an interest
in that sort of container.
The library's main goals are
1) Optimized storage by allowing dynamic addition/removal of fields.
2) General performance help (cache coherency, vector processing, etc.)
3) Enabling the use of many types in a single SOA.
4) Provide an interface similar other STL containers
5) Type safety (of course)
The library is similar to a tuple of vectors, except all vectors are forced to have the
same size, and the members of the tuple aren't fixed. Internally, the data is stored
as vectors for each field, but there's also an interface for getting a slice of the data.
Hi,
could you give more precisse information. I'm unable to understand what is the goal of your library.
Best,
Vicente
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
-- Alp Mestan --- http://blog.mestan.fr/ --- http://alp.developpez.com/ --- In charge of the Qt, Algorithms and Artificial Intelligence sections on Developpez
The following example shows how to use some of the interface for the class. The container is a basically a map of variants, but internally it holds variant<vector<T1>, vector<T2>, ...>. The result is a container that provides access to individual fields for fast processing in parallel. The list of fields is growable, because that was one of the requirements for the program I'm working on. I'm not sure that's a good feature to have in general though. Chase template<class Key, class Variant> class basic_soa; int main() { typedef basic_soa<std::string, boost::variant<int, float> > MySoa; MySoa s; // Make internal vectors hold 100 elements. s.resize(100); // Create a fields named x and y. This sort of makes the object a // container of 2D points. s.add_field<float>("x"); s.add_field<float>("y"); fill( s.field<float>("x").begin(), s.field<float>("x").end(), rand); fill( s.field<float>("y").begin(), s.field<float>("y").end(), rand); float x_sum = 0.0f; float y_sum = 0.0f; // Inefficiently work over the slices for( MySoa::iterator iter=s.begin(); iter != s.end(); ++iter ) { x_sum += iter->get<float>("x"); y_sum += iter->get<float>("y"); } // Add a new field for numbering the points. s.add_field<int>("index"); // Apply indexing by slice for( size_t i=0; i<s.size(); ++i ) { // This is also inefficient. s[i].get<int>("index") = i; } // Redo the indexing by only accessing the "index" field. std::vector<int>& indexes = s.get_field<int>("index"); for( size_t i=0; i<indexes.size(); ++i ) { indexes[i] = i; } } -----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of vicente.botet Sent: Thursday, July 24, 2008 10:47 AM To: boost@lists.boost.org Subject: Re: [boost] generict container for structure of arrays ----- Original Message ----- From: "Alp Mestan" <alpmestan@gmail.com> To: <boost@lists.boost.org> Sent: Thursday, July 24, 2008 7:11 PM Subject: Re: [boost] generict container for structure of arrays
Seeing a sample would be fine.
yes this can help. Vicente _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Wouldn't a vector of boost::any act like that ? On 7/24/08, Bradford, Chase <CHASE.BRADFORD@saic.com> wrote:
The following example shows how to use some of the interface for the class. The container is a basically a map of variants, but internally it holds variant<vector<T1>, vector<T2>, ...>.
The result is a container that provides access to individual fields for fast processing in parallel. The list of fields is growable, because that was one of the requirements for the program I'm working on. I'm not sure that's a good feature to have in general though.
Chase
template<class Key, class Variant> class basic_soa;
int main() { typedef basic_soa<std::string, boost::variant<int, float> > MySoa;
MySoa s;
// Make internal vectors hold 100 elements. s.resize(100);
// Create a fields named x and y. This sort of makes the object a // container of 2D points. s.add_field<float>("x"); s.add_field<float>("y");
fill( s.field<float>("x").begin(), s.field<float>("x").end(), rand); fill( s.field<float>("y").begin(), s.field<float>("y").end(), rand);
float x_sum = 0.0f; float y_sum = 0.0f;
// Inefficiently work over the slices for( MySoa::iterator iter=s.begin(); iter != s.end(); ++iter ) { x_sum += iter->get<float>("x"); y_sum += iter->get<float>("y"); }
// Add a new field for numbering the points. s.add_field<int>("index");
// Apply indexing by slice for( size_t i=0; i<s.size(); ++i ) { // This is also inefficient. s[i].get<int>("index") = i; }
// Redo the indexing by only accessing the "index" field. std::vector<int>& indexes = s.get_field<int>("index"); for( size_t i=0; i<indexes.size(); ++i ) { indexes[i] = i;
} }
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of vicente.botet Sent: Thursday, July 24, 2008 10:47 AM To: boost@lists.boost.org
Subject: Re: [boost] generict container for structure of arrays
----- Original Message ----- From: "Alp Mestan" <alpmestan@gmail.com> To: <boost@lists.boost.org> Sent: Thursday, July 24, 2008 7:11 PM Subject: Re: [boost] generict container for structure of arrays
Seeing a sample would be fine.
yes this can help.
Vicente
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
-- Alp Mestan --- http://blog.mestan.fr/ --- http://alp.developpez.com/ --- In charge of the Qt, Algorithms and Artificial Intelligence sections on Developpez
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Alp Mestan Sent: Thursday, July 24, 2008 1:33 PM To: boost@lists.boost.org Subject: Re: [boost] generict container for structure of arrays
Wouldn't a vector of boost::any act like that ?
My first draft used boost::any, but it became a pain to do things like resize the internal containers or manage their iterators. By using a variant, where its types are all vector<T>, I could write static visitor implementations that work against the container or iterator interface, without having to know about the contained type. Chase
Bradford, Chase wrote:
int main() { typedef basic_soa<std::string, boost::variant<int, float> > MySoa;
MySoa s;
// Make internal vectors hold 100 elements. s.resize(100);
// Create a fields named x and y. This sort of makes the object a // container of 2D points. s.add_field<float>("x"); s.add_field<float>("y");
fill( s.field<float>("x").begin(), s.field<float>("x").end(), rand); fill( s.field<float>("y").begin(), s.field<float>("y").end(), rand);
If I understand your example correctly, it's quite similar to a std::(unordered_)map<Key, std::vector<Value> >.
On Thursday 24 July 2008 14:02:37 Mathias Gaunard wrote:
Bradford, Chase wrote:
int main() { typedef basic_soa<std::string, boost::variant<int, float> > MySoa;
MySoa s;
// Make internal vectors hold 100 elements. s.resize(100);
// Create a fields named x and y. This sort of makes the object a // container of 2D points. s.add_field<float>("x"); s.add_field<float>("y");
fill( s.field<float>("x").begin(), s.field<float>("x").end(), rand); fill( s.field<float>("y").begin(), s.field<float>("y").end(), rand);
If I understand your example correctly, it's quite similar to a std::(unordered_)map<Key, std::vector<Value> >. except that it seems to be able to handle different value-types.
In my field, this is what we call an n-tuple: http://www.gnu.org/software/gsl/manual/html_node/N_002dtuples.html http://www.gnu.org/software/gsl/manual/html_node/Ntuple-References-and-Furth... cheers, sebastien. -- ################################### # Dr. Sebastien Binet # # Lawrence Berkeley National Lab. # # 1 Cyclotron Road # # Berkeley, CA 94720 # ###################################
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Mathias Gaunard Sent: Thursday, July 24, 2008 2:03 PM To: boost@lists.boost.org Subject: Re: [boost] generict container for structure of arrays
Bradford, Chase wrote:
int main() { typedef basic_soa<std::string, boost::variant<int, float> > MySoa;
MySoa s;
// Make internal vectors hold 100 elements. s.resize(100);
// Create a fields named x and y. This sort of makes the object a // container of 2D points. s.add_field<float>("x"); s.add_field<float>("y");
fill( s.field<float>("x").begin(), s.field<float>("x").end(), rand); fill( s.field<float>("y").begin(), s.field<float>("y").end(), rand);
If I understand your example correctly, it's quite similar to a std::(unordered_)map<Key, std::vector<Value> >.
In many ways they are similar except: 1) All map value types are vector<T>. In the example, the underlying types can simultaneously be vector<int> or vector<float>, since its driven by a variant. 2) A map of key, vector pairs cannot ensure that all vectors are identical in length. 3) Map doesn't provide any mechanism for viewing slices of the data. The main purpose of this container is to store data in a structure-of-arrays layout, instead of an array-of-structures. A quick comparison of the two approaches can be found here: hectorgon.blogspot.com/2006/08/array-of-structures-vs-structure-of.html Chase
I once created something like that, which used boost::zip_iterator to iterate through the vectors. Is that similar to your approach?
participants (6)
-
Alp Mestan -
Bradford, Chase -
Kevin Sopp -
Mathias Gaunard -
Sebastien Binet -
vicente.botet