Hi, On 09:59 Sun 16 Oct , degski wrote:
In your example, which I think is flawed as an example, you create in memory a 20+ GB data structure (hope I got the maths right).
300M * 4 * 4B = 2.4GB. IMHO not too large. Even 20GB isn't that much in HPC.
This data structure optimizes one operation (calculating the average) at the cost of pessimising almost any other operation (relocation if the vector needs to grow, insert, delete, push_back etc are all done 4 times in your example)
Keep in mind that this is a sketch of what he would like to use, not a polished solution. The point is: with a Struct-of-Arrays (SoA) memory layout you can implement kernels operating on that data that can be vectorized. Depending on your CPU, vectorization can yield you a speedup of 2x - 8x. If implemented properly then insert, delete, push_back etc. can also be efficient for SoA.
while iterating over the records will land you right into cache-miss-heaven.
Depends. If your access pattern is very random and your algorithm needs to access every member of an object, then a std::vector might be faster. But if your algorithm has some (piecewise) continuity, then the SoA structure will benefit just as well from the CPU's caches. Cheers -Andreas -- ========================================================== Andreas Schäfer HPC and Supercomputing Institute for Multiscale Simulation Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany +49 9131 85-20866 PGP/GPG key via keyserver http://www.libgeodecomp.org ========================================================== (\___/) (+'.'+) (")_(") This is Bunny. Copy and paste Bunny into your signature to help him gain world domination!