On 01/28/2011 01:01 PM, Dave Abrahams wrote:
At Fri, 28 Jan 2011 12:26:59 -0800 (PST), Artyom wrote:
Give me a break... Who uses NUMA for string processing?!
Anyone running on a multiprocessor system with AMD Opterons or Intel Nehalem or Tukwila processors. You don't always get to choose the kind of architecture your code will run on, and those systems are all NUMA. But even when you do get to choose, some very large problems that would be appropriate to NUMA involve lots of strings.
What hasn't been explained is how NUMA is relevant to the design of this string type. As far as I understand, the main consideration required for NUMA systems is to try to have each processor use the portion of memory that it can access most efficiently. Potentially this suggests that if a large string is to be shared read-only between two threads running on different NUMA nodes, it might be advantageous for it to be copied into memory that can be accessed quickly by the second thread (assuming it is initially in memory optimal for the first thread). Thus, it seems it would be useful for a string/buffer type that is by default reference-counted/COW to also support explicit copying. This could also be useful for a reference-counted string type with a compile time policy of using non-locking, non-atomic operations to handle the reference counting (for use only by a single thread), as atomic operations often incur significant overhead compared to a non-atomic operation. However, as far as whether to use contiguous virtual memory or break the string up into non-contiguous chunks, I don't see the relevance.