Hey, So with three responses so far all talking about stack allocation, I will clarify by saying that on the 32bit and 64bit linux platform that we are running with the default stack size is 8mb. However this is expected and we see exactly that in the profiling results. The large allocation (which is indeed all virtual) on 64bit comes from: line 45 in once.cpp: data=malloc(sizeof(boost::uintmax_t)); // mmap's about 90mb per thread I have done some additional digging and this doesn't seem to specifically be a boost problem as I first thought. I have found that if you allocate any memory in a pthread the large allocation is seen. I am unsure as to why this is happening, although I am aware that virtual memory is relatively 'free' on 64bit systems so it might just be a greedy optimisation of some type. Would still be interested if someone could clarify exactly what is going on. On 27/08/12 21:44, Jeff Flinn wrote:

On 8/26/2012 9:14 PM, Conrad Mercer wrote:

...

Hey,

I have been using boost threads on 32bit linux for some time and am very happy with their performance so far. Recently the project was moved to a 64bit platform and we saw a huge increase in memory usage (from about 2.5gb to 16-17gb). I have done profiling and found that the boost threads are the source of the huge allocation. Each thread is allocating about 10x what it was doing on 32bit.

I profiled using valgrind's massif and have confirmed the issue using only boost threads in a separate test application. I also tried using std::threads instead and these do not exhibit the large memory allocation issue.

I am wondering if anyone else has seen this behaviour and knows what the problem is? Well each thread get's it's own stack.

Jeff

_______________________________________________ Unsubscribe& other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Le 28/08/12 18:32, Phil Endecott a écrit :

Conrad Mercer wrote:

...

I have been using boost threads on 32bit linux for some time and am very happy with their performance so far. Recently the project was moved to a 64bit platform and we saw a huge increase in memory usage (from about 2.5gb to 16-17gb). I have done profiling and found that the boost threads are the source of the huge allocation. Each thread is allocating about 10x what it was doing on 32bit.

I profiled using valgrind's massif and have confirmed the issue using only boost threads in a separate test application. I also tried using std::threads instead and these do not exhibit the large memory allocation issue.

I believe that 32-bit Linux sets the default stack size to 2 MB, and much larger on 64-bit. So I would expect to see it using much more _virtual_ memory - but if you don't actually use it, then no physical memory is ever mapped to those pages and there is no performance or other concern.

A deficiency of Boost.Thread and std::thread is that they provide no way to change the stack size of new threads. This has been discussed a few times here. It is possible that the libstdc++ implementation of std::thread chooses to set a smaller default stack size.

Boost.Thread contains since Boost 1.50 a way to set thread attributes and in particular the stack size #2741 <http://svn.boost.org/trac/boost/ticket/2741> Proposal to manage portable and non portable thread attributes. See http://www.boost.org/doc/libs/1_51_0/doc/html/thread/thread_management.html#... and http://www.boost.org/doc/libs/1_51_0/doc/html/thread/thread_management.html#.... Best, Vicente