Apostolos Apostolidis wrote:
I am using gcc v 4.1.3 on ubuntu 7.10 with 512MB of RAM. Memory is not a problem, since I checked and verified that the thread group does not consume more memory than available. Also, the maximum number of threads defined by the system is 8055, so no problem with that also. Any ideas?
I think the problem comes from address space filling up, rather than actual memory consumption. On many 32-bit operating systems (with a flat address space,) only a limited part of the whole 4 GiB address space is accessible to the program itself (for example, on win32 the limit is 2 GiB in normal cases.) I have no in-depth knowledge about pthreads and linux, but in win32, the default reserved size for each thread's stack is 1 MiB. That's why normally, one can only have ~2000 threads, which amounts to 2000 MiB of address space, plus program code and heap, that make the total 2048 MiB or 2 GiB. Note that this size (at least on Windows) is the "reserved" stack size. It means that until you actually access this pages, no actual memory is allocated, but they consume address space nonetheless. The limit that you mention on the number of threads is probably related to process table size rather than memory limits. (The descriptor tables (GDT and LDT and IDT) on Intel processors are limited to 8192 entries. I wonder whether this is related to that?) In any case, I still think you should investigate either lowering the number of threads you use (hundreds of threads seems A LOT to me for any common application, although I admit that I'm not an expert) or decreasing the default stack reserve size for each. On win32 threads, you specify the stack size at the time of creation, but Boost.Threads does not expose such functionality AFAIK. In MSVC, you can set the default thread stack size that each application creates at link time. It's probable that you can do so on GCC just as well. -yzt -- "Programming is an art that fights back!"