RE: [Boost-Users] more then 254 threads?
Eske Christiansen said:
Hej boost-users.
I'm compiling a test thread program where I create 255 thread but I get an "Program received signal SIGABRT, Aborted." after thread 254 I running redhat 7.3 and using boost 1.28.00. Is it an OS problem or a boost problem?
I find it strange that it stopped at 255 threads. Maximum number of threads in linux is 1024 as defined by PTHREAD_THREADS_MAX in /usr/include/bits/local_lim.h. With default stacksize you would have allocated 500M of virtual memory, but I am not too sure if that would result in thread creation failure.
Eske Christiansen said:
Hej boost-users.
I'm compiling a test thread program where I create 255 thread but I get an "Program received signal SIGABRT, Aborted." after thread 254 I running redhat 7.3 and using boost 1.28.00. Is it an OS problem or a boost problem?
I find it strange that it stopped at 255 threads. Maximum number of threads in linux is 1024 as defined by PTHREAD_THREADS_MAX in /usr/include/bits/local_lim.h. With default stacksize you would have allocated 500M of virtual memory, but I am not too sure if that would result in thread creation failure.
To quote the FAQ I posted a link to in my response: "D.10: My application needs to create thousands of threads, or maybe even more. Can I do this with LinuxThreads? No. You're going to run into several hard limits: * Each thread, from the kernel's standpoint, is one process. Stock Linux kernels (version 2.2 and earlier) are limited to at most 512 processes for the super-user, and half this number for regular users. This can be changed by changing NR_TASKS in include/linux/tasks.h and recompiling the kernel. On the x86 processors at least, architectural constraints seem to limit NR_TASKS to 4090 at most. (It seems that 2.4 kernels have higher limits, though.) * LinuxThreads contains a table of all active threads. This table has room for 1024 threads at most. To increase this limit, you must change PTHREAD_THREADS_MAX in the LinuxThreads/glibc sources and recompile. * By default, each thread reserves 2M of virtual memory space for its stack. This space is just reserved; actual memory is allocated for the stack on demand. But still, on a 32-bit processor, the total virtual memory space available for the stacks is on the order of 1G, meaning that more than 500 threads will have a hard time fitting in. You can overcome this limitation by moving to a 64-bit platform, or by allocating smaller stacks yourself using the setstackaddr attribute. * Finally, the Linux kernel contains many algorithms that run in time proportional to the number of process table entries. Increasing this number drastically will slow down the kernel operations noticeably. (Other POSIX threads libraries have similar limitations, by the way.) For all these reasons, you'd better restructure your application so that it doesn't need more than, say, 100 threads. For instance, in the case of a multithreaded server, instead of creating a new thread for each connection, maintain a fixed-size pool of worker threads that pick incoming connection requests from a queue." I can't begin to explain why PTHREADS_THREADS_MAX is set to 1024 by default, when there's either a 512 or 256 hard limit in the OS by default, but that's not really relevant. It appears he hit the 256 limit on the number of processes allowed for user accounts (since a thread in Linux is actually a process). William E. Kempf
I came to to the same conclusion and it set me back a little. My ideer to create that many threads was to code a kind of AI Life, where each life square lives in its own thread. It was only to test how well the linux kernel handle the creation and delettion of many threads each second. So I will insted try to make a micro-thread system inside 255 threads and se how well the kernel scale. Does windows 2k and after have the same problems? yours eske
Eske Christiansen said:
Hej boost-users.
I'm compiling a test thread program where I create 255 thread but I get an "Program received signal SIGABRT, Aborted." after thread 254 I running redhat 7.3 and using boost 1.28.00. Is it an OS problem or a boost problem?
I find it strange that it stopped at 255 threads. Maximum number of threads in linux is 1024 as defined by PTHREAD_THREADS_MAX in /usr/include/bits/local_lim.h. With default stacksize you would have allocated 500M of virtual memory, but I am not too sure if that would result in thread creation failure.
To quote the FAQ I posted a link to in my response: "D.10: My application needs to create thousands of threads, or maybe even more. Can I do this with LinuxThreads? No. You're going to run into several hard limits: * Each thread, from the kernel's standpoint, is one process. Stock Linux kernels (version 2.2 and earlier) are limited to at most 512 processes for the super-user, and half this number for regular users. This can be changed by changing NR_TASKS in include/linux/tasks.h and recompiling the kernel. On the x86 processors at least, architectural constraints seem to limit NR_TASKS to 4090 at most. (It seems that 2.4 kernels have higher limits, though.) * LinuxThreads contains a table of all active threads. This table has room for 1024 threads at most. To increase this limit, you must change PTHREAD_THREADS_MAX in the LinuxThreads/glibc sources and recompile. * By default, each thread reserves 2M of virtual memory space for its stack. This space is just reserved; actual memory is allocated for the stack on demand. But still, on a 32-bit processor, the total virtual memory space available for the stacks is on the order of 1G, meaning that more than 500 threads will have a hard time fitting in. You can overcome this limitation by moving to a 64-bit platform, or by allocating smaller stacks yourself using the setstackaddr attribute. * Finally, the Linux kernel contains many algorithms that run in time proportional to the number of process table entries. Increasing this number drastically will slow down the kernel operations noticeably. (Other POSIX threads libraries have similar limitations, by the way.) For all these reasons, you'd better restructure your application so that it doesn't need more than, say, 100 threads. For instance, in the case of a multithreaded server, instead of creating a new thread for each connection, maintain a fixed-size pool of worker threads that pick incoming connection requests from a queue." I can't begin to explain why PTHREADS_THREADS_MAX is set to 1024 by default, when there's either a 512 or 256 hard limit in the OS by default, but that's not really relevant. It appears he hit the 256 limit on the number of processes allowed for user accounts (since a thread in Linux is actually a process). William E. Kempf Info: http://www.boost.org Wiki: http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl Unsubscribe: mailto:boost-users-unsubscribe@yahoogroups.com Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
participants (3)
-
Eske Christiansen -
schalk_cronje＠nai.com -
William E. Kempf