
Hi all. I have developed some code allowing a thread group of 500 threads to
run.
However, as soon as the 381st thread is spawned, I get the following runtime
error :
terminate called after throwing an instance of
'boost::thread_resource_error'
what(): boost::thread_resource_error
The Boost documentation does not clarify the nature of the error, so I was
wondering if anyone
can help me out here. The (simple) program code follows.
#include

Apostolos Apostolidis wrote:
Hi all. I have developed some code allowing a thread group of 500 threads to run. However, as soon as the 381st thread is spawned, I get the following runtime error :
terminate called after throwing an instance of 'boost::thread_resource_error' what(): boost::thread_resource_error
The Boost documentation does not clarify the nature of the error, so I was wondering if anyone can help me out here. The (simple) program code follows.
I can't be sure, but it can be that your program has run out of stack space to assign to new threads (only one possibility.) You've not mentioned what platform and compiler you use. I don't know whether it's possible, but increase the limit of the application stack (should be among linker options) or set a lower per-thread stack size.
#include
#include #include using namespace std;
boost::mutex io_mutex;
void foo(int i) { boost::mutex::scoped_lock lock(io_mutex); cout << i << endl; }
int main(int argc, char *arvg[]) { boost::thread_group tg;
for(int i = 0;i < 500;i++) tg.create_thread(boost::bind(&foo,i)); tg.join_all();
}
On the other hand, it is possible in this program you've provided that the threads created at the beginning terminate before other threads are created, specially if the "cout" is fast compared to thread creation. When I tested this on Windows XP, using VC8sp1, no more than a few (3-4) threads were alive at any given time and the program ended normally. When I changed the foo() function to: void foo(int i) { { boost::mutex::scoped_lock lock(io_mutex); cout << i << endl; } sleep_relative (1000.0); // Sleep 1000 seconds from now. } I could create 2022 additional threads before the program would run out of resource and throw the exception you mentioned. -yzt -- "Programming is an art that fights back!"

On Mon, 2007-10-22 at 09:51 +0100, Apostolos Apostolidis wrote:
Hi all. I have developed some code allowing a thread group of 500 threads to run. However, as soon as the 381st thread is spawned, I get the following runtime error :
terminate called after throwing an instance of 'boost::thread_resource_error' what(): boost::thread_resource_error
That's a lot of threads. For both win32 threading and Posix pthreads I have an idea each thread consumes at least a megabyte of address space and who knows what other limited resources. How much memory do you have ? Generally you want to structure your apps so that there is some reasonable number of active "compute" threads (where "reasonable" is some number not much higher than the number of CPU cores you have) and so that the number of "I/O threads" (anything expected to make a blocking call on an external service or device) is kept under control too. Tools you might use to do this include mutex-protected queues of work items being fed to the compute threads, and boost::asio. Spawning 100s of threads is a fun thing to do, but it's not a very efficient use of resources, and the OSes aren't really expecting anyone to do it. Have fun Tim [Of course, if you are actually developing on some monster machine with 100s of cores, please ignore the above!]

Apostolos Apostolidis wrote:
Hi all. I have developed some code allowing a thread group of 500 threads to run. However, as soon as the 381st thread is spawned, I get the following runtime error :
terminate called after throwing an instance of 'boost::thread_resource_error' what(): boost::thread_resource_error
The Boost documentation does not clarify the nature of the error, so I was wondering if anyone can help me out here. The (simple) program code follows.
I can't be sure, but it can be that your program has run out of stack space to assign to new threads (only one possibility.) You've not mentioned what platform and compiler you use. I don't know whether it's possible, but increase the limit of the application stack (should be among linker options) or set a lower per-thread stack size.
#include
#include #include using namespace std;
boost::mutex io_mutex;
void foo(int i) { boost::mutex::scoped_lock lock(io_mutex); cout << i << endl; }
int main(int argc, char *arvg[]) { boost::thread_group tg;
for(int i = 0;i < 500;i++) tg.create_thread(boost::bind(&foo,i)); tg.join_all();
}
On the other hand, it is possible in this program you've provided that the threads created at the beginning terminate before other threads are created, specially if the "cout" is fast compared to thread creation. When I tested this on Windows XP, using VC8sp1, no more than a few (3-4) threads were alive at any given time and the program ended normally. When I changed the foo() function to: void foo(int i) { { boost::mutex::scoped_lock lock(io_mutex); cout << i << endl; } sleep_relative (1000.0); // Sleep 1000 seconds from now. } I could create 2022 additional threads before the program would run out of resource and throw the exception you mentioned. -yzt -- "Programming is an art that fights back!"

I am using gcc v 4.1.3 on ubuntu 7.10 with 512MB of RAM. Memory is not a
problem, since I checked
and verified that the thread group does not consume more memory than
available. Also, the maximum
number of threads defined by the system is 8055, so no problem with that
also. Any ideas?
On 22/10/2007, Yaser Zhian
Apostolos Apostolidis wrote:
Hi all. I have developed some code allowing a thread group of 500 threads to run. However, as soon as the 381st thread is spawned, I get the following runtime error :
terminate called after throwing an instance of 'boost::thread_resource_error' what(): boost::thread_resource_error
The Boost documentation does not clarify the nature of the error, so I was wondering if anyone can help me out here. The (simple) program code follows.
I can't be sure, but it can be that your program has run out of stack space to assign to new threads (only one possibility.) You've not mentioned what platform and compiler you use. I don't know whether it's possible, but increase the limit of the application stack (should be among linker options) or set a lower per-thread stack size.
#include
#include #include using namespace std;
boost::mutex io_mutex;
void foo(int i) { boost::mutex::scoped_lock lock(io_mutex); cout << i << endl; }
int main(int argc, char *arvg[]) { boost::thread_group tg;
for(int i = 0;i < 500;i++) tg.create_thread(boost::bind(&foo,i)); tg.join_all();
}
On the other hand, it is possible in this program you've provided that the threads created at the beginning terminate before other threads are created, specially if the "cout" is fast compared to thread creation. When I tested this on Windows XP, using VC8sp1, no more than a few (3-4) threads were alive at any given time and the program ended normally.
When I changed the foo() function to:
void foo(int i) { { boost::mutex::scoped_lock lock(io_mutex); cout << i << endl; } sleep_relative (1000.0); // Sleep 1000 seconds from now. }
I could create 2022 additional threads before the program would run out of resource and throw the exception you mentioned.
-yzt
-- "Programming is an art that fights back!"
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Apostolos Apostolidis wrote:
I am using gcc v 4.1.3 on ubuntu 7.10 with 512MB of RAM. Memory is not a problem, since I checked and verified that the thread group does not consume more memory than available. Also, the maximum number of threads defined by the system is 8055, so no problem with that also. Any ideas?
I think the problem comes from address space filling up, rather than actual memory consumption. On many 32-bit operating systems (with a flat address space,) only a limited part of the whole 4 GiB address space is accessible to the program itself (for example, on win32 the limit is 2 GiB in normal cases.) I have no in-depth knowledge about pthreads and linux, but in win32, the default reserved size for each thread's stack is 1 MiB. That's why normally, one can only have ~2000 threads, which amounts to 2000 MiB of address space, plus program code and heap, that make the total 2048 MiB or 2 GiB. Note that this size (at least on Windows) is the "reserved" stack size. It means that until you actually access this pages, no actual memory is allocated, but they consume address space nonetheless. The limit that you mention on the number of threads is probably related to process table size rather than memory limits. (The descriptor tables (GDT and LDT and IDT) on Intel processors are limited to 8192 entries. I wonder whether this is related to that?) In any case, I still think you should investigate either lowering the number of threads you use (hundreds of threads seems A LOT to me for any common application, although I admit that I'm not an expert) or decreasing the default stack reserve size for each. On win32 threads, you specify the stack size at the time of creation, but Boost.Threads does not expose such functionality AFAIK. In MSVC, you can set the default thread stack size that each application creates at link time. It's probable that you can do so on GCC just as well. -yzt -- "Programming is an art that fights back!"

On Mon, 2007-22-10 at 09:51 +0100, Apostolos Apostolidis wrote:
Hi all. I have developed some code allowing a thread group of 500 threads to run. However, as soon as the 381st thread is spawned, I get the following runtime error :
I think you want to take a look at _POSIX_THREAD_THREADS_MAX or PTHREADS_THREADS_MAX. One of the two (I forget which) dictates what you are guaranteed to be able to do. Sohail

I have checked both variables and set them way beyond 400. Nothing changed
and I keep getting the same error.
Note that this also happens if I spawn POSIX threads from a C program. Any
ideas (again)?
On 23/10/2007, Sohail Somani
On Mon, 2007-22-10 at 09:51 +0100, Apostolos Apostolidis wrote:
Hi all. I have developed some code allowing a thread group of 500 threads to run. However, as soon as the 381st thread is spawned, I get the following runtime error :
I think you want to take a look at _POSIX_THREAD_THREADS_MAX or PTHREADS_THREADS_MAX. One of the two (I forget which) dictates what you are guaranteed to be able to do.
Sohail
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Apostolos Apostolidis wrote:
I have checked both variables and set them way beyond 400. Nothing changed and I keep getting the same error.
At the risk of sounding foolish -- I do not know these variables -- I wonder whether their function is merely to report the capability of the host system, rather than to allow overriding it?
Note that this also happens if I spawn POSIX threads from a C program.
That's actually good news. Since it's not a Boost-specific issue, you can ask in a list where you're more likely to find thread gurus.
On 23/10/2007, *Sohail Somani*
mailto:s.somani@fincad.com> wrote: On Mon, 2007-22-10 at 09:51 +0100, Apostolos Apostolidis wrote: > Hi all. I have developed some code allowing a thread group of 500 > threads to run. > However, as soon as the 381st thread is spawned, I get the following > runtime error : >
I think you want to take a look at _POSIX_THREAD_THREADS_MAX or PTHREADS_THREADS_MAX. One of the two (I forget which) dictates what you are guaranteed to be able to do.

On Wed, 2007-24-10 at 11:56 -0400, Nat Goodspeed wrote:
I have checked both variables and set them way beyond 400. Nothing changed and I keep getting the same error.
At the risk of sounding foolish -- I do not know these variables -- I wonder whether their function is merely to report the capability of the host system, rather than to allow overriding it?
Yes, you are right. I suggest the OP read the man page (is there one?) for these variables. Sohail
participants (5)
-
Apostolos Apostolidis
-
Nat Goodspeed
-
Sohail Somani
-
Tim Day
-
Yaser Zhian