That said, why do you even have to keep the boost::thread objects? There aren't many useful things that you could do with a boost::thread; if you run 24/7 you should be able to just spawn the threads "detached":

The threads are doing socket stuff, and I want to terminate them gracefully when the application exits (currently they use a shared event that I signal when its time to terminate). I don't like the idea of my application closing while socket threads are still active - I like things to be as clean as possible.

You can replace your version of tss_hooks.cpp with this one:

Thankyou! Just what I was after. FWIW, I think your try_join proposal is an excellent idea.

Prashant Thakre writes:

On Windows the ~thread() doesnot detach the thread. Hence, the below listed code will throw boost::thread_resource_error().

I tried this code and no exceptions were thrown (boost 1.33.1, Windows, VS2005).

Roland Schwarz wrote:

I agree that it would be nice if such a jobs queue would be available in boost. But this doesn't strictly belong to boost::threads.

see for http://threadpool.sourceforge.net/ for a nice implementation.

This is kind of an abstraction that should be built on top of boost threads.

yeah! i'd like to see this layer provide more syncronisation related functionality like: - multi threaded containers - futures - locking_ptr - ... what else? cheers sascha

Kevin Wheatley wrote:

...

- multi threaded containers

like a generic work item queue that buffers between threads, and should allow for fixed size (blocks when full), growable (but bounded) and 'unlimited' size policies. This would allow for the pipeline model.

yes. in my current project i'm using a queue like this: template<typename T> class queue {...} This has static bounds. Using a policy here would allow to change the queues capacity at runtime.

...

- locking_ptr

Isn't this what shared_ptr already does... I'd like to see a shared ptr without locking :-)

I was thinking of a pointer that locks on dereference (maybe the name wasn't a good choice). It can be used to synchronize access to objects. foo f; synchronize_ptr<foo> p(&f, monitor); p->do_something(); // sync call to method Of course this isn't very efficient most of the time: p->do1(); p->do2(); p->do3(); This would lock/unlock three times although a single lock/unlock around the calls would have been sufficient. In this case it would be nice to have a lock-mechanism similar to boost::weak_ptr. This way we avoid multiple locks/unlocks: synchronize_ptr<T> p(&f); // sync access to f { lock_ptr<T> l = p.lock(); // blocks // we have exclusive access now l->do1(); l->do2(); l->do3(); } // l goes out of scope -> release lock p itself is not dereferencable (as boost::weak_ptr is not). Does this make any sense?

I'd maybe like something layered upon the thread group concept that handles event notifications for the group of threads tied in with the work queue idea. This should work in a generic way so that you would not need to know if it is a single thread or a collection of them available to do the work (Boss-Worker Model)

It may be you wish to share the thread groups by considering the type of work done, e.g. separating I/O threads from CPU bound threads by putting them in different pools. Under the pipeline model you could then have separate queues for each stage which signal the appropriate container of the type of thread that there is work to do.

This is an off the top of my head formulation which is never a good idea in parallel programming.... so may need refinement :-)

Interesting. Kind of dispatcher delegating queued work items to different worker pools. This should be easy to implement once there is a solid thread_pool implementation available. Sascha

Roland Schwarz writes:

What do you expect? You cant simply create threads beyond the system limits, which is roughly 2000 on my box. I tried your code with a really short lived thread: by using function void f() {}. This runs "forever". When running in a debugger, then setting a breakpoint, you will see no threads beyond the main running.

On Solaris 10, this example works just fine. I am just trying to say that behaviour on Windows is different from what one would observe on systems with pthreads, since storage for the *thread* can be reclaimed when that thread terminates(pthread_detach). I guess one has to necessarily call thread.join() on Windows.

BTW.: There might be a misconception about the usage of threads. One normally wouldn't like to pay the cost of starting up a thread for short lived tasks.

Of course, thread_pool should be used since thread creation is not all that cheap.

I agree that it would be nice if such a jobs queue would be available in boost. But this doesn't strictly belong to boost::threads. This is kind of an abstraction that should be built on top of boost threads.

Roland

-- regards, Prashant

On Oct 11, 2006, at 5:19 PM, Peter Dimov wrote:

...

1. I have an application that creates lots of relatively short-lived threads. When a thread is created I store the handle in an array, and at regular intervals, I remove completed threads from the array by checking the thread state via ::GetExitCodeThread. The application runs 24/7 so I don't want an array containing thousands of "dead" handles.

I thought I'd replace this code with a boost::thread_group, but I cannot see any way to remove completed threads from the internal list. This means the std::list used by boost::thread_group would keep on increasing in size until the application exits, which I want to avoid. If I decide to roll my own thread group code, how can I check if a boost::thread is still active or not?

You can't.

Interesting timing; I submitted a proposal for a threading API to the C++ committee:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2090.html

that lets you do that. You can keep a vector< thread::handle > and periodically sweep it using thread::try_join as a predicate. I've been having troubles convincing some committee members of its utility, though.

For the record, I'm the committee member Peter speaks of here. And yes, Peter has been having trouble convincing me that a copyable std::thread is in our best interest for C++0X. However I'm fully convinced of the utility of std::vectorstd::thread. In C++0X this should require only movability, not copyability. Containers of threads should be extremely useful, and I'm aware that the lack of this functionality for boost::thread is frustrating. C++0X should elegantly lift this restriction, allowing vector<thread>. I know this doesn't help you today. But my job is to plan for C++0X, not boost, sorry. The other functionality you request, thread::try_join(), is not supported by myself. However I am supporting higher level standard- supplied objects that do support try_join() (with a different spelling, but I'm not concerned about the spelling for now). For example: std::vector v; ... v.push_back(std::launch_in_thread(some_functional)); ... if (v.back().ready()) // try_join // join with it else // do something else The cost of std::future<void> is approximately the same cost as Peter's copyable thread with try_join functionality, but significantly more expensive than today's boost::thread (or my proposed std::thread, approximately equivalent to boost::thread). When it is obtained from std::launch_in_thread, there is a one to one mapping of it to an OS thread and it has "try_join" functionality. Reference: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2094.html Therein, you'll also find a std::thread_pool which may be even more applicable to your design constraints: std::vector v; std::thread_pool launch_in_pool; ... v.push_back(launch_in_pool(some_functional)); ... if (v.back().ready()) // try_join // join with it else // do something else In a nutshell, I think the availability of move semantics in C++0X strongly impacts the multithreading API that should be designed. I also believe a layered approach is pragmatics. Clients can pay for only what functionality they need (boost::thread while spartan, is quite useful, and will become even more so when movable). I'm quite interested to hear about use cases such as yours to make sure we are covering these bases, with a minimum of cost, and a maximum of efficiency/performance across as many platforms as we can manage. -Howard