----- Original Message ----- From: "Anthony Williams" <anthony.ajw@gmail.com> To: <boost@lists.boost.org> Sent: Monday, November 03, 2008 3:19 PM Subject: Re: [boost] [threadpool] new version v12
"vicente.botet" <vicente.botet@wanadoo.fr> writes:
----- Original Message ----- From: "Anthony Williams" <anthony.ajw@gmail.com> To: <boost@lists.boost.org> Sent: Monday, November 03, 2008 1:19 PM Subject: Re: [boost] [threadpool] new version v12
"Oliver Kowalke" <k-oli@gmx.de> writes:
Here's the problem: if you take a task from the global queue or the worker queue of another thread and *don't* do thread migration, then you risk delaying a task and potentially deadlocking the pool.
Consider the sequence above: at step 5, task A is blocked, and the only task it is waiting for (B) has been picked up by an idle worker thread. The thread running task A now takes a new task from the global queue (task N). Suppose this is a really long-running task (calculate PI to 3 million digits, defrag a hard disk, etc.). Without thread migration, task A cannot resume until task N is complete.
Obviously, if task N blocks on a future you can then reschedule task A, but if it doesn't then you don't get that opportunity. If task N waits for another event to be triggered from task A (e.g. a notify on a condition variable) then it will never get it because task A is suspended, and so the pool thread will deadlock *even when task A is ready to run*.
Wouldn't this also happen in the recursive execution of tasks (without fibers)?
Yes. I found that the safest thing to do is spawn another thread rather than recursively executing tasks, unless I could execute the task being waited for (in which case you only get deadlock if you would with separate threads too). You need to pay attention to how many threads are running, but it seems to work better.
This is interesting feature to combine with the disable work stealing, the thread_pool will have no more that n running working threads, others can be blocked, and others are frozen. The working thread scheduler will spawn another thread or unfreeze a frozen thread before blocking. These threads could be added to the thread pool only when some thread is blocked and no frozen thread exists, so at the end there are always no more than the initial number of threads running. The working thread scheduler can put itself on the frozen list when there are already the initial number of threads running.
Yes.
At least with fibers you *can* migrate the task to another thread, if your task is able to handle it.
I don't see any major issue to migrate tasks when the blocking function get() calls recursivelly to the working thread scheduler. Is there one?
If the task migrates across threads its ID changes, and thread locals change.
So if the task do not depends on thread specific this is safe. Vicente