In most systems you get a more-or-less FIFO approach and it won't be far out - and its cheap. That or perhaps have a 'task group' facility where you can optionally associate tasks with a group and check the waiting+executing count per group, which might use an atomic counter and an event that fires iff the atomic decrement went to 0.
That's a good idea. That would exhibit different things I guess.
I think you have to allow pools to have extremely bursty usage patterns and high throughput - anything that. locks, sorts or otherwise stalls the queue is Bad News.
I haven't reviewed the threadpool code. The only glance I had at it indicated that yes, there seem to be way too many locks around. I will wait for the tests to point out some strange behavior before dwelling further, because I'm incredibly lazy. :) -- EA