On Aug 14, 2011, at 1:38 PM, Jack Bryan
My program needs to run many (about 10,000+) computing tasks.
Each task's run time is very very short (< 0.1 second or even less).
At each iteration, all these tasks are run in parallel. Some of them need to exchange some data (it is very small) and then go on.
I want to keep all 144 cores as busy as possible so that my program can be done as fast as possible.
So, I want to associate each task with a distinct thread and schedule threads as many as possible. Also, try yo make the workload balance among these cores.
How can I do that from the point of programming ?
A "thread pool" is the usual solution. You might want to try this one (based on boost): http://threadpool.sourceforge.net/ Haven't tried it myself. If you mean the tasks need to exchange data *with each other*, then that's getting more complicated and you might need to look at mpi or something. However if you mean just block for I/O then you might be able to run more threads than cores. Cheers Gordon