Re: [Boost-users] [Thread] Beginner question regarding thread
Hi, So if I understand your suggestion, I should create a container holding rays yet to be processed and then run a while loop until this queue/container is empty. Within that loop, I create a smaller number of threads by popping rays from the queue and let the threads run until they finish before going on to the next iteration. Did I understand you correctly ? It's a good idea and, from the little I know on the subject, it sounds a lot like a "thread pool". Guess I misunderstood the concept behind thread_group since I thought it was doing that already ... There's even a hardware_concurrency() function to determine the number of cores/cpus which I thought was intended to control how many threads were to be running at once. Regards, Michel Lestrade Crosslight Software
------------------------------
Message: 7 Date: Sun, 24 Aug 2008 21:56:49 -0400 From: "Scott McMurray"
Subject: Re: [Boost-users] [Thread] Beginner question regarding thread groups To: boost-users@lists.boost.org Message-ID: Content-Type: text/plain; charset=UTF-8 On Sun, Aug 24, 2008 at 18:58, Michel Lestrade
wrote: I am considering rewriting part of our ray tracing code to use the Boost thread library. As many of you might know, ray tracing is a task that lends itself well to a parallel approach since each ray is independent from the other. However, the code right now is in Fortran which doesn't support task level parallelism easily ... The thread_group class seems ideally suited to my needs (one thread == one ray) but there is at least one problem I envision that I would like to solve before embarking on this fairly time-consuming rewrite.
It's worth pointing out that, as I recall, you don't really want more continually-active threads than, say, twice your number of cores. It would be nice to run one thread per ray, but the switching and OS management overhead will kill you if you attempt to run thousands of threads at once, which I assume you'd need with 1:1. (Erlang, for example, which is conceptually based on many, many communicating processes, uses its own implementation, rather than OS threads of processes.)
I think you'd be much better off creating a "ray queue" of some sort, then using a thread_group with cores+1 threads that all continually poll the ray queue for which one to work on.
So that's not exactly an answer to the question you asked, but I think that's the usual wisdom with regards to threads.
HTH, ~ Scott
------------------------------
On Sun, Aug 24, 2008 at 23:48, Michel Lestrade
Hi,
So if I understand your suggestion, I should create a container holding rays yet to be processed and then run a while loop until this queue/container is empty. Within that loop, I create a smaller number of threads by popping rays from the queue and let the threads run until they finish before going on to the next iteration.
Actually, the while loop lives inside the threads, since creating and destroying threads is relatively expensive, and you don't need to.
Did I understand you correctly ? It's a good idea and, from the little I know on the subject, it sounds a lot like a "thread pool".
Yes, it's exactly a thread pool.
Guess I misunderstood the concept behind thread_group since I thought it was doing that already ... There's even a hardware_concurrency() function to determine the number of cores/cpus which I thought was intended to control how many threads were to be running at once.
thread_group is just a container for threads; It doesn't have to be a thread pool (since the threads can run different functions). That said, it is obviously convenient for creating thread pools. I think what you want goes something like this: mutex main; int working = 0; queue<ray> q; void ray_processor() { ray r; for (;;) { if (!has_ray) { lock _(main); if (!working && q.empty()) { // nobody has anything to do, exit function break; } if (!q.empty()) { r = q.front(); q.pop(); has_ray = true; ++working; } } if (!has_ray) { // nothing to do, but one of the others does, so wait a bit yield(); continue; } new_rays = process_ray(r); { lock _(main); if (new_rays.empty()) { // no more work, will have to get some from queue --working; has_ray = false; } else { r = new_rays[0]; has_ray = true; for i in [1, new_rays.size()) { q.push(new_rays[i]); } } } } } q.push(initial_ray); thread_group g; for i in [0, hardware_concurrency()+1) { g.add(ray_processor); } g.join_all();
participants (2)
-
Michel Lestrade
-
Scott McMurray