Semaphore and barriers for boost-threads?
Hi, I am new to Boost, and like what I see. I hope to be able to use more Boost libraries in the C++ projects that I work on. Would it be a good thing to add semaphore and barrier classes to Boost Threads? For semaphores, having something analogous to the POSIX functions sem_wait() and sem_post() would be nice. For barriers, having something analogous to pthread_barrier_wait() would be nice. It is not very hard to roll your own versions of these primitives using boost::condition and boost::mutex classes, but it would be nice to have them in a library. Any opinions? Thanks. -- Craig Rodrigues http://crodrigues.org rodrigc@crodrigues.org
I argued vociferously for semaphore and lost. Apparently they seem to think that mutex is a simpler function than semaphore (and they consider it "safer"). I have no idea what POSIX calls it, but semaphore is wait and signal (at least according to the man who invented them... well he called them P and V) At Monday 2004-02-16 15:18, you wrote:
Hi,
I am new to Boost, and like what I see. I hope to be able to use more Boost libraries in the C++ projects that I work on.
Would it be a good thing to add semaphore and barrier classes to Boost Threads? For semaphores, having something analogous to the POSIX functions sem_wait() and sem_post() would be nice.
For barriers, having something analogous to pthread_barrier_wait() would be nice.
It is not very hard to roll your own versions of these primitives using boost::condition and boost::mutex classes, but it would be nice to have them in a library.
Any opinions?
Thanks.
-- Craig Rodrigues http://crodrigues.org rodrigc@crodrigues.org _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"
"Craig Rodrigues" <rodrigc@crodrigues.org> wrote in message news:20040216221856.GA61960@crodrigues.org...
Hi,
Would it be a good thing to add semaphore and barrier classes to Boost Threads? For semaphores, having something analogous to the POSIX functions sem_wait() and sem_post() would be nice.
The original designer of the Boost.Thread library and others felt that semaphores were too easily misused, leading to errors (see http://boost.org/libs/thread/doc/rationale.html#events). Others have argued the issue on the various Boost mailing lists at some length more than once; some have been convinced that the "too easily misused" argument has merit, others have not.
For barriers, having something analogous to pthread_barrier_wait() would be nice.
As far as adding a barrier class, however: one has existed in the thread_dev branch in CVS for quite some time; I hope to move it to the main branch very soon. Mike
It is not very hard to roll your own versions of these primitives using boost::condition and boost::mutex classes, but it would be nice to have them in a library.
Any opinions?
Thanks.
-- Craig Rodrigues http://crodrigues.org rodrigc@crodrigues.org
On Mon, Feb 16, 2004 at 08:33:47PM -0500, Michael Glassford wrote:
The original designer of the Boost.Thread library and others felt that semaphores were too easily misused, leading to errors (see http://boost.org/libs/thread/doc/rationale.html#events). Others have argued the issue on the various Boost mailing lists at some length more than once; some have been convinced that the "too easily misused" argument has merit, others have not.
OK, well I don't do much Windows programming, so I don't really understand what event variables are. I'll have to do my homework and read up on them. In terms of explaining what a semaphore is, my frame of reference is the stuff in POSIX: http://www.opengroup.org/onlinepubs/007904975/functions/sem_post.html http://www.opengroup.org/onlinepubs/007904975/functions/sem_wait.html http://www.opengroup.org/onlinepubs/007904975/functions/sem_trywait.html http://www.opengroup.org/onlinepubs/007904975/functions/sem_timedwait.html Basically, a semaphore is initialized with a count. When you call sem_post(), the semaphore count is incremented. When you call sem_wait(), if the semaphore count is 0, then the thread blocks, else the semaphore count is decremented. It is possible to implement semaphore's with mutexes and condition variables. I looked at the source code for FreeBSD 5.2's implementation of semaphores to see how this was done: http://www.freebsd.org/cgi/cvsweb.cgi/~checkout~/src/lib/libpthread/thread/t... I even took a whack at writing one based on Boost (it may be buggy): #include <boost/thread.hpp> #include <boost/thread/mutex.hpp> struct mysem { mysem(unsigned int count): count_(count) {} void post() { boost::mutex::scoped_lock l(sem_mutex_); ++count_; sem_cond_.notify_all(); } void wait() { boost::mutex::scoped_lock l(sem_mutex_); for(;;) { if( count_ == 0 ) sem_cond_.wait(l); else { --count_; break; } } } private: unsigned int count_; boost::mutex sem_mutex_; boost::condition sem_cond_; }; What kinds of things can go wrong if I use this class?
As far as adding a barrier class, however: one has existed in the thread_dev branch in CVS for quite some time; I hope to move it to the main branch very soon.
Cool, thanks! -- Craig Rodrigues http://crodrigues.org rodrigc@crodrigues.org
On Mon, Feb 16, 2004 at 08:33:47PM -0500, Michael Glassford wrote:
The original designer of the Boost.Thread library and others felt
"Craig Rodrigues" <rodrigc@crodrigues.org> wrote in message news:20040217214636.GA69703@crodrigues.org... that
semaphores were too easily misused, leading to errors (see http://boost.org/libs/thread/doc/rationale.html#events). Others have argued the issue on the various Boost mailing lists at some length more than once; some have been convinced that the "too easily misused" argument has merit, others have not.
OK, well I don't do much Windows programming, so I don't really understand what event variables are. I'll have to do my homework and read up on them.
Sorry, I confused the semaphore debate with the event debate. What I said and the reference I gave obviously applies to events, not semaphores.
In terms of explaining what a semaphore is, my frame of reference is the stuff in POSIX:
http://www.opengroup.org/onlinepubs/007904975/functions/sem_post.html
http://www.opengroup.org/onlinepubs/007904975/functions/sem_wait.html
http://www.opengroup.org/onlinepubs/007904975/functions/sem_trywait.html
http://www.opengroup.org/onlinepubs/007904975/functions/sem_timedwait.html
Basically, a semaphore is initialized with a count. When you call sem_post(), the semaphore count is incremented. When you call sem_wait(), if the semaphore count is 0, then the thread blocks, else the semaphore count is decremented.
Yes, I have Butenhof's Programming with POSIX Threads as a reference for this, though I haven't read it all yet.
It is possible to implement semaphore's with mutexes and condition variables.
Possibly this is why they weren't included in Boost.Thread? I'll have to do some research.
I looked at the source code for FreeBSD 5.2's implementation of semaphores to see how this was done:
http://www.freebsd.org/cgi/cvsweb.cgi/~checkout~/src/lib/libpthread/thread/t...
I even took a whack at writing one based on Boost (it may be buggy):
#include <boost/thread.hpp> #include <boost/thread/mutex.hpp>
struct mysem { mysem(unsigned int count): count_(count) {}
void post() { boost::mutex::scoped_lock l(sem_mutex_); ++count_; sem_cond_.notify_all(); }
void wait() { boost::mutex::scoped_lock l(sem_mutex_); for(;;) {
if( count_ == 0 ) sem_cond_.wait(l); else { --count_; break; } } }
private: unsigned int count_; boost::mutex sem_mutex_; boost::condition sem_cond_; };
What kinds of things can go wrong if I use this class?
As far as adding a barrier class, however: one has existed in the thread_dev branch in CVS for quite some time; I hope to move it to
the
main branch very soon.
Cool, thanks!
-- Craig Rodrigues http://crodrigues.org rodrigc@crodrigues.org
Craig Rodrigues wrote:
OK, well I don't do much Windows programming, so I don't really understand what event variables are. I'll have to do my homework and read up on them.
In terms of explaining what a semaphore is, my frame of reference is the stuff in POSIX:
http://www.opengroup.org/onlinepubs/007904975/functions/sem_post.html http://www.opengroup.org/onlinepubs/007904975/functions/sem_wait.html http://www.opengroup.org/onlinepubs/007904975/functions/sem_trywait.html http://www.opengroup.org/onlinepubs/007904975/functions/sem_timedwait.html
Basically, a semaphore is initialized with a count. When you call sem_post(), the semaphore count is incremented. When you call sem_wait(), if the semaphore count is 0, then the thread blocks, else the semaphore count is decremented.
The interesting part of "POSIX on semaphores" is in http://tinyurl.com/2s89 Look the the phrase "Much experience with semaphores shows" and futher paragraphs. In essense, it's says that semaphores are for signal handlers and for inter-process synchronization and discourages their use anywhere else. - Volodya
On Wed, Feb 18, 2004 at 06:15:58PM +0300, Vladimir Prus wrote:
The interesting part of "POSIX on semaphores" is in
Look the the phrase "Much experience with semaphores shows" and futher paragraphs. In essense, it's says that semaphores are for signal handlers and for inter-process synchronization and discourages their use anywhere else.
That's an interesting link, thanks! I don't interpret the text in the same way that you do, but maybe I am not understanding something properly. Further down from "Much experience with semaphores shows", there is this text: "Counting semaphores are well matched to dealing with producer/consumer problems, including those that might exist between threads of different processes, or between a signal handler and a thread. In the former case, there may be little or no memory shared by the processes; in the latter case, one is not communicating between co-equal threads, but between a thread and an interrupt-like entity. It is for these reasons that IEEE Std 1003.1-2001 allows semaphores to be used by threads." This is the use-case that I am interested in. -- Craig Rodrigues http://crodrigues.org rodrigc@crodrigues.org
Craig Rodrigues wrote:
Look the the phrase "Much experience with semaphores shows" and futher paragraphs. In essense, it's says that semaphores are for signal handlers and for inter-process synchronization and discourages their use anywhere else.
That's an interesting link, thanks! I don't interpret the text in the same way that you do, but maybe I am not understanding something properly.
Maybe I don't, either. The standard cannot make definitive statements that "semaphore is bad". Folks on comp.programming.threads newsgroup seem to make such statements in more clear language ;-)
Further down from "Much experience with semaphores shows", there is this text:
"Counting semaphores are well matched to dealing with producer/consumer problems, including those that might exist between threads of different processes, or between a signal handler and a thread. In the former case, there may be little or no memory shared by the processes; in the latter case, one is not communicating between co-equal threads, but between a thread and an interrupt-like entity. It is for these reasons that IEEE Std 1003.1-2001 allows semaphores to be used by threads."
Doesn't it say that semaphores are retained for inter-process communication and for signal handlers?
This is the use-case that I am interested in.
Which one? Signal handler or inter-process communication? - Volodya
At Thursday 2004-02-19 00:08, you wrote:
Craig Rodrigues wrote:
Look the the phrase "Much experience with semaphores shows" and futher paragraphs. In essense, it's says that semaphores are for signal handlers and for inter-process synchronization and discourages their use anywhere else.
That's an interesting link, thanks! I don't interpret the text in the same way that you do, but maybe I am not understanding something properly.
Maybe I don't, either. The standard cannot make definitive statements that "semaphore is bad". Folks on comp.programming.threads newsgroup seem to make such statements in more clear language ;-)
Further down from "Much experience with semaphores shows", there is this text:
"Counting semaphores are well matched to dealing with producer/consumer problems, including those that might exist between threads of different processes, or between a signal handler and a thread. In the former case, there may be little or no memory shared by the processes; in the latter case, one is not communicating between co-equal threads, but between a thread and an interrupt-like entity. It is for these reasons that IEEE Std 1003.1-2001 allows semaphores to be used by threads."
Doesn't it say that semaphores are retained for inter-process communication and for signal handlers?
This is the use-case that I am interested in.
Which one? Signal handler or inter-process communication?
does it really matter? semaphores are useful (and can be misused, like a lot of other things in life). Why the resistance to putting them in?
- Volodya
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"
At Thursday 2004-02-19 00:08, you wrote:
Craig Rodrigues wrote:
Look the the phrase "Much experience with semaphores shows" and futher paragraphs. In essense, it's says that semaphores are for signal handlers and for inter-process synchronization and discourages their use anywhere else.
That's an interesting link, thanks! I don't interpret the text in the same way that you do, but maybe I am not understanding something properly.
Maybe I don't, either. The standard cannot make definitive statements that "semaphore is bad". Folks on comp.programming.threads newsgroup seem to make such statements in more clear language ;-)
Further down from "Much experience with semaphores shows", there is this text:
"Counting semaphores are well matched to dealing with
problems, including those that might exist between threads of different processes, or between a signal handler and a thread. In the former case, there may be little or no memory shared by the processes; in the latter case, one is not communicating between co-equal threads, but between a thread and an interrupt-like entity. It is for these reasons that IEEE Std 1003.1-2001 allows semaphores to be used by threads."
Doesn't it say that semaphores are retained for inter-process communication and for signal handlers?
This is the use-case that I am interested in.
Which one? Signal handler or inter-process communication?
does it really matter? semaphores are useful (and can be misused,
"Victor A. Wagner, Jr." <vawjr@rudbek.com> wrote in message news:6.0.3.0.2.20040219140935.0448ef10@mail.rudbek.com... producer/consumer like a
lot of other things in life). Why the resistance to putting them in?
For my part, I don't really have a strong opinion one way or the other yet, but even if I decided I strongly supported adding semaphores to Boost.Thread, I wouldn't have time to do anything about it for quite a while. Are you (or is anyone else) willing to submit a design that minimizes the possibilities of misuse, since that appears to be the main concern? And, if Boosters like the design, are you (or is anyone else) willing to implement it on the same platforms supported by the current Boost.Thread? And write documentation that clearly states the intended use cases and drawbacks of semaphores? And write unit tests, examples, etc.? Perhaps one way to limit the possibilities for misuse would be to create a class or classes that, instead of directly exposing semaphore functionality, somehow supports the main use cases for semaphores? I don't really have a clear idea what this would mean--I'm just thinking out loud. Any ideas? Mike
- Volodya
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"
At Thursday 2004-02-19 15:02, you wrote:
Which one? Signal handler or inter-process communication?
does it really matter? semaphores are useful (and can be misused,
"Victor A. Wagner, Jr." <vawjr@rudbek.com> wrote in message news:6.0.3.0.2.20040219140935.0448ef10@mail.rudbek.com... [deleted] like a
lot of other things in life). Why the resistance to putting them in?
For my part, I don't really have a strong opinion one way or the other yet, but even if I decided I strongly supported adding semaphores to Boost.Thread, I wouldn't have time to do anything about it for quite a while. Are you (or is anyone else) willing to submit a design that minimizes the possibilities of misuse, since that appears to be the main concern?
I'm not sure what "misuse" has everyone concerned.
And, if Boosters like the design, are you (or is anyone else) willing to implement it on the same platforms supported by the current Boost.Thread? And write documentation that clearly states the intended use cases and drawbacks of semaphores?
given that I don't think there are any drawbacks, and the limitations are obvious to anyone who reads what they do, I'll have difficulty doing this. Semaphores are _very_ low level components and often used to construct other interlocks.
And write unit tests, examples, etc.?
Perhaps one way to limit the possibilities for misuse would be to create a class or classes that, instead of directly exposing semaphore functionality, somehow supports the main use cases for semaphores?
We used them _mostly_ encapsulated, but our theory was "If it's useful for us, maybe it's useful for our customers also". We were selling embedded mini-computers and a small RTX. Although I wasn't the architect of the RTX, it became my sole responsibility shortly after version 1.0 was delivered to us. The next two generations of real-time kernel/OSs also used semaphore as the basis for all of the interlocks....and we made them available to our customers also.
I don't really have a clear idea what this would mean--I'm just thinking out loud.
Any ideas?
not any more, really. I would like to note that we had to write all this stuff in languages that were NOT typesafe (ASM, BCPL, C) and if someone "told us" that something was a semaphore, we blindly did the "semaphore stuff" on the block of memory referenced. I don't recall anyone ever complaining that it wasn't "safe". Of course, any program could have cleared (filled with 0s) all of memory back then, so any safety we could have added would have been somewhat illusory. I really believe the same still holds, and would appreciate anyone who thinks that semaphores are "unsafe" to give me a real world example of the possible harm.
Mike
- Volodya
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"
At Wednesday 2004-02-18 08:15, you wrote:
Craig Rodrigues wrote:
OK, well I don't do much Windows programming, so I don't really understand what event variables are. I'll have to do my homework and read up on them.
In terms of explaining what a semaphore is, my frame of reference is the stuff in POSIX:
http://www.opengroup.org/onlinepubs/007904975/functions/sem_post.html http://www.opengroup.org/onlinepubs/007904975/functions/sem_wait.html http://www.opengroup.org/onlinepubs/007904975/functions/sem_trywait.html http://www.opengroup.org/onlinepubs/007904975/functions/sem_timedwait.html
Basically, a semaphore is initialized with a count. When you call sem_post(), the semaphore count is incremented. When you call sem_wait(), if the semaphore count is 0, then the thread blocks, else the semaphore count is decremented.
The interesting part of "POSIX on semaphores" is in
Look the the phrase "Much experience with semaphores shows" and futher paragraphs. In essense, it's says that semaphores are for signal handlers and for inter-process synchronization and discourages their use anywhere else.
fine....does this mean we can get semaphores now? or is it still considered "too dangerous" to let people who've been aware of the limitations since before boost existed actually USE them?
- Volodya
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"
Victor A. Wagner, Jr. wrote:
The interesting part of "POSIX on semaphores" is in
Look the the phrase "Much experience with semaphores shows" and futher paragraphs. In essense, it's says that semaphores are for signal handlers and for inter-process synchronization and discourages their use anywhere else.
fine....does this mean we can get semaphores now?
I don't know. I'm just sharing pieces of information which made *me* suspicious of semaphores.
or is it still considered "too dangerous" to let people who've been aware of the limitations since before boost existed actually USE them?
Don't know again -- I'm not the maintainer. But it's sure thing that if semaphores are available many will try to use them without even reading docs which say that condition variables are safer. - Volodya
participants (5)
-
Craig Rodrigues -
Michael Glassford -
Victor A. Wagner Jr. -
Victor A. Wagner, Jr.
-
Vladimir Prus