[thread] thread interruption semantics
Hey all, I just finally checked out the subversion version of boost (I was still on CVS, oops!). I noticed that the threading library now supports something called 'interruption', which seems to be at the thread level. While I applaud this effort (see a previous post where I created an interruptable_mutex) it has some major deficiencies. First, its presence in the boost::thread class gives the impression the thread can be interrupted at any time. This would mean behavior something akin to a pthread_kill() call in POSIX - which it does not. Second, the only thing it can interrupt in its current implementation is a condition - but unfortunately it does this by doing a broadcast to that condition which means two things. 1) ALL threads waiting on that condition wake up - which is not desired. 2) All threads waiting on that condition try to acquire the same mutex, meaning the thread you interrupted may have to wait for n other threads to acquire the mutex, check to see if they should have woken up, possibly do something, and then go back into the condition before the interrupted thread finally gets the lock, and the exception is thrown. And all this is assuming the coder wrote his code correctly to check to see if he got a condition notification correctly (ie. doesn't assume he can do something if he gets notified, and tries to do something erroneously). Third, interruption should also apply to locks - which would in part solve the above situation. The problem with this of course is that it is less efficient to use a mutex/condition crafted to work with interruptions at the lock level. If you want to continue to offer interruption (which, by the way, I fully support!) - I believe it should be done one of two ways. 1) Use a signal on POSIX systems and find some equivalent mechanism on other systems. This will then actually mean what the user thinks - thread::interrupt() interrupts the thread, not just breaks out of a condition if it happens to be in one. Or 2) Move interruption to its own series of structures. Like I did, with an interruptable_mutex. The advantage of this is twofold: 1) It can be implemented in a non-platform specific manner, using existing data structures, so you implement the interruptable version once (utilizing boost::mutex and boost::condition) and you don't have to care about the underlying platform. This means it doesn't require any OS specific support. 2) It works exactly as the user thinks. An interruptable_mutex can be interrupted while blocked on something to do with it (meaning when trying to acquire a lock on it, OR when waiting on a condition with it). Once again, I'll post the code to my interruptable_mutex - though of course it is based on the CVS (aka. boost 1.34.1) threading library: http://www.neuromancy.net/fisheye/browse/mantra/trunk/Mantra-I/mantra/utils/... It uses an interruptable_pred as the predicate that signals the interrupt should happen. So you create an interruptable_pred that has a reference to an interruptable_mutex, and then lock the predicate or pass the predicate to a condition (I had to specialize the wait() functions in condition to make this work, but it DOES work). This allows you to have a single lock, but then have each thread have its own predicate, or even multiple threads to use the same predicate (so a single interrupt interrupts multiple threads potentially). Once again, I applaud you for including interruptions into boost::threads, but it really should be complete. I am probably going to try and re-write my interruptable_mutex to work with the new threading architecture (I will have to understand it better, and figure out how to replicate the current functionality), but in the mean time, something for you all to think about. PreZ :)
"Preston A. Elder" <prez@neuromancy.net> writes:
I just finally checked out the subversion version of boost (I was still on CVS, oops!). I noticed that the threading library now supports something called 'interruption', which seems to be at the thread level.
While I applaud this effort (see a previous post where I created an interruptable_mutex) it has some major deficiencies.
First, its presence in the boost::thread class gives the impression the thread can be interrupted at any time. This would mean behavior something akin to a pthread_kill() call in POSIX - which it does not.
Second, the only thing it can interrupt in its current implementation is a condition
The interface is the same as the "cancellation" interface in N2320 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2320.html), except renamed to "interruption". The interruption points are: thread::join, thread::timed_join this_thread::sleep condition_variable::wait, condition_variable::timed_wait condition_variable_any::wait, condition_variable_any::timed_wait this_thread::interruption_point This mirrors the required cancellation points on POSIX platforms.
- but unfortunately it does this by doing a broadcast to that condition which means two things.
Firstly, the broadcast only happens on POSIX platforms --- on Windows, the thread is notified directly.
1) ALL threads waiting on that condition wake up - which is not desired. 2) All threads waiting on that condition try to acquire the same mutex, meaning the thread you interrupted may have to wait for n other threads to acquire the mutex, check to see if they should have woken up, possibly do something, and then go back into the condition before the interrupted thread finally gets the lock, and the exception is thrown. And all this is assuming the coder wrote his code correctly to check to see if he got a condition notification correctly (ie. doesn't assume he can do something if he gets notified, and tries to do something erroneously).
Condition variables are allowed spurious wake-ups, so the user of the condition variable has to allow for this possibility. Yes, it is less than ideal that all the other threads are woken too, for the reasons you outline. At the moment I'm not sure of a better scheme.
Third, interruption should also apply to locks - which would in part solve the above situation. The problem with this of course is that it is less efficient to use a mutex/condition crafted to work with interruptions at the lock level.
Yes, and that's why it doesn't happen. All the interruption points are where the thread is waiting for some event (e.g. a condition to be notified, a thread to finish, or a time to elapse). Locks should only be held for a short period, so blocking for a lock should not be a long wait. I don't agree that it should be an interruption point.
If you want to continue to offer interruption (which, by the way, I fully support!) - I believe it should be done one of two ways.
1) Use a signal on POSIX systems and find some equivalent mechanism on other systems. This will then actually mean what the user thinks - thread::interrupt() interrupts the thread, not just breaks out of a condition if it happens to be in one.
If a thread is waiting on a condition variable, a signal is not guaranteed to abort the wait: "If a signal is delivered to a thread waiting for a condition variable, upon return from the signal handler the thread resumes waiting for the condition variable as if it was not interrupted, or it shall return zero due to spurious wakeup." The only way I know to ensure that the thread being interrupted wakes from the condition variable is to broadcast the condition variable. The alternative is that boost::condition_variable does not actually use a pthread_cond_t in the implementation, and I'm not keen on that idea at all.
2) Move interruption to its own series of structures. Like I did, with an interruptable_mutex. The advantage of this is twofold: 1) It can be implemented in a non-platform specific manner, using existing data structures, so you implement the interruptable version once (utilizing boost::mutex and boost::condition) and you don't have to care about the underlying platform. This means it doesn't require any OS specific support. 2) It works exactly as the user thinks. An interruptable_mutex can be interrupted while blocked on something to do with it (meaning when trying to acquire a lock on it, OR when waiting on a condition with it).
I'm not keen on this approach, as it really does mean that the thread can only be interrupted at the defined points.
Once again, I applaud you for including interruptions into boost::threads, but it really should be complete. I am probably going to try and re-write my interruptable_mutex to work with the new threading architecture (I will have to understand it better, and figure out how to replicate the current functionality), but in the mean time, something for you all to think about.
Thank you for your input. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams wrote:
The interruption points are:
thread::join, thread::timed_join this_thread::sleep condition_variable::wait, condition_variable::timed_wait condition_variable_any::wait, condition_variable_any::timed_wait this_thread::interruption_point
Am I the only one who thinks that not having cancellation points at blocking I/O makes this fairly useless? Perhaps I am! I'm curious to hear about use-cases where Anthony or Preston's proposals are appropriate.
"Preston A. Elder" <prez@neuromancy.net> writes:
Third, interruption should also apply to locks - which would in part solve the above situation. The problem with this of course is that it is less efficient to use a mutex/condition crafted to work with interruptions at the lock level.
Yes, and that's why it doesn't happen. All the interruption points are where the thread is waiting for some event (e.g. a condition to be notified, a thread to finish, or a time to elapse). Locks should only be held for a short period, so blocking for a lock should not be a long wait. I don't agree that it should be an interruption point.
I think I agree with Preston. Yes, many locks are only held for short periods, but that's not the only way they can be used. One possibility would be to make lock acquisition a cancellation point iff that lock acquisition would block. This is straightforward with the futex code that I posted, and would not slow down the fast un-contended case. As I proposed when I presented my mutex benchmark results, it may make sense to offer different types of locks for applications with very low probability of contention and for non-trivial probabilities of contention. One application for cancellation is to cleanly kill off threads or groups of threads when something goes wrong. For example, I may observe no activity and assume that a deadlock has occurred. This would require that lock-acquisition be a cancellation point.
If you want to continue to offer interruption (which, by the way, I fully support!) - I believe it should be done one of two ways.
1) Use a signal on POSIX systems and find some equivalent mechanism on other systems. This will then actually mean what the user thinks - thread::interrupt() interrupts the thread, not just breaks out of a condition if it happens to be in one.
If a thread is waiting on a condition variable, a signal is not guaranteed to abort the wait:
"If a signal is delivered to a thread waiting for a condition variable, upon return from the signal handler the thread resumes waiting for the condition variable as if it was not interrupted, or it shall return zero due to spurious wakeup."
I think that, even if this problem could be avoided, delivering a signal to the right thread is very hard.
The only way I know to ensure that the thread being interrupted wakes from the condition variable is to broadcast the condition variable. The alternative is that boost::condition_variable does not actually use a pthread_cond_t in the implementation, and I'm not keen on that idea at all.
Interesting; I was planning to look at implementing conditions, bypassing the pthreads layer, as I did for my futex code. What have you discovered that has put you off that idea?
2) Move interruption to its own series of structures. Like I did, with an interruptable_mutex. The advantage of this is twofold: 1) It can be implemented in a non-platform specific manner, using existing data structures, so you implement the interruptable version once (utilizing boost::mutex and boost::condition) and you don't have to care about the underlying platform. This means it doesn't require any OS specific support. 2) It works exactly as the user thinks. An interruptable_mutex can be interrupted while blocked on something to do with it (meaning when trying to acquire a lock on it, OR when waiting on a condition with it).
I'm not keen on this approach, as it really does mean that the thread can only be interrupted at the defined points.
It's even further from being able to cancel at blocking I/O.... Can I suggest that we don't entirely forget about 3) use pthread_cancel It has some disadvantages, but it's the only solution that offers cancellation at blocking I/O. Regards, Phil.
On Tue, 06 Nov 2007 12:40:40 +0000, Phil Endecott wrote:
Anthony Williams wrote:
The interruption points are:
thread::join, thread::timed_join this_thread::sleep condition_variable::wait, condition_variable::timed_wait condition_variable_any::wait, condition_variable_any::timed_wait this_thread::interruption_point
Am I the only one who thinks that not having cancellation points at blocking I/O makes this fairly useless? Perhaps I am! I'm curious to hear about use-cases where Anthony or Preston's proposals are appropriate.
I have a use case in my own app for an interruptable lock and condition. Though being able to interrupt a sleep would also be cool. The use case is, I have multiple messages going through my system, they have a pre-determined path. I want to be able to interrupt them if they are blocked, and do things like re-route them, hold them, drop them, or invoke a specific function call on them in the thread that currently owns the message. To do this each message is given a UUID, and (using my own version's idioms) an interruptable_pred. As the message goes through its path, along the way it goes through both conditions and mutexes where it must be interrupted. To facilitate this, all mutexes it will wait on in its path are interruptable_mutexes. The interruptable_pred is updated along the way (set when we are going to lock, unset when we have got the lock). This way, I can do something like this: boost::shared_ptr<message> m = message::find(uuid); m->interrupt_drop(); And the mutex it is waiting on will be interrupted (an exception thrown) and the message will be dropped. Actually, to make things work cleaner, I have actually implemented into my interruptable_pred a boost::function callback. So that if the callback is defined, it will be called, otherwise an exception will be thrown (the idea being that function can throw the exception if it REALLY wants to interrupt). So in my above implementation - on interrupt, it will interrupt the mutex (see my previous email on how that works), call a function, which will change some internal state of the message object and then throw an exception. Thereby enabling me to control my message as it goes through the system, even if it is in condition waits or mutex locks. Even if I have no control over the duration a lock is held (as long as they use an interruptable mutex).
One application for cancellation is to cleanly kill off threads or groups of threads when something goes wrong. For example, I may observe no activity and assume that a deadlock has occurred. This would require that lock-acquisition be a cancellation point.
Yes, this is another purpose I had for this, was breaking a deadlock. I've long wanted a way to do this without pthread_cancel, which is very dirty. And an interruptable mutex would be ideal for this.
I think that, even if this problem could be avoided, delivering a signal to the right thread is very hard. There used to be a pthread_kill() function equivalent to the regular kill () function in posix. I don't see it anymore. Maybe I immagined it ;)
Can I suggest that we don't entirely forget about 3) use pthread_cancel
It has some disadvantages, but it's the only solution that offers cancellation at blocking I/O.
Yah, but pthread_cancel is more or less a kill -9 of the thread. All resources are left dangling in the wind, no destructors are called, and more or less you are in a Bad Situation (tm). Interrupting a lock is a much preferred method. PreZ :)
Phil Endecott wrote:
Anthony Williams wrote:
The interruption points are:
thread::join, thread::timed_join this_thread::sleep condition_variable::wait, condition_variable::timed_wait condition_variable_any::wait, condition_variable_any::timed_wait this_thread::interruption_point
Am I the only one who thinks that not having cancellation points at blocking I/O makes this fairly useless? Perhaps I am! I'm curious to hear about use-cases where Anthony or Preston's proposals are appropriate.
The folks in the C++ committee's ad hoc thread working groups have been discussing this for the last year or so. Several points that came up in those discussions may be worth repeating: * For some types of I/O on some operating systems there is simply no way to interrupt a blocking I/O operation once it has started. Even rewriting the operating system itself wouldn't help for some hardware. Rewriting the operating system isn't practical, anyhow. * Experienced (and respected) users of similar cooperative interrupt facilities report that while far from perfect, these imperfect facilities are still far better and more useful than having no interrupt facilities at all. AFAICR, those comments applied both to cooperative interruption in general and the problem of blocking I/O in particular. Very smart and knowledgeable designers have tried for decades to come up with really good solutions to the cancellation problem in general. They have all failed, regardless of the language involved. This is one tough problem. The only solutions that seem to work at all are the limited cooperative solutions. --Beman
Beman Dawes wrote:
Phil Endecott wrote:
Anthony Williams wrote:
The interruption points are:
thread::join, thread::timed_join this_thread::sleep condition_variable::wait, condition_variable::timed_wait condition_variable_any::wait, condition_variable_any::timed_wait this_thread::interruption_point
Am I the only one who thinks that not having cancellation points at blocking I/O makes this fairly useless? Perhaps I am! I'm curious to hear about use-cases where Anthony or Preston's proposals are appropriate.
The folks in the C++ committee's ad hoc thread working groups have been discussing this for the last year or so. Several points that came up in those discussions may be worth repeating:
* For some types of I/O on some operating systems there is simply no way to interrupt a blocking I/O operation once it has started. Even rewriting the operating system itself wouldn't help for some hardware. Rewriting the operating system isn't practical, anyhow.
A balance is needed between supporting as wide a range of systems as possible and not holding back everyone else. I would be happy for interruption / cancellation to be specified with provision for each platform to indicate which underlying OS operations are interruptible and which are not.
Very smart and knowledgeable designers have tried for decades to come up with really good solutions to the cancellation problem in general. They have all failed, regardless of the language involved. This is one tough problem. The only solutions that seem to work at all are the limited cooperative solutions.
What problems will a C programmer using pthread_cancel() encounter? Are you saying that that solution doesn't work at all? Regards, Phil.
Phil Endecott wrote:
Beman Dawes wrote:
Anthony Williams wrote:
The interruption points are:
thread::join, thread::timed_join this_thread::sleep condition_variable::wait, condition_variable::timed_wait condition_variable_any::wait, condition_variable_any::timed_wait this_thread::interruption_point Am I the only one who thinks that not having cancellation points at blocking I/O makes this fairly useless? Perhaps I am! I'm curious to hear about use-cases where Anthony or Preston's proposals are appropriate. The folks in the C++ committee's ad hoc thread working groups have been discussing this for the last year or so. Several points that came up in
Phil Endecott wrote: those discussions may be worth repeating:
* For some types of I/O on some operating systems there is simply no way to interrupt a blocking I/O operation once it has started. Even rewriting the operating system itself wouldn't help for some hardware. Rewriting the operating system isn't practical, anyhow.
A balance is needed between supporting as wide a range of systems as possible and not holding back everyone else.
Agreed.
I would be happy for interruption / cancellation to be specified with provision for each platform to indicate which underlying OS operations are interruptible and which are not.
I don't know enough to know if that is a workable solution. We clearly aren't ready to standardize anything yet. So Boost can perform a valuable service to the wider community by developing a workable solution, and then putting it into use to gain real world feedback.
Very smart and knowledgeable designers have tried for decades to come up with really good solutions to the cancellation problem in general. They have all failed, regardless of the language involved. This is one tough problem. The only solutions that seem to work at all are the limited cooperative solutions.
What problems will a C programmer using pthread_cancel() encounter?
I don't know for C. For C++, issues like "will destructors run?" and "will some resources attempt to be used after they have already been released?". But you need to ask a POSIX expert for details.
Are you saying that that solution doesn't work at all?
My (limited) understanding is that pthread_cancel() may have correct semantics on some platforms but not others, and some of the POSIX folks are dead set against changing the specification to ensure it supplies useful C++ semantics. It appears, as an example, that Mac OS X would have to make changes that aren't acceptable to Apple as currently envisioned. But the situation is murky. Clashing examples and counter-examples abound, and there appear to be serious non-technical issues. --Beman
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Beman Dawes wrote:
Phil Endecott wrote:
Anthony Williams wrote:
The interruption points are:
thread::join, thread::timed_join this_thread::sleep condition_variable::wait, condition_variable::timed_wait condition_variable_any::wait, condition_variable_any::timed_wait this_thread::interruption_point
Am I the only one who thinks that not having cancellation points at blocking I/O makes this fairly useless? Perhaps I am! I'm curious to hear about use-cases where Anthony or Preston's proposals are appropriate.
The folks in the C++ committee's ad hoc thread working groups have been discussing this for the last year or so. Several points that came up in those discussions may be worth repeating:
* For some types of I/O on some operating systems there is simply no way to interrupt a blocking I/O operation once it has started. Even rewriting the operating system itself wouldn't help for some hardware. Rewriting the operating system isn't practical, anyhow.
A balance is needed between supporting as wide a range of systems as possible and not holding back everyone else.
I would be happy for interruption / cancellation to be specified with provision for each platform to indicate which underlying OS operations are interruptible and which are not.
The problem comes with functions that operate on generic handles (e.g. unix file descriptors) that can reference a number of different underlying IO mechanisms (e.g. sockets, network files, local files, pipes, etc.). The same OS function call is then interruptible or not, depending on what the supplied handle actually refers to. This applies to POSIX cancellation points, and is one reason why there are so many "optional" cancellation points --- not only does the OS not have to offer cancellation in those functions, but if they do then they don't have to guarantee that they are always cancellation points. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
On Tue, 06 Nov 2007 10:21:26 +0000, Anthony Williams wrote:
"Preston A. Elder" <prez@neuromancy.net> writes: The interface is the same as the "cancellation" interface in N2320 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2320.html), except renamed to "interruption".
The interruption points are:
thread::join, thread::timed_join this_thread::sleep condition_variable::wait, condition_variable::timed_wait condition_variable_any::wait, condition_variable_any::timed_wait this_thread::interruption_point I appreciate that, however that does not mean you can't go a step further :)
- but unfortunately it does this by doing a broadcast to that condition which means two things.
Firstly, the broadcast only happens on POSIX platforms --- on Windows, the thread is notified directly. My point was, on any single platform, it broadcasts - which in and of itself is not a bad thing, my interruptable_mutex used a broadcast too, however the lock the condition was waiting on was not the lock the end- user would acquire (it was a private lock).
Condition variables are allowed spurious wake-ups, so the user of the condition variable has to allow for this possibility. Yes, it is less than ideal that all the other threads are woken too, for the reasons you outline. At the moment I'm not sure of a better scheme.
See below.
Third, interruption should also apply to locks - which would in part solve the above situation. The problem with this of course is that it is less efficient to use a mutex/condition crafted to work with interruptions at the lock level.
Yes, and that's why it doesn't happen. All the interruption points are where the thread is waiting for some event (e.g. a condition to be notified, a thread to finish, or a time to elapse). Locks should only be held for a short period, so blocking for a lock should not be a long wait. I don't agree that it should be an interruption point.
I agree, but lets face it, sometimes locks are, by necessity NOT held for short periods. For example when writing to an I/O device that you want to ensure that there is only one entity writing to it at a time. In this case, the writer may hold that lock for significant amounts of time (lets avoid the design issue of moving the I/O to its own thread, lets for arguments sake say this is not possible for some reason). In this case, of course, you could create a condition and get woken up when you are allowed to write, but why not just lock the mutex and wait until you get the lock? Thats all you really want to do. Besides - 1) to even enter the condition you have to acquire the lock so you can release it, and 2) as soon as the condition is signalled, you will wait on that lock anyway.
If a thread is waiting on a condition variable, a signal is not guaranteed to abort the wait:
"If a signal is delivered to a thread waiting for a condition variable, upon return from the signal handler the thread resumes waiting for the condition variable as if it was not interrupted, or it shall return zero due to spurious wakeup."
The only way I know to ensure that the thread being interrupted wakes from the condition variable is to broadcast the condition variable. The alternative is that boost::condition_variable does not actually use a pthread_cond_t in the implementation, and I'm not keen on that idea at all. But if you do exactly what you do inside your check - ie. throw an exception from within the signal, that should break you out of any wait.
2) Move interruption to its own series of structures. Like I did, with an interruptable_mutex. The advantage of this is twofold: 1) It can be implemented in a non-platform specific manner, using existing data structures, so you implement the interruptable version once (utilizing boost::mutex and boost::condition) and you don't have to care about the underlying platform. This means it doesn't require any OS specific support. 2) It works exactly as the user thinks. An interruptable_mutex can be interrupted while blocked on something to do with it (meaning when trying to acquire a lock on it, OR when waiting on a condition with it).
I'm not keen on this approach, as it really does mean that the thread can only be interrupted at the defined points.
Fine, then keep the interrupt in the thread - in fact, it is NICE to be able to interrupt a sleep() and so forth. However, change the way mutex works. Change mutex::lock() to something like: IF (!interruption_enabled) do hard-lock; ELSE interruption_checker check(internal_cond); res = lock.trylock(); WHILE (!res) { internal_cond.wait(internal_mutex); IF (check.interrupted) throw thread_interrupted; res = lock.trylock(); } Then unlock() becomes: lock.unlock(); internal_cond.notify_one(); This is more or less the crux of my interruptable_mutex anyway. The key is that while it still uses a broadcast to interrupt, when the condition wakes up, it will acquire the internal lock (ie. a lock on the mutex's own meta-data, NOT a the lock the user wants to acquire). This gives you an interruption point, but also means that if you weren't the thread to be interrupted, you can go back to waiting for the mutex to be available. Passing these to conditions is also pretty easy, for example, condition::wait() becomes something like: IF (!interruption_enabled) cond.wait(lock); ELSE interruption_checker check(cond); lock.unlock(); lock.internal_lock.lock(); cond.wait(lock.internal_lock); lock.internal_lock.unlock(); if (check.interrupted) throw thread_interrupted; lock.lock(); // uses the above mutex::lock(), so interruptable. This way the condition still has the same interruption semantics as you currently have, however if interruption is enabled, then you do your condition wait on the thread's internal ('meta-data') mutex, not on the 'main' mutex. Either way your condition ends by acquiring the mutex the user requested - and ends with something that will synchronize multiple threads and act as a mutex should. But in addition, you will have created an interruption point on mutex::lock(). This gives you a LOT of power and convenience. I still say that interruption semantics are incomplete without interruption of a mutex waiting to block. The above can be implemented in a platform non-specific manner without much trouble and just use the platform-specific version of mutex. Preston. If you can dream it, it can be done :)
On Tue, 06 Nov 2007 13:22:00 +0000, Preston A. Elder wrote: Anthony, I take it from the fact you let this thread die, you have no intention of implementing mutex interrupt semantics (as stated below, it is in actuality quite easy, and you can disable it using your existing semantics for disabling lock interruption - and does not need to be a separate class). I ask because if you don't, I will have to as I require the ability to be able to interrupt not just a condition, but a mutex that is waiting, but has not yet acquired a lock. In fact, as previously stated, I don't think interruption is effective at all without such a feature (as breaking out of a condition will not matter if the first thing it does after it breaks the condition is try to acquire a contested mutex). Unfortunately, if I have to implement this, I will have to do it as a 'band-aid' implementation in my own namespace, when really it does belong as part of boost::thread. PreZ :)
On Tue, 06 Nov 2007 10:21:26 +0000, Anthony Williams wrote:
"Preston A. Elder" <prez@neuromancy.net> writes: The interface is the same as the "cancellation" interface in N2320 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2320.html), except renamed to "interruption".
The interruption points are:
thread::join, thread::timed_join this_thread::sleep condition_variable::wait, condition_variable::timed_wait condition_variable_any::wait, condition_variable_any::timed_wait this_thread::interruption_point I appreciate that, however that does not mean you can't go a step further :)
- but unfortunately it does this by doing a broadcast to that condition which means two things.
Firstly, the broadcast only happens on POSIX platforms --- on Windows, the thread is notified directly. My point was, on any single platform, it broadcasts - which in and of itself is not a bad thing, my interruptable_mutex used a broadcast too, however the lock the condition was waiting on was not the lock the end- user would acquire (it was a private lock).
Condition variables are allowed spurious wake-ups, so the user of the condition variable has to allow for this possibility. Yes, it is less than ideal that all the other threads are woken too, for the reasons you outline. At the moment I'm not sure of a better scheme.
See below.
Third, interruption should also apply to locks - which would in part solve the above situation. The problem with this of course is that it is less efficient to use a mutex/condition crafted to work with interruptions at the lock level.
Yes, and that's why it doesn't happen. All the interruption points are where the thread is waiting for some event (e.g. a condition to be notified, a thread to finish, or a time to elapse). Locks should only be held for a short period, so blocking for a lock should not be a long wait. I don't agree that it should be an interruption point.
I agree, but lets face it, sometimes locks are, by necessity NOT held for short periods. For example when writing to an I/O device that you want to ensure that there is only one entity writing to it at a time. In this case, the writer may hold that lock for significant amounts of time (lets avoid the design issue of moving the I/O to its own thread, lets for arguments sake say this is not possible for some reason).
In this case, of course, you could create a condition and get woken up when you are allowed to write, but why not just lock the mutex and wait until you get the lock? Thats all you really want to do. Besides - 1) to even enter the condition you have to acquire the lock so you can release it, and 2) as soon as the condition is signalled, you will wait on that lock anyway.
If a thread is waiting on a condition variable, a signal is not guaranteed to abort the wait:
"If a signal is delivered to a thread waiting for a condition variable, upon return from the signal handler the thread resumes waiting for the condition variable as if it was not interrupted, or it shall return zero due to spurious wakeup."
The only way I know to ensure that the thread being interrupted wakes from the condition variable is to broadcast the condition variable. The alternative is that boost::condition_variable does not actually use a pthread_cond_t in the implementation, and I'm not keen on that idea at all. But if you do exactly what you do inside your check - ie. throw an exception from within the signal, that should break you out of any wait.
2) Move interruption to its own series of structures. Like I did, with an interruptable_mutex. The advantage of this is twofold: 1) It can be implemented in a non-platform specific manner, using existing data structures, so you implement the interruptable version once (utilizing boost::mutex and boost::condition) and you don't have to care about the underlying platform. This means it doesn't require any OS specific support. 2) It works exactly as the user thinks. An interruptable_mutex can be interrupted while blocked on something to do with it (meaning when trying to acquire a lock on it, OR when waiting on a condition with it).
I'm not keen on this approach, as it really does mean that the thread can only be interrupted at the defined points.
Fine, then keep the interrupt in the thread - in fact, it is NICE to be able to interrupt a sleep() and so forth. However, change the way mutex works. Change mutex::lock() to something like:
IF (!interruption_enabled) do hard-lock; ELSE interruption_checker check(internal_cond); res = lock.trylock(); WHILE (!res) { internal_cond.wait(internal_mutex); IF (check.interrupted) throw thread_interrupted; res = lock.trylock(); }
Then unlock() becomes: lock.unlock(); internal_cond.notify_one();
This is more or less the crux of my interruptable_mutex anyway. The key is that while it still uses a broadcast to interrupt, when the condition wakes up, it will acquire the internal lock (ie. a lock on the mutex's own meta-data, NOT a the lock the user wants to acquire). This gives you an interruption point, but also means that if you weren't the thread to be interrupted, you can go back to waiting for the mutex to be available.
Passing these to conditions is also pretty easy, for example, condition::wait() becomes something like:
IF (!interruption_enabled) cond.wait(lock); ELSE interruption_checker check(cond); lock.unlock(); lock.internal_lock.lock(); cond.wait(lock.internal_lock); lock.internal_lock.unlock(); if (check.interrupted) throw thread_interrupted; lock.lock(); // uses the above mutex::lock(), so interruptable.
This way the condition still has the same interruption semantics as you currently have, however if interruption is enabled, then you do your condition wait on the thread's internal ('meta-data') mutex, not on the 'main' mutex. Either way your condition ends by acquiring the mutex the user requested - and ends with something that will synchronize multiple threads and act as a mutex should. But in addition, you will have created an interruption point on mutex::lock(). This gives you a LOT of power and convenience.
I still say that interruption semantics are incomplete without interruption of a mutex waiting to block. The above can be implemented in a platform non-specific manner without much trouble and just use the platform-specific version of mutex.
Preston.
If you can dream it, it can be done :)
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
"Preston A. Elder" <prez@neuromancy.net> writes:
On Tue, 06 Nov 2007 13:22:00 +0000, Preston A. Elder wrote: I take it from the fact you let this thread die, you have no intention of implementing mutex interrupt semantics (as stated below, it is in actuality quite easy, and you can disable it using your existing semantics for disabling lock interruption - and does not need to be a separate class).
It is true that I don't plan to add interruption semantics to boost::mutex, but that's not why I haven't replied --- this thread is marked for replying in my newsreader --- it's just that I haven't got round to it replying yet. Whether or not it's part of boost.thread, I think that an interruptible mutex needs to be a separate class: it cannot be implemented as a thin wrapper around pthread_mutex_lock, and many users will expect to be able to get that.
I ask because if you don't, I will have to as I require the ability to be able to interrupt not just a condition, but a mutex that is waiting, but has not yet acquired a lock. In fact, as previously stated, I don't think interruption is effective at all without such a feature (as breaking out of a condition will not matter if the first thing it does after it breaks the condition is try to acquire a contested mutex).
I agree that it could be useful, but I don't have time to work on this right now. If you've got the time and the need, go for it.
Unfortunately, if I have to implement this, I will have to do it as a 'band-aid' implementation in my own namespace, when really it does belong as part of boost::thread.
Maybe it can be added in the future. For now, I'm content with the interruption points and semantics proposed to the C++ Standards committee in N2320. One principle behind the new lock templates is that it should be easy to incorporate new mutex and lock types, and they will still work with the existing facilities (e.g. condition_variable_any). Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
On Nov 13, 2007, at 10:55 AM, Anthony Williams wrote:
One principle behind the new lock templates is that it should be easy to incorporate new mutex and lock types, and they will still work with the existing facilities (e.g. condition_variable_any).
<nod> This is the reason I'm interested in Preston's work. I see it as a test of the N2447 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2447.htm ) interface. I've been perusing interruptable_mutex.h this morning and noting that a lot of the complication is in dealing with the older boost::mutex interface (lock_ops and all that). I'd be very interested in seeing this recast to the newer interface, just for the purpose of testing the new interface. -Howard
Ironically, the new interface makes this whole thing harder. Mainly because of two reasons: 1) inconsistent mechanisms for locking between different thread types, and no standard interface for locking. In other words, in pthreads, boost::mutex has a lock(), unlock(), etc. call. In the windows implementation it does not. This means my new implementation has to work through the proxy interfaces of scoped_lock and such. Which in and of itself makes things more complicated. 2) There is no mechanism to differentiate a lock/unlock when called from a condition, and when called by scoped_lock. This differentiation in the was critical in my implementation of conditions using my interruptable mutex. Namely it allowed me to use special handling when used in conditions to ensure that, inside a condition - I unlock the 'real' mutex, and acquire the 'internal' mutex for passing to the condition, and when the condition exited, I can unlock the 'internal' mutex, and re-lock the 'real' mutex in an interruptable fashion. Basically, the use of the lock_ops class, which I could specialize made implementation much easier. There is a lot more concrete code and inconsistent interfaces (especially at the interface level) for me to be able to do this through anything but a scoped_lock. But anyway, I still think interruption is a dangerous facility when you cannot interrupt a mutex.lock() call waiting on a contested resource. And to be honest, I would think that you should have interruptable_condition if you are going to have interruptable_mutex. The fact one can be interrupted and the other not is just strange a little jarring. PreZ :) On Tue, 13 Nov 2007 11:18:12 -0500, Howard Hinnant wrote:
On Nov 13, 2007, at 10:55 AM, Anthony Williams wrote:
One principle behind the new lock templates is that it should be easy to incorporate new mutex and lock types, and they will still work with the existing facilities (e.g. condition_variable_any).
<nod> This is the reason I'm interested in Preston's work. I see it as a test of the N2447 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2447.htm ) interface. I've been perusing interruptable_mutex.h this morning and noting that a lot of the complication is in dealing with the older boost::mutex interface (lock_ops and all that). I'd be very interested in seeing this recast to the newer interface, just for the purpose of testing the new interface.
-Howard
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Tue, 13 Nov 2007 19:50:04 +0000, Preston A. Elder wrote:
Ironically, the new interface makes this whole thing harder. Mainly because of two reasons:
OK, I take it all back! Here is my new interruptable_mutex, using the new boost::thread: http://www.neuromancy.net/fisheye/browse/mantra/trunk/Mantra-I/mantra/utils/... It turns out it is much simpler to implement ;) Here is my test app: -----------8<--------------------------------------------------------------- #include <boost/thread.hpp> #include <interruptable_mutex.h> using namespace boost; typedef interruptable_mutex test_mutex; // typedef mutex test_mutex; test_mutex m; condition c; bool stopwait; void locking_cond() { std::string tid = lexical_cast<std::string>(this_thread::get_id()); printf("Started thread %s\n", tid.c_str()); try { test_mutex::scoped_lock sl(m); printf("%s: Lock acquired!\n", tid.c_str()); while (!stopwait) c.wait(sl); printf("%s: Exited condition!\n", tid.c_str()); sleep(5); } catch (thread_interrupted &e) { printf("%s: Lock interrupted!\n", tid.c_str()); } printf("Ended thread %s\n", tid.c_str()); } void locking_func() { std::string tid = lexical_cast<std::string>(this_thread::get_id()); printf("Started thread %s\n", tid.c_str()); try { test_mutex::scoped_lock sl(m); printf("%s: Lock acquired!\n", tid.c_str()); sleep(5); } catch (thread_interrupted &e) { printf("%s: Lock interrupted!\n", tid.c_str()); } printf("Ended thread %s\n", tid.c_str()); } int main() { stopwait = false; thread *t1 = new thread(&locking_cond); sleep(1); thread *t2 = new thread(&locking_cond); sleep(1); test_mutex::scoped_lock sl(m); printf("Interrupting thread %s\n", boost::lexical_cast<std::string>(t2->get_id()).c_str()); t2->interrupt(); sleep(5); sl.unlock(); stopwait = true; c.notify_all(); sleep(15); return 0; } -----------8<--------------------------------------------------------------- Simply change the typedef of test_mutex to see the difference between a regular mutex and interruptable_mutex. And change the boost::thread function to change between the lock and the condition test. The nice thing about this is it is a template, so any lock type can be used with this lock. Of course, it has t have a try_lock() and unlock() function. OH, and in case you want to, I give full permission for this to be included inside boost and released under the boost software license. PreZ :)
On Wed, 14 Nov 2007 01:44:46 +0000, Preston A. Elder wrote:
On Tue, 13 Nov 2007 19:50:04 +0000, Preston A. Elder wrote:
Ironically, the new interface makes this whole thing harder. Mainly because of two reasons:
OK, I take it all back!
Here is my new interruptable_mutex, using the new boost::thread: http://www.neuromancy.net/fisheye/browse/mantra/trunk/Mantra-I/mantra/ utils/interruptable_mutex.h?r=428
It turns out it is much simpler to implement ;)
Seems I spoke too soon, again ;) I am having one difficulty - though not with anything already implemented. The way I used interruptable_pred is as such. I have a message that passes through various tasks. Inside each task (which for the sake of argument, is a function), they create an interruptable_pred and associate it with the interruptable_mutex for that task. Then they 'give' the interruptable_pred to the message while they want the message to be interruptable. Once they are no longer interruptable, they removed it from the message. This way, I could, from another thread do message->interrupt() and it would check to see if it had a pred, and if so, call the interrupt function on that pred which would then interrupt the other thread. This then enabled me to also to call a function call instead of throwing an exception (a minor detail, and not important at this point). The new interruption syntax promises to make this even easier - for all I have to do is now store the thread pointer in the message when interruptable, and then when I call message->interrupt() it calls thread->interrupt() and has the same effect as the interruptable_pred. The problem is, the tasks need to be able to give the CORRECT thread pointer to the message. But they can't. Because they did not create the thread, they don't have the thread *, and all I can really do from within a thread is call this_thread::get_id() - which, while useful, does not help in this circumstance. I can probably work around this whole mess (maybe using a thread-specific pointer to store the thread pointer, since I do have control over who DOES spawn the thread, this IS possible, however annoying ;P). But the fact a thread cannot more or less look up its own thread for the purpose of telling another thread "If you want to interrupt me, use THIS handle" makes writing this kind of functionality difficult, and if I did not have control over who creates the threads - impossible. So, is it possible to get this fixed? :) PreZ :)
On Wed, 14 Nov 2007 03:22:29 +0000, Preston A. Elder wrote:
So, is it possible to get this fixed? :)
interruption_requested() tells me that), I should be able to cancel the
.. And while I'm at it, a way to cancel pending interrupts would also be good. If I know an interrupt has been requested (hell, thread- pending interrupt and take action myself. Rather than having to acquire a lock spuriously so I can get interrupted (which clears the pending interrupt) and catch the exception to THEN take action. PreZ :)
"Preston A. Elder" <prez@neuromancy.net> writes:
On Wed, 14 Nov 2007 03:22:29 +0000, Preston A. Elder wrote:
So, is it possible to get this fixed? :)
interruption_requested() tells me that), I should be able to cancel the
.. And while I'm at it, a way to cancel pending interrupts would also be good. If I know an interrupt has been requested (hell, thread- pending interrupt and take action myself. Rather than having to acquire a lock spuriously so I can get interrupted (which clears the pending interrupt) and catch the exception to THEN take action.
It's a little messy, but try { this_thread::interruption_point(); } catch(thread_interrupted const&) {} does the trick. You could easily wrap it in a function. However, your code might be interrupted again straight away, so it's not 100% reliable. Why do you need this? Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
On Wed, 14 Nov 2007 10:51:40 +0000, Anthony Williams wrote:
It's a little messy, but
try { this_thread::interruption_point(); } catch(thread_interrupted const&) {}
does the trick. You could easily wrap it in a function. However, your code might be interrupted again straight away, so it's not 100% reliable.
Why do you need this?
Lets say thread A is running, and not at an interruptable point (lock, condition, etc. and its doing some long-running task that will not hit an interruptable point. At some point during this task, I might check thread->interruption_requested() to see if someone has requested an interrupt to break my out of my current task. If this happens, I want to handle that interruption a specific way. But because I now KNOW that the interruptoin has been requested, and I can take appropriate action, I need to clear the interrupt, because I have now handled it. Which, by the way, may mean ignoring it (which your above code does). But the above code would work, just putting in my own interruption points will work to resolve this I think. PreZ :)
On Wed, 14 Nov 2007 03:22:29 +0000, Preston A. Elder wrote:
So, is it possible to get this fixed? :)
Last email for today (I'm going to bed :P) if (!thread_ptr || *thread_ptr != boost::thread()) return false; The above code does not work anymore. At least, my app stopped working because of the above code. I had to change it to: if (!thread_ptr || thread_ptr->get_id() != boost::this_thread::get_id()) return false; But a lot of code will have been written for the former ;) My tests are on the pthreads implementation. Preston.
"Preston A. Elder" <prez@neuromancy.net> writes:
On Wed, 14 Nov 2007 03:22:29 +0000, Preston A. Elder wrote:
So, is it possible to get this fixed? :)
Last email for today (I'm going to bed :P)
if (!thread_ptr || *thread_ptr != boost::thread()) return false;
The above code does not work anymore. At least, my app stopped working because of the above code. I had to change it to: if (!thread_ptr || thread_ptr->get_id() != boost::this_thread::get_id()) return false;
But a lot of code will have been written for the former ;)
My tests are on the pthreads implementation.
Aha. This is a breaking change. A default-constructed boost::thread used to refer to the current thread. Now it refers to "not any thread", so it always compares not equal to any running thread. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
"Preston A. Elder" <prez@neuromancy.net> writes:
The new interruption syntax promises to make this even easier - for all I have to do is now store the thread pointer in the message when interruptable, and then when I call message->interrupt() it calls thread->interrupt() and has the same effect as the interruptable_pred.
The problem is, the tasks need to be able to give the CORRECT thread pointer to the message. But they can't. Because they did not create the thread, they don't have the thread *, and all I can really do from within a thread is call this_thread::get_id() - which, while useful, does not help in this circumstance.
I can probably work around this whole mess (maybe using a thread-specific pointer to store the thread pointer, since I do have control over who DOES spawn the thread, this IS possible, however annoying ;P). But the fact a thread cannot more or less look up its own thread for the purpose of telling another thread "If you want to interrupt me, use THIS handle" makes writing this kind of functionality difficult, and if I did not have control over who creates the threads - impossible.
So, is it possible to get this fixed? :)
The Windows implementation has thread::self() which returns a thread object referring to the current thread. However, I'm a bit concerned about this because it means there are now two thread objects referring to the same thread, so I haven't added it to the POSIX implementation yet. Another option is to add the concept of a "interruption handle" (maybe as part of thread::id) which can interrupt a thread without allowing for other things like detach or join. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
On Wed, 14 Nov 2007 11:05:20 +0000, Anthony Williams wrote:
Another option is to add the concept of a "interruption handle" (maybe as part of thread::id) which can interrupt a thread without allowing for other things like detach or join.
This is a better option, but the problem is, just as with doing a join, interrupting one's self does not make sense either. In the end, all of these actions are things only another thread would execute on 'your' thread, however the problem is we need a method of telling the other thread "This is my thread, if you want to perform actions on me!". Though the problem of having multiple references to the same thread * is a real one, if two threads did a pthread_join() you would have no end of pain. So I guess this, at least, is safe to have multiple entities of, since the 'interruption context' should be lifetime managed by the thread itself, not by the existance of a thread * (which, after all, you can delete without killing the running thread). But then, operator== and operator!= also should be managed by the life of the thread too. But since these are legacy interfaces, I could live without that. PreZ :)
"Preston A. Elder" <prez@neuromancy.net> writes:
On Tue, 13 Nov 2007 19:50:04 +0000, Preston A. Elder wrote:
Ironically, the new interface makes this whole thing harder. Mainly because of two reasons:
OK, I take it all back!
;-)
Here is my new interruptable_mutex, using the new boost::thread: http://www.neuromancy.net/fisheye/browse/mantra/trunk/Mantra-I/mantra/utils/...
It turns out it is much simpler to implement ;)
Excellent. I think there's a bug in your timed_lock --- it doesn't quit when the timeout expires. Easy to fix --- just check the return value from m_cond.timed_wait. Anyway, your code demonstrates why it needs to be a separate type to boost::mutex --- it uses two mutexes (one internal, one external) and a condition variable in the implementation, which is quite heavyweight.
The nice thing about this is it is a template, so any lock type can be used with this lock. Of course, it has t have a try_lock() and unlock() function.
That's good --- it fits in nicely with the new thread library interfaces. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams wrote:
"Preston A. Elder" <prez@neuromancy.net> writes:
On Tue, 13 Nov 2007 19:50:04 +0000, Preston A. Elder wrote:
Ironically, the new interface makes this whole thing harder. Mainly because of two reasons:
OK, I take it all back!
;-)
Here is my new interruptable_mutex, using the new boost::thread: http://www.neuromancy.net/fisheye/browse/mantra/trunk/Mantra-I/mantra/utils/...
It turns out it is much simpler to implement ;)
Excellent. I think there's a bug in your timed_lock --- it doesn't quit when the timeout expires. Easy to fix --- just check the return value from m_cond.timed_wait.
This isn't the only bug. It uses condition variable totally incorrectly. regards, alexander.
Alexander Terekhov <terekhov@web.de> writes:
Anthony Williams wrote:
"Preston A. Elder" <prez@neuromancy.net> writes:
On Tue, 13 Nov 2007 19:50:04 +0000, Preston A. Elder wrote:
Ironically, the new interface makes this whole thing harder. Mainly because of two reasons:
OK, I take it all back!
;-)
Here is my new interruptable_mutex, using the new boost::thread: http://www.neuromancy.net/fisheye/browse/mantra/trunk/Mantra-I/mantra/utils/...
It turns out it is much simpler to implement ;)
Excellent. I think there's a bug in your timed_lock --- it doesn't quit when the timeout expires. Easy to fix --- just check the return value from m_cond.timed_wait.
This isn't the only bug. It uses condition variable totally incorrectly.
That's an interesting comment. Can you elaborate? Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams wrote:
Alexander Terekhov <terekhov@web.de> writes:
Anthony Williams wrote:
"Preston A. Elder" <prez@neuromancy.net> writes:
On Tue, 13 Nov 2007 19:50:04 +0000, Preston A. Elder wrote:
Ironically, the new interface makes this whole thing harder. Mainly because of two reasons:
OK, I take it all back!
;-)
Here is my new interruptable_mutex, using the new boost::thread: http://www.neuromancy.net/fisheye/browse/mantra/trunk/Mantra-I/mantra/utils/...
It turns out it is much simpler to implement ;)
Excellent. I think there's a bug in your timed_lock --- it doesn't quit when the timeout expires. Easy to fix --- just check the return value from m_cond.timed_wait.
This isn't the only bug. It uses condition variable totally incorrectly.
That's an interesting comment. Can you elaborate?
t0: thread A holds the lock t1: thread B calls lock() and does internal try_lock() twice (both fail) t2: thread A calls unlock() and does notify_all() (nobody's waiting) t3: thread B enters wait() on condvar and waits and waits and waits... albeit interruptably. :-) regards, alexander.
On Wed, 14 Nov 2007 13:23:05 +0100, Alexander Terekhov wrote:
Anthony Williams wrote: t0: thread A holds the lock t1: thread B calls lock() and does internal try_lock() twice (both fail) t2: thread A calls unlock() and does notify_all() (nobody's waiting) t3: thread B enters wait() on condvar and waits and waits and waits...
albeit interruptably. :-)
regards, alexander.
This would be easily fixed by getting it to lock the internal lock on unlock (before the notify_all). Doing so would mean that the notify_all could not happen until B is in the wait. Thanks for noticing, an easy bug to fix though :) PreZ :)
On Wed, 14 Nov 2007 12:27:44 +0000, Preston A. Elder wrote:
This would be easily fixed by getting it to lock the internal lock on unlock (before the notify_all). Doing so would mean that the notify_all could not happen until B is in the wait.
And now I think about it, it should probably be a notify_one - there is no point waking up all threads when only one will be able to acquire the lock after an unlock. But then, I guess, since my class is templated, there might actually be a concept of lock sharing (read locks for example), so maybe I could be unlocking the writer, and then notifying a bunch of readers. Hrrrm .. :) This, of course, is assuming I had a 'view' on the shared_lock that would EITHER call lock_shared or lock() depending on my 'view'. Hrm, another new though, should my lock be able to specify which function to call (try_lock / try_shared_lock / try_lock_upgrade) .. I love stream of conciousness moments ;) PreZ :)
"Preston A. Elder" wrote:
On Wed, 14 Nov 2007 12:27:44 +0000, Preston A. Elder wrote:
This would be easily fixed by getting it to lock the internal lock on unlock (before the notify_all). Doing so would mean that the notify_all could not happen until B is in the wait.
And now I think about it, it should probably be a notify_one - there is no point waking up all threads when only one will be able to acquire the lock after an unlock.
You would then need to be a bit more careful in timed_lock(). if (!try_lock()) { mutex::scoped_lock sl(m_internal); while (!try_lock()) if (!m_cond.timed_wait(sl)) return try_lock(); } Alternatively, on timeout, you can return failure but only after resignaling condvar. (Timeout on condvar is allowed to "steal" concurrent signal.) unlock: m_mutex.unlock(); { mutex::scoped_lock sl(m_internal); } m_cond.notify_one(); Also note that such wrapping doesn't preserve destruction safety of Mutex ala POSIX pthread_mutex_t. To preserve it, m_mutex.unlock() must be done after signaling (or signaling must be done while holding m_internal lock which itself must be destruction safe as pthread_mutex_t). regards, alexander.
On Wed, 14 Nov 2007 17:13:21 +0100, Alexander Terekhov wrote:
You would then need to be a bit more careful in timed_lock().
if (!try_lock()) { mutex::scoped_lock sl(m_internal); while (!try_lock()) if (!m_cond.timed_wait(sl)) return try_lock(); } I am ashamed, your code looks so much prettier than mine *yoink!* :)
So Anthony, back to one of my original questions - will interruptable_mutex be put into boost::threads? I can send you a copy of the .h if you don't want to pull it from my SVN (I have not posted the latest version, using nicer looking code like the above). Though you would probably want to make a _fwd file for the typedef's or simply put the implementation into detail. PreZ :)
Alexander Terekhov <terekhov@web.de> writes:
Anthony Williams wrote:
Alexander Terekhov <terekhov@web.de> writes:
Anthony Williams wrote:
"Preston A. Elder" <prez@neuromancy.net> writes:
On Tue, 13 Nov 2007 19:50:04 +0000, Preston A. Elder wrote:
Ironically, the new interface makes this whole thing harder. Mainly because of two reasons:
OK, I take it all back!
;-)
Here is my new interruptable_mutex, using the new boost::thread: http://www.neuromancy.net/fisheye/browse/mantra/trunk/Mantra-I/mantra/utils/...
It turns out it is much simpler to implement ;)
Excellent. I think there's a bug in your timed_lock --- it doesn't quit when the timeout expires. Easy to fix --- just check the return value from m_cond.timed_wait.
This isn't the only bug. It uses condition variable totally incorrectly.
That's an interesting comment. Can you elaborate?
t0: thread A holds the lock t1: thread B calls lock() and does internal try_lock() twice (both fail) t2: thread A calls unlock() and does notify_all() (nobody's waiting) t3: thread B enters wait() on condvar and waits and waits and waits...
Oh, that problem. I hadn't looked at the code close enough to spot that one, but it's a common issue --- you need to hold the internal lock when you modify the data being checked by the predicate (in this case unlocking the external mutex). Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
On Wed, 14 Nov 2007 12:34:01 +0000, Anthony Williams wrote:
Excellent. I think there's a bug in your timed_lock --- it doesn't quit when the timeout expires. Easy to fix --- just check the return value from m_cond.timed_wait.
t0: thread A holds the lock t1: thread B calls lock() and does internal try_lock() twice (both fail) t2: thread A calls unlock() and does notify_all() (nobody's waiting) t3: thread B enters wait() on condvar and waits and waits and waits...
OK, these are both fixed and checked in to my SVN: http://www.neuromancy.net/fisheye/browse/mantra/trunk/Mantra-I/mantra/utils/... The timed lock issue was silliness on my part. My original implementation using the old threading interface did not have this bug, but I rushed trying to convert it from the old interface to the new, and since I don't use timed_lock that much, did not test it. My bad. PreZ :)
"Preston A. Elder" <prez@neuromancy.net> writes:
Ironically, the new interface makes this whole thing harder. Mainly because of two reasons:
1) inconsistent mechanisms for locking between different thread types, and no standard interface for locking. In other words, in pthreads, boost::mutex has a lock(), unlock(), etc. call. In the windows implementation it does not. This means my new implementation has to work through the proxy interfaces of scoped_lock and such. Which in and of itself makes things more complicated.
The Windows implementation has lock(), unlock() etc as well. They are in the public base class detail::basic_timed_mutex
2) There is no mechanism to differentiate a lock/unlock when called from a condition, and when called by scoped_lock. This differentiation in the was critical in my implementation of conditions using my interruptable mutex. Namely it allowed me to use special handling when used in conditions to ensure that, inside a condition - I unlock the 'real' mutex, and acquire the 'internal' mutex for passing to the condition, and when the condition exited, I can unlock the 'internal' mutex, and re-lock the 'real' mutex in an interruptable fashion.
I still don't see why you need to do that.
Basically, the use of the lock_ops class, which I could specialize made implementation much easier. There is a lot more concrete code and inconsistent interfaces (especially at the interface level) for me to be able to do this through anything but a scoped_lock.
It seems to me that the interfaces are much more consistent with the new implementation.
But anyway, I still think interruption is a dangerous facility when you cannot interrupt a mutex.lock() call waiting on a contested resource.
I agree there are circumstances where that is useful.
And to be honest, I would think that you should have interruptable_condition if you are going to have interruptable_mutex. The fact one can be interrupted and the other not is just strange a little jarring.
I can understand that point of view, but the current boost interface matches POSIX cancellation, so it doesn't seem jarring to me. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams wrote:
One principle behind the new lock templates is that it should be easy to incorporate new mutex and lock types, and they will still work with the existing facilities (e.g. condition_variable_any).
Hi Anthony, Can you clarify what you mean by that please? Are you saying that if I have a new mutex (e.g. my futex implementation) I should be able to use it with the existing condition_variable_any? (Is that what the "_any" means?) If that's true I'm impressed; I thought that it was necessary to have some sort of atomic unlock-A-and-lock-B method to do that. Regards, Phil.
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Anthony Williams wrote:
One principle behind the new lock templates is that it should be easy to incorporate new mutex and lock types, and they will still work with the existing facilities (e.g. condition_variable_any).
Hi Anthony,
Can you clarify what you mean by that please? Are you saying that if I have a new mutex (e.g. my futex implementation) I should be able to use it with the existing condition_variable_any? (Is that what the "_any" means?) If that's true I'm impressed; I thought that it was necessary to have some sort of atomic unlock-A-and-lock-B method to do that.
Yes, that's what I meant, and that's what the _any means (as opposed to condition_variable which only works with unique_lock<mutex> (also known as mutex::scoped_lock)). No, you don't need an atomic unlock-A-and-lock-B method, but it does need an extra internal mutex. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams wrote:
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Anthony Williams wrote:
One principle behind the new lock templates is that it should be easy to incorporate new mutex and lock types, and they will still work with the existing facilities (e.g. condition_variable_any). Hi Anthony,
Can you clarify what you mean by that please? Are you saying that if I have a new mutex (e.g. my futex implementation) I should be able to use it with the existing condition_variable_any? (Is that what the "_any" means?) If that's true I'm impressed; I thought that it was necessary to have some sort of atomic unlock-A-and-lock-B method to do that.
Yes, that's what I meant, and that's what the _any means (as opposed to condition_variable which only works with unique_lock<mutex> (also known as mutex::scoped_lock)).
I'm not totally enthralled by the condition_variable_any name. If someone has a suggestion for a more meaningful name, now is a good time to speak up. --Beman
Beman Dawes wrote:
Anthony Williams wrote:
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Anthony Williams wrote:
One principle behind the new lock templates is that it should be easy to incorporate new mutex and lock types, and they will still work with the existing facilities (e.g. condition_variable_any). Hi Anthony,
Can you clarify what you mean by that please? Are you saying that if I have a new mutex (e.g. my futex implementation) I should be able to use it with the existing condition_variable_any? (Is that what the "_any" means?) If that's true I'm impressed; I thought that it was necessary to have some sort of atomic unlock-A-and-lock-B method to do that. Yes, that's what I meant, and that's what the _any means (as opposed to condition_variable which only works with unique_lock<mutex> (also known as mutex::scoped_lock)).
I'm not totally enthralled by the condition_variable_any name. If someone has a suggestion for a more meaningful name, now is a good time to speak up.
I wonder why not making condition_variable a template with a mutex type in its template parameter. We could specialize it on the boost::mutex type to have the current optimized condition_variable implementation and leave it implemented as condition_variable_any for a general case.
On Nov 14, 2007, at 1:29 PM, Andrey Semashev wrote:
Beman Dawes wrote:
I'm not totally enthralled by the condition_variable_any name. If someone has a suggestion for a more meaningful name, now is a good time to speak up.
I wonder why not making condition_variable a template with a mutex type in its template parameter. We could specialize it on the boost::mutex type to have the current optimized condition_variable implementation and leave it implemented as condition_variable_any for a general case.
The rationale for the current design (with different names) can be found here: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2406.html#condition... There is a bullet that begins with:
If the condition is templated on mutex type, ...
-Howard
Howard Hinnant wrote:
Andrey Semashev wrote:
Beman Dawes wrote:
I'm not totally enthralled by the condition_variable_any name. If someone has a suggestion for a more meaningful name, now is a good time to speak up.
cond_var would better for my carpal tunnels.... I was also happy with 'condition'. (Very long names should perhap be reserved for bad things. The time that it takes me to type reinterpret_cast is normally long enough to make me reconsider whether it's appropriate!)
I wonder why not making condition_variable a template with a mutex type in its template parameter. We could specialize it on the boost::mutex type to have the current optimized condition_variable implementation and leave it implemented as condition_variable_any for a general case.
The rationale for the current design (with different names) can be found here:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2406.html#condition...
Thanks for the link. Quote: "For those platforms which offer a native condition variable (e.g. pthread_cond_t), C++ should offer as thin a wrapper as possible around that OS functionality." I think this is a mistake. What is the motivation? Portability, perhaps? By wrapping the existing functionality, we bound the performance that we can achieve: we'll never be faster than C code. High-performance concurrency primitives will be crucial to getting the most from the next generations of multi-core hardware. We should at least be considering building from the most basic building blocks available. (Consider the shared_ptr implementation, for example, which only falls back to pthread_mutex if it doesn't have a better solution using atomics.) My experience with pthread_mutex is that it's simple to write significantly faster (but less-portable) mutexes, and I think that Boost should be taking that approach, with the wrapped OS functionality offered only as a fallback. I have not yet tried to benchmark pthread_cond, but I would be surprised to find it faster than the condition variable using mutexes that I posted elsewhere in this thread [though that code may well have some horrible flaw in it.....]. Apart from performance, another problem with wrapping existing functionality is that the lowest common denominator of all the constraints from the existing implementations has to be passed on to the user. For example, in the condition variable code that I posted, a mutex is locked from one thread and then unlocked from another. I think this works for all of the mutexes that I've written (spinlock, loop-calling-yield, futex) (though I may be missing something - I'm not an expert in any of this). Because it works with futex, I guess that the Linux pthread_mutex may also work in DEFAULT mode, though it's possible that there is something in glibc that breaks it. However, POSIX says that "if a thread attempts to unlock a mutex that it has not locked ... undefined behavior results." So the wording in N2447 also requires that "the current thread of execution shall own the mutex" before it can call unlock(). Regards, Phil.
On Nov 14, 2007, at 5:14 PM, Phil Endecott wrote:
Howard Hinnant wrote:
Andrey Semashev wrote:
Beman Dawes wrote:
I'm not totally enthralled by the condition_variable_any name. If someone has a suggestion for a more meaningful name, now is a good time to speak up.
cond_var would better for my carpal tunnels.... I was also happy with 'condition'. (Very long names should perhap be reserved for bad things. The time that it takes me to type reinterpret_cast is normally long enough to make me reconsider whether it's appropriate!)
<nod> I'm not terribly fond of any of the names and my favorites (least hated) so far have been cond_var and gen_cond_var. However the committee decided otherwise, and I can live with that.
I wonder why not making condition_variable a template with a mutex type in its template parameter. We could specialize it on the boost::mutex type to have the current optimized condition_variable implementation and leave it implemented as condition_variable_any for a general case.
The rationale for the current design (with different names) can be found here:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2406.html#condition...
Thanks for the link. Quote:
"For those platforms which offer a native condition variable (e.g. pthread_cond_t), C++ should offer as thin a wrapper as possible around that OS functionality."
I think this is a mistake. What is the motivation? Portability, perhaps?
By wrapping the existing functionality, we bound the performance that we can achieve: we'll never be faster than C code. High-performance concurrency primitives will be crucial to getting the most from the next generations of multi-core hardware. We should at least be considering building from the most basic building blocks available. (Consider the shared_ptr implementation, for example, which only falls back to pthread_mutex if it doesn't have a better solution using atomics.)
My experience with pthread_mutex is that it's simple to write significantly faster (but less-portable) mutexes, and I think that Boost should be taking that approach, with the wrapped OS functionality offered only as a fallback.
I have not yet tried to benchmark pthread_cond, but I would be surprised to find it faster than the condition variable using mutexes that I posted elsewhere in this thread [though that code may well have some horrible flaw in it.....].
We are approaching this from two different perspectives. The above link refers to a C++ committee paper. I'm representing the path that C ++ should take in C++0X, and not boost. The wording is intended for performance, both in cpu and memory. The wording is not intended to dictate a particular implementation, but to simply allow for the highest performance implementation on any given platform. The condition variable is by now a tried and true multithread primitive. It predates C++ itself. We should have it. How it is implemented is an implementation detail. If pthread semantics can be implemented more efficiently using atomics (which C++0X will have), then that's great. There is no requirement that pthread_cond_t be at the center of std::condition_variable. On the other hand, this is an OS-level functionality. If the semantics of pthread_cond_t can be implemented more efficiently on some platform than is currently done, I imagine that the OS vendor will likely do it, and call it pthread_cond_t.
Apart from performance, another problem with wrapping existing functionality is that the lowest common denominator of all the constraints from the existing implementations has to be passed on to the user. For example, in the condition variable code that I posted, a mutex is locked from one thread and then unlocked from another. I think this works for all of the mutexes that I've written (spinlock, loop-calling-yield, futex) (though I may be missing something - I'm not an expert in any of this). Because it works with futex, I guess that the Linux pthread_mutex may also work in DEFAULT mode, though it's possible that there is something in glibc that breaks it. However, POSIX says that "if a thread attempts to unlock a mutex that it has not locked ... undefined behavior results." So the wording in N2447 also requires that "the current thread of execution shall own the mutex" before it can call unlock().
The Posix committee has considerable experience and expertise in this area. While I do not trust their advice blindly, I also do not dismiss it lightly (quite the contrary). Transferring mutex ownership among threads is not one of the decisions they have made that I've questioned. I've lacked the motivation. If you have strong motivation for relaxing this restriction, coupled with some insight into the posix decision and why it should be relaxed, I am most interested. -Howard
Hi Howard, Howard Hinnant wrote:
If pthread semantics can be implemented more efficiently using atomics (which C++0X will have), then that's great. There is no requirement that pthread_cond_t be at the center of std::condition_variable.
Thanks for the clarification. You might like to adjust the documentation to say that the implementation is expected to be no slower than any native mutex / condition variable.
On the other hand, this is an OS-level functionality. If the semantics of pthread_cond_t can be implemented more efficiently on some platform than is currently done, I imagine that the OS vendor will likely do it, and call it pthread_cond_t.
The semantics of pthread_mutex&cond_t are more complex than are needed for std::mutex&cond_var. Slightly more efficient pthread functions could be implemented by inlining (maybe some implementations already do this). But the overhead of using a single type for all kinds of mutex (recursive, non-recursive, checking etc), with run-time look up of the kind, is unavoidable. Since the optimal uncontended mutex locking code is just two inline instructions, even adding one additional 'if kind==x' will measurably reduce performance.
Apart from performance, another problem with wrapping existing functionality is that the lowest common denominator of all the constraints from the existing implementations has to be passed on to the user. For example, in the condition variable code that I posted, a mutex is locked from one thread and then unlocked from another. I think this works for all of the mutexes that I've written (spinlock, loop-calling-yield, futex) (though I may be missing something - I'm not an expert in any of this). Because it works with futex, I guess that the Linux pthread_mutex may also work in DEFAULT mode, though it's possible that there is something in glibc that breaks it. However, POSIX says that "if a thread attempts to unlock a mutex that it has not locked ... undefined behavior results." So the wording in N2447 also requires that "the current thread of execution shall own the mutex" before it can call unlock().
The Posix committee has considerable experience and expertise in this area. While I do not trust their advice blindly, I also do not dismiss it lightly (quite the contrary). Transferring mutex ownership among threads is not one of the decisions they have made that I've questioned. I've lacked the motivation. If you have strong motivation for relaxing this restriction, coupled with some insight into the posix decision and why it should be relaxed, I am most interested.
You are not the only one to have argued in this thread that the status-quo has been arrived at as a result of much deliberation by experts whose conclusions we should trust. While I don't exactly disagree with that attitude, I suggest that the level of deference might perhaps be dropped back one notch. The rationale section in the pthread_mutex documentation says nothing about the restriction on locking in one thread and unlocking in another. Maybe some readers can propose an explanation? My guess is that someone, long ago, decided to prohibit behaviour that they couldn't think of a use for, and that this has simply been carried forward. My motivation for relaxation is simply that it's useful (essential?) in the algorithm that I presented for a condition variable. (I'm curious to know if anyone has spotted any fundamental problems with that algorithm, by the way.) Regards, Phil.
From: Phil Endecott
Howard Hinnant wrote:
The Posix committee has considerable experience and expertise in this area. While I do not trust their advice blindly, I also do not dismiss it lightly (quite the contrary). Transferring mutex ownership among threads is not one of the decisions they have made that I've questioned.
You are not the only one to have argued in this thread that the status-quo has been arrived at as a result of much deliberation by experts whose conclusions we should trust.
I don't think that's a fair characterization of Howard's argument. "As a result of much deliberation by experts whose conclusions we have to treat seriously even if we eventually discard them" seems much closer to me.
While I don't exactly disagree with that attitude, I suggest that the level of deference might perhaps be dropped back one notch.
I do not trust their advice blindly and If you have strong motivation for relaxing this restriction, coupled with some insight into the posix decision and why it should be relaxed, I am most interested. seems to be pretty much the requested drop back one notch.
-- Martin Bonner Senior Software Engineer/Team Leader PI SHURLOK LTD Telephone: +44 1223 441434 / 203894 (direct) Fax: +44 1223 203999 Email: martin.bonner@pi-shurlok.com www.pi-shurlok.com disclaimer
On Nov 15, 2007, at 7:37 AM, Phil Endecott wrote:
The semantics of pthread_mutex&cond_t are more complex than are needed for std::mutex&cond_var. Slightly more efficient pthread functions could be implemented by inlining (maybe some implementations already do this). But the overhead of using a single type for all kinds of mutex (recursive, non-recursive, checking etc), with run-time look up of the kind, is unavoidable. Since the optimal uncontended mutex locking code is just two inline instructions, even adding one additional 'if kind==x' will measurably reduce performance.
Your argument above addresses pthread_mutex_t but not pthread_cond_t. I too find it unfortunate that so much functionality got stuffed into pthread_mutex_t. And that is precisely why boost::mutex, and the proposed std::mutex separate out functionality into different types. http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2406.html#mutex_rat...
Like boost, and unlike POSIX, this design separates these functionalities into several distinct types which form a hierarchy of concepts. The rationale for the different types is to keep the most common, and simplest mutex type as small and efficient as possible. Each successive concept adds both functionality, and potentially expense.
That being said, the posix rationale includes the following statement on this issue:
Since most attributes only need to be checked when a thread is going to be blocked, the use of attributes does not slow the (common) mutex-locking case.
Apart from performance, another problem with wrapping existing functionality is that the lowest common denominator of all the constraints from the existing implementations has to be passed on to the user. For example, in the condition variable code that I posted, a mutex is locked from one thread and then unlocked from another. I think this works for all of the mutexes that I've written (spinlock, loop-calling-yield, futex) (though I may be missing something - I'm not an expert in any of this). Because it works with futex, I guess that the Linux pthread_mutex may also work in DEFAULT mode, though it's possible that there is something in glibc that breaks it. However, POSIX says that "if a thread attempts to unlock a mutex that it has not locked ... undefined behavior results." So the wording in N2447 also requires that "the current thread of execution shall own the mutex" before it can call unlock().
The Posix committee has considerable experience and expertise in this area. While I do not trust their advice blindly, I also do not dismiss it lightly (quite the contrary). Transferring mutex ownership among threads is not one of the decisions they have made that I've questioned. I've lacked the motivation. If you have strong motivation for relaxing this restriction, coupled with some insight into the posix decision and why it should be relaxed, I am most interested.
You are not the only one to have argued in this thread that the status-quo has been arrived at as a result of much deliberation by experts whose conclusions we should trust. While I don't exactly disagree with that attitude, I suggest that the level of deference might perhaps be dropped back one notch.
The rationale section in the pthread_mutex documentation says nothing about the restriction on locking in one thread and unlocking in another. Maybe some readers can propose an explanation? My guess is that someone, long ago, decided to prohibit behaviour that they couldn't think of a use for, and that this has simply been carried forward.
My motivation for relaxation is simply that it's useful (essential?) in the algorithm that I presented for a condition variable. (I'm curious to know if anyone has spotted any fundamental problems with that algorithm, by the way.)
David Butenhof (Programming with POSIX Threads) recommends semaphores for use cases where you want to lock in one thread and unlock in another. -Howard
Howard Hinnant wrote:
On Nov 15, 2007, at 7:37 AM, Phil Endecott wrote:
The semantics of pthread_mutex&cond_t are more complex than are needed for std::mutex&cond_var. Slightly more efficient pthread functions could be implemented by inlining (maybe some implementations already do this). But the overhead of using a single type for all kinds of mutex (recursive, non-recursive, checking etc), with run-time look up of the kind, is unavoidable. Since the optimal uncontended mutex locking code is just two inline instructions, even adding one additional 'if kind==x' will measurably reduce performance.
Your argument above addresses pthread_mutex_t but not pthread_cond_t.
pthread_cond_wait takes a pthread_mutex argument, so it inherits any problems it has. I haven't looked at pthread_cond enough to understand any further issues.
I too find it unfortunate that so much functionality got stuffed into pthread_mutex_t. And that is precisely why boost::mutex, and the proposed std::mutex separate out functionality into different types.
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2406.html#mutex_rat...
Like boost, and unlike POSIX, this design separates these functionalities into several distinct types which form a hierarchy of concepts. The rationale for the different types is to keep the most common, and simplest mutex type as small and efficient as possible. Each successive concept adds both functionality, and potentially expense.
Absolutely.
That being said, the posix rationale includes the following statement on this issue:
Since most attributes only need to be checked when a thread is going to be blocked, the use of attributes does not slow the (common) mutex-locking case.
Yes, I noticed that. But in the common case of locking an uncontended mutex, if it is recursive you need to record your thread ID in the mutex structure. So you either need to check what kind of mutex you are or store the id unconditionally. (Unless I have missed something.)
David Butenhof (Programming with POSIX Threads) recommends semaphores for use cases where you want to lock in one thread and unlock in another.
Yes, that's a good point. I should investigate it further. Regards, Phil.
On 11/15/07, Phil Endecott <spam_from_boost_dev@chezphil.org> wrote:
Howard Hinnant wrote:
On Nov 15, 2007, at 7:37 AM, Phil Endecott wrote:
The semantics of pthread_mutex&cond_t are more complex than are needed for std::mutex&cond_var. Slightly more efficient pthread functions could be implemented by inlining (maybe some implementations already do this). But the overhead of using a single type for all kinds of mutex (recursive, non-recursive, checking etc), with run-time look up of the kind, is unavoidable. Since the optimal uncontended mutex locking code is just two inline instructions, even adding one additional 'if kind==x' will measurably reduce performance.
Your argument above addresses pthread_mutex_t but not pthread_cond_t.
pthread_cond_wait takes a pthread_mutex argument, so it inherits any problems it has. I haven't looked at pthread_cond enough to understand any further issues.
But it *really* is a preformance problem? Even on the fast path you need to perform at least an atomic/ordered operation. In comparison, the cost of an (easy predictable) compare is very small. -- gpd
Anthony Williams wrote:
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Anthony Williams wrote:
One principle behind the new lock templates is that it should be easy to incorporate new mutex and lock types, and they will still work with the existing facilities (e.g. condition_variable_any).
Hi Anthony,
Can you clarify what you mean by that please? Are you saying that if I have a new mutex (e.g. my futex implementation) I should be able to use it with the existing condition_variable_any? (Is that what the "_any" means?) If that's true I'm impressed; I thought that it was necessary to have some sort of atomic unlock-A-and-lock-B method to do that.
Yes, that's what I meant, and that's what the _any means (as opposed to condition_variable which only works with unique_lock<mutex> (also known as mutex::scoped_lock)).
No, you don't need an atomic unlock-A-and-lock-B method, but it does need an extra internal mutex.
Thanks! I've found your source (for the pthreads version) here: http://svn.boost.org/svn/boost/trunk/boost/thread/pthread/condition_variable... So you're using a pthread_mutex and also a pthread_cond inside condition_variable_any. I was hoping to find that the internal mutex was the same type as the external one, and that the condition variable was implemented just using the mutexes. Hmm, actually I might want to use a different type of mutex for the internal and external locks e.g. a spinlock for the external one and a "proper" mutex (e.g. futex) for the internal one, since typically the external lock is probably low-contention while the condition itself blocks. Here's a completely rubbish pseudo-code sketch of the sort of thing I had in mind: template <typename INTERNAL_MUTEX = my_futex> class condition { std::set<INTERNAL_MUTEX> waiters; public: template <typename EXTERNAL_MUTEX> void wait(Lock<EXTERNAL_MUTEX>& l) { // This is full of races. What's the minimum functionality needed // to make this safe? (Atomic lock transfer between mutexes maybe?) INTERNAL_MUTEX w; w.lock(); waiters.insert(w); l.unlock(); w.lock(); l.lock(); waiters.erase(w); } void notify() { *(waiters.begin()).unlock(); // or all of them for notify_all() } }; I note that N2447 and your code have some swap functions for locks. Are these atomic? Also operator=. (I don't think they are.) Regards, Phil.
On Nov 14, 2007, at 10:23 AM, Phil Endecott wrote:
I note that N2447 and your code have some swap functions for locks. Are these atomic? Also operator=. (I don't think they are.)
No. locks are not intended to be referenced by more than one thread. Only the mutex lock, try_lock, timed_lock and unlock functions are intended to be referenced simultaneously by multiple threads. The swap on locks does absolutely nothing to the referenced mutexes. It simply swaps mutex references and ownership status among the locks. Move assignment is similar except that it will call unlock() on any currently referenced mutex if need be prior to the ownership transfer from the source. -Howard
Phil Endecott wrote:
Anthony Williams wrote:
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Anthony Williams wrote:
One principle behind the new lock templates is that it should be easy to incorporate new mutex and lock types, and they will still work with the existing facilities (e.g. condition_variable_any).
Hi Anthony,
Can you clarify what you mean by that please? Are you saying that if I have a new mutex (e.g. my futex implementation) I should be able to use it with the existing condition_variable_any? (Is that what the "_any" means?) If that's true I'm impressed; I thought that it was necessary to have some sort of atomic unlock-A-and-lock-B method to do that.
Yes, that's what I meant, and that's what the _any means (as opposed to condition_variable which only works with unique_lock<mutex> (also known as mutex::scoped_lock)).
No, you don't need an atomic unlock-A-and-lock-B method, but it does need an extra internal mutex.
Thanks! I've found your source (for the pthreads version) here:
http://svn.boost.org/svn/boost/trunk/boost/thread/pthread/condition_variable...
That condition_variable_any wrapper doesn't ensure same destruction safety of condition varaible as POSIX pthread_cond_t. That's not good. regards, alexander.
On Nov 15, 2007, at 7:02 AM, Alexander Terekhov wrote:
http://svn.boost.org/svn/boost/trunk/boost/thread/pthread/condition_variable...
That condition_variable_any wrapper doesn't ensure same destruction safety of condition varaible as POSIX pthread_cond_t. That's not good.
Thanks for this observation Alexander. Do you have a recommendation for fixing it? The best I'm coming up with at the moment is to keep a count of waiters as member data and have ~condition_variable_any() wait until the count drops to zero. -Howard
Howard Hinnant wrote:
On Nov 15, 2007, at 7:02 AM, Alexander Terekhov wrote:
http://svn.boost.org/svn/boost/trunk/boost/thread/pthread/condition_variable...
That condition_variable_any wrapper doesn't ensure same destruction safety of condition varaible as POSIX pthread_cond_t. That's not good.
Thanks for this observation Alexander. Do you have a recommendation for fixing it? The best I'm coming up with at the moment is to keep a count of waiters as member data and have ~condition_variable_any() wait until the count drops to zero.
Or simply use shared_ptr<pthread_mutex_t> for internal_mutex. Peter will like that. :-) BTW, condition_variable_any's (timed_}wait() above doesn't seem to m.lock() in the case of threads cancel (interruption). That doesn't match POSIX (and condition_variable thin wrapper) behavior. regards, alexander.
On Nov 15, 2007, at 10:24 AM, Alexander Terekhov wrote:
Howard Hinnant wrote:
On Nov 15, 2007, at 7:02 AM, Alexander Terekhov wrote:
http://svn.boost.org/svn/boost/trunk/boost/thread/pthread/condition_variable...
That condition_variable_any wrapper doesn't ensure same destruction safety of condition varaible as POSIX pthread_cond_t. That's not good.
Thanks for this observation Alexander. Do you have a recommendation for fixing it? The best I'm coming up with at the moment is to keep a count of waiters as member data and have ~condition_variable_any() wait until the count drops to zero.
Or simply use shared_ptr<pthread_mutex_t> for internal_mutex. Peter will like that. :-)
:-) Thanks Alexander. I've been debugging this beast for years now. Maybe, just maybe, it's debugged now. :-) struct __lock_external { template <class _Lock> void operator()(_Lock* __m) {__m->lock();} }; class condition_variable_any { condition_variable cv_; shared_ptr<mutex> mut_; public: condition_variable_any() : mut_(new mutex) {} // ~condition_variable_any() = default; // condition_variable_any(const condition_variable_any&) = delete; // condition_variable_any& operator=(const condition_variable_any&) = delete; void notify_one(); void notify_all(); template <class Lock> void wait(Lock& lock); template <class Lock, class Predicate> void wait(Lock& lock, Predicate pred); template <class Lock> bool timed_wait(Lock& lock, const system_time& abs_time); template <class Lock, class Predicate> bool timed_wait(Lock& lock, const system_time& abs_time, Predicate pred); template <class Lock, class Duration, class Predicate> bool timed_wait(Lock& lock, const Duration& rel_time, Predicate pred); }; inline void condition_variable_any::notify_one() { lock_guard<mutex> _(*mut_); cv_.notify_one(); } inline void condition_variable_any::notify_all() { lock_guard<mutex> _(*mut_); cv_.notify_all(); } template <class Lock> void condition_variable_any::wait(Lock& lock) { shared_ptr<mutex> mut = mut_; unique_lock<mutex> lk(*mut); lock.unlock(); unique_ptr<Lock, __lock_external> external_guard(&lock); lock_guard<unique_lock<mutex>> internal_guard(lk, adopt_lock); cv_.wait(lk); } // mut_.unlock(), lock.lock() template <class Lock, class Predicate> inline void condition_variable_any::wait(Lock& lock, Predicate pred) { while (!pred()) wait(lock); } template <class Lock> bool condition_variable_any::timed_wait(Lock& lock, const system_time& abs_time) { shared_ptr<mutex> mut = mut_; unique_lock<mutex> lk(*mut); lock.unlock(); unique_ptr<Lock, __lock_external> external_guard(&lock); lock_guard<unique_lock<mutex>> internal_guard(lk, adopt_lock); return cv_.timed_wait(lk, abs_time); } // mut_.unlock(), lock.lock() template <class Lock, class Predicate> inline bool condition_variable_any::timed_wait(Lock& lock, const system_time& abs_time, Predicate pred) { while (!pred()) if (!timed_wait(lock, abs_time)) return pred(); return true; } template <class Lock, class Duration, class Predicate> inline bool condition_variable_any::timed_wait(Lock& lock, const Duration& rel_time, Predicate pred) { return timed_wait(lock, get_system_time() + rel_time, std::move(pred)); }
BTW, condition_variable_any's (timed_}wait() above doesn't seem to m.lock() in the case of threads cancel (interruption). That doesn't match POSIX (and condition_variable thin wrapper) behavior.
<nod> -Howard
Howard Hinnant wrote:
On Nov 15, 2007, at 10:24 AM, Alexander Terekhov wrote:
Howard Hinnant wrote:
On Nov 15, 2007, at 7:02 AM, Alexander Terekhov wrote:
http://svn.boost.org/svn/boost/trunk/boost/thread/pthread/condition_variable...
That condition_variable_any wrapper doesn't ensure same destruction safety of condition varaible as POSIX pthread_cond_t. That's not good.
Thanks for this observation Alexander. Do you have a recommendation for fixing it? The best I'm coming up with at the moment is to keep a count of waiters as member data and have ~condition_variable_any() wait until the count drops to zero.
Or simply use shared_ptr<pthread_mutex_t> for internal_mutex. Peter will like that. :-)
:-)
Thanks Alexander. I've been debugging this beast for years now. Maybe, just maybe, it's debugged now. :-)
struct __lock_external { template <class _Lock> void operator()(_Lock* __m) {__m->lock();} };
class condition_variable_any { condition_variable cv_;
/*
shared_ptr<mutex> mut_;
*/ The next step is to ask Peter to generalize a bit his shared_ptr so that it can use *mut_ itself for refcount sync. special_sync_shared_ptr<mutex> mut_; or some such. (Details omitted for brevity. :-) )
public:
condition_variable_any() : mut_(new mutex) {} // ~condition_variable_any() = default;
Effect on mut_: mut_.reset_with_sync_on(unique_lock<mutex>(*mut_)) so to speak.
// condition_variable_any(const condition_variable_any&) = delete; // condition_variable_any& operator=(const condition_variable_any&) = delete;
void notify_one(); void notify_all(); template <class Lock> void wait(Lock& lock); template <class Lock, class Predicate> void wait(Lock& lock, Predicate pred); template <class Lock> bool timed_wait(Lock& lock, const system_time& abs_time); template <class Lock, class Predicate> bool timed_wait(Lock& lock, const system_time& abs_time, Predicate pred); template <class Lock, class Duration, class Predicate> bool timed_wait(Lock& lock, const Duration& rel_time, Predicate pred); };
inline void condition_variable_any::notify_one() { lock_guard<mutex> _(*mut_); cv_.notify_one(); }
No need to hold the lock while signaling. I'd add {}.
inline void condition_variable_any::notify_all() { lock_guard<mutex> _(*mut_); cv_.notify_all(); }
No need to hold the lock while signaling. I'd add {}.
template <class Lock> void condition_variable_any::wait(Lock& lock) {
/*
shared_ptr<mutex> mut = mut_; unique_lock<mutex> lk(*mut);
*/ special_sync_shared_ptr<mutex>::unique_lock_and_addref lk(mut);
lock.unlock(); unique_ptr<Lock, __lock_external> external_guard(&lock);
/*
lock_guard<unique_lock<mutex>> internal_guard(lk, adopt_lock);
*/ Put here something to adopt "unique_lock_and_addref lk" above.
cv_.wait(lk); } // mut_.unlock(), lock.lock()
// unref, unlock mutex, delete mutex if refcount reached zero, lock.lock() regards, alexander.
"Preston A. Elder" wrote: [...]
IF (!interruption_enabled) cond.wait(lock); ELSE interruption_checker check(cond); lock.unlock(); lock.internal_lock.lock(); cond.wait(lock.internal_lock); lock.internal_lock.unlock(); if (check.interrupted) throw thread_interrupted; lock.lock(); // uses the above mutex::lock(), so interruptable.
This way the condition still has the same interruption semantics as you
This is broken condition variable. It doesn't ensure "atomic" release of a lock and blocking the calling thread ("atomic" with respect to locking that same lock by another thread and then signaling condition variable). regards, alexander.
Alexander Terekhov <terekhov@web.de> writes:
"Preston A. Elder" wrote: [...]
IF (!interruption_enabled) cond.wait(lock); ELSE interruption_checker check(cond); lock.unlock(); lock.internal_lock.lock(); cond.wait(lock.internal_lock); lock.internal_lock.unlock(); if (check.interrupted) throw thread_interrupted; lock.lock(); // uses the above mutex::lock(), so interruptable.
This way the condition still has the same interruption semantics as you
This is broken condition variable. It doesn't ensure "atomic" release of a lock and blocking the calling thread ("atomic" with respect to locking that same lock by another thread and then signaling condition variable).
Yes. You have to lock the internal mutex before you unlock the external one. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
participants (9)
-
Alexander Terekhov -
Andrey Semashev -
Anthony Williams -
Beman Dawes -
Giovanni Piero Deretta -
Howard Hinnant -
Martin Bonner -
Phil Endecott -
Preston A. Elder