Hi, After reading most of N2320, I have a few comments: * thread::id has operator== and operator!= declared twice. Once in namespace scope, and a second time inside class thread. Second time is with operator< and his friends. * Thread cancellation is new, and disable_cancellation/restore_cancellation are even newer. They are new for C++ programmers, and maybe new for *all* programmers (I never heard of a language with them). I'm not sure if it's a good idea to standardize them before we get some real-life experience with thread cancellation. * Time issues. To my eyes, it looks not pretty trying to get threading with time issues standardized before we have std::date_time. Making it a templated type not because we want genericity, but because we don't have the type we want yet, makes it look coerced. I think it's best to drop the time-related stuff, and add it properly together with date_time. Using the timed version of thread::join(), mutex::lock() and condition::wait() are very rare, and I think (hope) they can be implemented externally using native_handle(). * I see that you chose not go accept a mutex in condition's constructor. If that is so, why make the condition class templated, instead of only the wait() functions, as is in Boost.Thread? * If C++0x is going to have lambda support (is it?), then maybe the not-accepting-predicate overloads of condition::wait are no longer needed, and can be removed? I think the major reason why people use that overload is because they are too lazy to write a functor. With core support for lambda, it's a breeze. This will solve the other problem (mentioned in some other thread in this ML) about absolute vs. relative times. * Back then we had a discussion about unique_lock, which was never finished. I claim that there is no need for a state where mutex() != 0 && owns() == false, which means that defer_lock is not needed. There is one example in the document using defer_lock, which I think can be done in another way. That example goes like: Record& Record::operator=(const Record& r) { if (this != &r) { std::unique_lock<std::mutex> this_lock(mut, std::defer_lock); std::unique_lock<std::mutex> that_lock(r.mut, std::defer_lock); std::lock(this_lock, that_lock); // Both source and destination are locked // Safe to assign // ... } return *this; } If we change the first 3 lines of the 'if' to: std::lock(mut, r.mut); std::unique_lock<std::mutex> this_lock(mut, std::accept_ownership); std::unique_lock<std::mutex> that_lock(r.mut, std::accept_ownership); then it works without defer_lock. This allows us to: 1. Make unique_lock's interface simpler. 2. Make sizeof(unique_lock) smaller - no need for bool owns. 3. Make unique_lock more similar to unique_ptr, which makes it more intuitive. 4. Make std::lock simpler in the way that it doesn't need to accept locks, only mutexes. In general it seems you paid a lot of attention to being able to pass a lock instead of a mutex wherever possible. I think it's absolutely not necessary. It's an undue complication. * Other than that - very nice :) Yuval
Yuval Ronen wrote:
Hi, After reading most of N2320, I have a few comments:
...snip several thoughts...
* Time issues. To my eyes, it looks not pretty trying to get threading with time issues standardized before we have std::date_time. Making it a templated type not because we want genericity, but because we don't have the type we want yet, makes it look coerced. I think it's best to drop the time-related stuff, and add it properly together with date_time. Using the timed version of thread::join(), mutex::lock() and condition::wait() are very rare, and I think (hope) they can be implemented externally using native_handle().
A minimal subset of date-time *is* being added. During the Kona meeting we worked on unifying N2411 with N2320 to go into the working paper. You can read more about it at: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2411.html Basically allowing you to write code like: std::this_thread::sleep(std::seconds(1)); Lock l; std::condition::timed_wait(l, std::microseconds(100)); std::condition::timed_wait(l, std::milliseconds(100)); std::condition::timed_wait(l, std::nanoseconds(100)); std::condition::timed_wait(l, std::seconds(1)); etc. The templatization of the time interfaces is at my request. The primary motivation is that 1) users/platform developers need the ability to create and use their own time types (eg: picoseconds), and 2) unlike boost::date_time, there is no "universal time_duration type" in the new proposals. The history on this is that I've been uncomfortable with the 'universal time_duration type' for some time, but going without makes the date-time implementation harder (more template magic) so it makes it less portable to older compilers. And with the backward compatibility issues, I haven't removed time_duration from date-time yet. Anyway, I'm uncomfortable with the time_duration type because it locks you into an underlying representation with a particular size. While a 64 bit integers for these types work 95% of the time, there are date-time users that compile the library using 96 bit internals b/c 64 isn't enough for them. However, the 'recompile the library' solution doesn't really work for the standard. So, a better option is to allow user extensibility and eliminate the universal time duration. That way if they want a 96 bit picosecond type it's easy to create and use. Jeff
Jeff Garland wrote:
Yuval Ronen wrote:
Hi, After reading most of N2320, I have a few comments:
...snip several thoughts...
* Time issues. To my eyes, it looks not pretty trying to get threading with time issues standardized before we have std::date_time. Making it a templated type not because we want genericity, but because we don't have the type we want yet, makes it look coerced. I think it's best to drop the time-related stuff, and add it properly together with date_time. Using the timed version of thread::join(), mutex::lock() and condition::wait() are very rare, and I think (hope) they can be implemented externally using native_handle().
A minimal subset of date-time *is* being added. During the Kona meeting we worked on unifying N2411 with N2320 to go into the working paper. You can read more about it at:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2411.html
Basically allowing you to write code like:
std::this_thread::sleep(std::seconds(1));
Lock l; std::condition::timed_wait(l, std::microseconds(100)); std::condition::timed_wait(l, std::milliseconds(100)); std::condition::timed_wait(l, std::nanoseconds(100)); std::condition::timed_wait(l, std::seconds(1));
etc.
The templatization of the time interfaces is at my request. The primary motivation is that 1) users/platform developers need the ability to create and use their own time types (eg: picoseconds), and 2) unlike boost::date_time, there is no "universal time_duration type" in the new proposals. The history on this is that I've been uncomfortable with the 'universal time_duration type' for some time, but going without makes the date-time implementation harder (more template magic) so it makes it less portable to older compilers. And with the backward compatibility issues, I haven't removed time_duration from date-time yet. Anyway, I'm uncomfortable with the time_duration type because it locks you into an underlying representation with a particular size. While a 64 bit integers for these types work 95% of the time, there are date-time users that compile the library using 96 bit internals b/c 64 isn't enough for them. However, the 'recompile the library' solution doesn't really work for the standard. So, a better option is to allow user extensibility and eliminate the universal time duration. That way if they want a 96 bit picosecond type it's easy to create and use.
After reading that there is no universal time_duration type, I asked myself "then what is the return type of time_point subtraction?" So I looked in N2411, and saw that the answer is "nanoseconds". It also makes sense given the fact the utc_time is defined to have nanoseconds resolution. So my conclusion from it, is that nanoseconds /is/ that universal duration_time type. All the rest are (or can be) simple logic-less wrappers around nanoseconds. Had there wasn't any universal time point type (utc_time), but seconds_utc_time, nanoseconds_utc_time, etc, then you could say that there isn't a universal time, but as it is now, I think there is. Now I also understand better the sentence that appear in N2320 in all timed functions: "ElapsedTime shall be explicitly convertible to nanoseconds". It's because nanoseconds are the basic for all. Now lets say you want to allow a demanding user to write his own picoseconds class. This class needs to be "explicitly convertible to nanoseconds". What does it mean? The only answer I can think of is "it has an explicit conversion operator to nanoseconds" (assuming C++0x have an explicit conversion operator feature). That makes the picoseconds class different from all the other time_duration types, because microseconds isn't explicitly convertible to milliseconds, for example. And there's probably a good reason it isn't convertible - because it looses resolution. So why should picoseconds be convertible to nanoseconds? So it seems to me that threading timed function should accept the nanoseconds class. Any alleged picoseconds class will somehow have to support rounding-conversion to nanoseconds. Don't bother threading functions with that, let the picoseconds class handle it.
Yuval Ronen wrote:
Jeff Garland wrote:
Yuval Ronen wrote:
Hi, After reading most of N2320, I have a few comments: ...snip several thoughts...
After reading that there is no universal time_duration type, I asked myself "then what is the return type of time_point subtraction?" So I looked in N2411, and saw that the answer is "nanoseconds". It also makes sense given the fact the utc_time is defined to have nanoseconds resolution. So my conclusion from it, is that nanoseconds /is/ that universal duration_time type. All the rest are (or can be) simple logic-less wrappers around nanoseconds. Had there wasn't any universal time point type (utc_time), but seconds_utc_time, nanoseconds_utc_time, etc, then you could say that there isn't a universal time, but as it is now, I think there is.
No, there is still no universal time duration. You can send any type that meets the qualifications into a sleep or wait. As for utc_time (now renamed to system_time in the latest drafts) using nanoseconds, that is just an indication that the system_time class has a maximum resolution of nanoseconds.
Now I also understand better the sentence that appear in N2320 in all timed functions: "ElapsedTime shall be explicitly convertible to nanoseconds". It's because nanoseconds are the basic for all. Now lets
I'll check, but that's incorrect and has been removed. They don't need to be convertible to nanoseconds, they need to provide get_count and a ticks_per_second trait to allow conversion to the system time resolution.
say you want to allow a demanding user to write his own picoseconds class. This class needs to be "explicitly convertible to nanoseconds". What does it mean? The only answer I can think of is "it has an explicit conversion operator to nanoseconds" (assuming C++0x have an explicit conversion operator feature). That makes the picoseconds class different from all the other time_duration types, because microseconds isn't explicitly convertible to milliseconds, for example. And there's probably a good reason it isn't convertible - because it looses resolution. So why should picoseconds be convertible to nanoseconds?
It won't be -- as I said, that part of N2320 wasn't quite right. Of course, the conversion problem still exists if you use a higher resolution duration than the system supports. In that case, the system will round up to the nearest supported resolution on the system (eg: nanoseconds --> microseconds).
So it seems to me that threading timed function should accept the nanoseconds class. Any alleged picoseconds class will somehow have to support rounding-conversion to nanoseconds. Don't bother threading functions with that, let the picoseconds class handle it.
Yes, it will, but the time duration type provides all the machinery to support the needed rounding. Frankly rounding will be the majority of the cases if you use nanoseconds because few systems currently have clock support at that resolution. Jeff
Jeff Garland wrote:
Yuval Ronen wrote:
Now I also understand better the sentence that appear in N2320 in all timed functions: "ElapsedTime shall be explicitly convertible to nanoseconds". It's because nanoseconds are the basic for all. Now lets
I'll check, but that's incorrect and has been removed. They don't need to be convertible to nanoseconds, they need to provide get_count and a ticks_per_second trait to allow conversion to the system time resolution.
say you want to allow a demanding user to write his own picoseconds class. This class needs to be "explicitly convertible to nanoseconds". What does it mean? The only answer I can think of is "it has an explicit conversion operator to nanoseconds" (assuming C++0x have an explicit conversion operator feature). That makes the picoseconds class different from all the other time_duration types, because microseconds isn't explicitly convertible to milliseconds, for example. And there's probably a good reason it isn't convertible - because it looses resolution. So why should picoseconds be convertible to nanoseconds?
It won't be -- as I said, that part of N2320 wasn't quite right. Of course, the conversion problem still exists if you use a higher resolution duration than the system supports. In that case, the system will round up to the nearest supported resolution on the system (eg: nanoseconds --> microseconds).
Okay, that makes a lot of difference. N2320 needs some fixing. However...
After reading that there is no universal time_duration type, I asked myself "then what is the return type of time_point subtraction?" So I looked in N2411, and saw that the answer is "nanoseconds". It also makes sense given the fact the utc_time is defined to have nanoseconds resolution. So my conclusion from it, is that nanoseconds /is/ that universal duration_time type. All the rest are (or can be) simple logic-less wrappers around nanoseconds. Had there wasn't any universal time point type (utc_time), but seconds_utc_time, nanoseconds_utc_time, etc, then you could say that there isn't a universal time, but as it is now, I think there is.
No, there is still no universal time duration. You can send any type that meets the qualifications into a sleep or wait. As for utc_time (now renamed to system_time in the latest drafts) using nanoseconds, that is just an indication that the system_time class has a maximum resolution of nanoseconds.
... in general (ignoring threading stuff for now), I see no point in providing genericity in time-duration types without parallel genericity in time-point types. These are too interconnected. The fact that the time-point type is defined to have a fixed (nanoseconds) resolution sterilize almost all advantages from time-duration resolution genericity. If sub-nanosecond resolution for time-duration is necessary, then I guess it's also needed for time-points. And another small question: why make those traits a function, and not compile-time constants?
Jeff Garland:
Now I also understand better the sentence that appear in N2320 in all timed functions: "ElapsedTime shall be explicitly convertible to nanoseconds". It's because nanoseconds are the basic for all. Now lets
I'll check, but that's incorrect and has been removed. They don't need to be convertible to nanoseconds, they need to provide get_count and a ticks_per_second trait to allow conversion to the system time resolution.
This strikes me as overgeneralization (speaking strictly of the thread-related portions of the standard). I don't quite see why struct std::abstime: public timespec { // constructors... }; struct std::reltime: public timespec { // constructors... }; // overloaded operators, a way to obtain the system time... isn't enough. POSIX uses timespec and Windows uses milliseconds.
Yes, it will, but the time duration type provides all the machinery to support the needed rounding. Frankly rounding will be the majority of the cases if you use nanoseconds because few systems currently have clock support at that resolution.
POSIX systems use nanoseconds in the interface. timespec is struct timespec { time_t tv_sec; long tv_nsec; }; or to be precise, its definition is: "The <time.h> header shall declare the structure timespec, which has at least the following members: time_t tv_sec Seconds. long tv_nsec Nanoseconds."
Yuval Ronen wrote:
Hi, After reading most of N2320, I have a few comments:
[snip]
* Thread cancellation is new, and disable_cancellation/restore_cancellation are even newer. They are new for C++ programmers, and maybe new for *all* programmers (I never heard of a language with them). I'm not sure if it's a good idea to standardize them before we get some real-life experience with thread cancellation.
What about POSIX Threads (not a language, but a platform)? I don't know to what extent cancellation is used "out in the wild", but the need for a standardized way to cancel a thread is certainly a reality.
* Time issues. To my eyes, it looks not pretty trying to get threading with time issues standardized before we have std::date_time. Making it a templated type not because we want genericity, but because we don't have the type we want yet, makes it look coerced. I think it's best to drop the time-related stuff, and add it properly together with date_time. Using the timed version of thread::join(), mutex::lock() and condition::wait() are very rare, and I think (hope) they can be implemented externally using native_handle().
I can only speak for myself, but I feel that the timed versions of mutex/condition::wait are definitely used often enough to warrant their inclusion. / Johan
On Oct 28, 2007, at 11:53 AM, Yuval Ronen wrote:
Hi, After reading most of N2320, I have a few comments:
Thanks for taking the time to do this. As was mentioned by Jeff, the current public draft is: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2411.html and this draft is known to be somewhat incomplete and incorrect (it was created at the last meeting site under severe time constraints). A revision to it is underway...
* thread::id has operator== and operator!= declared twice. Once in namespace scope, and a second time inside class thread. Second time is with operator< and his friends.
<nod>
* Thread cancellation is new, and disable_cancellation/restore_cancellation are even newer. They are new for C++ programmers, and maybe new for *all* programmers (I never heard of a language with them). I'm not sure if it's a good idea to standardize them before we get some real-life experience with thread cancellation.
Actually thread cancellation is old. Many (not all) thread API's have some form or another of cancellation. That being said, the compromise reached at the Kona meeting 4 weeks ago was to remove cancellation (or interruption) from the proposal. This removal is reflected in N2411.
* I see that you chose not go accept a mutex in condition's constructor. If that is so, why make the condition class templated, instead of only the wait() functions, as is in Boost.Thread?
There was an interim paper: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2406.html which changed the way cv's were handled from N2320. In Kona this was folded into N2411, and also renamed along the way. The condition's constructor still does not take a mutex. There are now two condition variable classes, and neither is templated. The rationale for this design is in N2406.
* If C++0x is going to have lambda support (is it?), then maybe the not-accepting-predicate overloads of condition::wait are no longer needed, and can be removed? I think the major reason why people use that overload is because they are too lazy to write a functor. With core support for lambda, it's a breeze. This will solve the other problem (mentioned in some other thread in this ML) about absolute vs. relative times.
I would be very hesitant to force one programming style over another. Sometimes, for whatever reasons, one might really need an explicit loop (say to handle more complex flow control than just while (!pred) cv.wait(lk)).
* Back then we had a discussion about unique_lock, which was never finished. I claim that there is no need for a state where mutex() != 0 && owns() == false, which means that defer_lock is not needed. There is one example in the document using defer_lock, which I think can be done in another way. That example goes like:
Record& Record::operator=(const Record& r) { if (this != &r) { std::unique_lock<std::mutex> this_lock(mut, std::defer_lock); std::unique_lock<std::mutex> that_lock(r.mut, std::defer_lock); std::lock(this_lock, that_lock); // Both source and destination are locked // Safe to assign // ... } return *this; }
If we change the first 3 lines of the 'if' to:
std::lock(mut, r.mut); std::unique_lock<std::mutex> this_lock(mut, std::accept_ownership); std::unique_lock<std::mutex> that_lock(r.mut, std::accept_ownership);
then it works without defer_lock.
<nod> However I'm not comfortable with claiming that defer_lock has no use beyond this use case, especially when defer_lock-support costs nothing (see below). This is especially true given that boost::thread has supported the defer_lock functionality (under a different spelling) for six or seven years and this is the first report I've seen that the functionality is sufficiently complicating that it should be removed.
This allows us to: 1. Make unique_lock's interface simpler. 2. Make sizeof(unique_lock) smaller - no need for bool owns.
Even if we remove defer_lock, unique_lock will still need the internal bool to support try and timed locks. This functionality still retains a reference to the mutex even if the try/timed lock fails (so that further action with the mutex can be easily taken).
3. Make unique_lock more similar to unique_ptr, which makes it more intuitive. 4. Make std::lock simpler in the way that it doesn't need to accept locks, only mutexes.
std::lock isn't further complicated by accepting both mutexes and locks. It accepts anything that supports lock(), try_lock() and unlock(). To disallow locks, we would have to make std::lock more complicated by detecting lock template arguments and then actively disabling them. I see no motivation to add this complication.
In general it seems you paid a lot of attention to being able to pass a lock instead of a mutex wherever possible. I think it's absolutely not necessary. It's an undue complication.
Actually I tried to pay attention to what the minimal requirements were on template parameters. It just turned out that some generic code would accept either locks or mutexes.
* Other than that - very nice :)
Thanks. Sorry the paper keeps changing out from under you. There are many people actively working a single document now and such changes are inevitable.
Howard Hinnant wrote:
Actually thread cancellation is old. Many (not all) thread API's have some form or another of cancellation. That being said, the compromise reached at the Kona meeting 4 weeks ago was to remove cancellation (or interruption) from the proposal. This removal is reflected in N2411.
That's unfortunate. I'm a little confused about how completely removing it is a "compromise"! This is something that I need. Does anyone know of an alternative (POSIX-compatible) C++ thread library that offers cancellation? If not I may try to write on. Phil.
Phil Endecott wrote:
That's unfortunate. I'm a little confused about how completely removing it is a "compromise"!
It's a compromise between C, which can't handle C++'s exception-based cancellation, and C++, which would be unhappy with most C-compatible cancellation mechanisms. Not sure if this is the intended meaning, but it is a possible one. Sebastian Redl
On Oct 30, 2007, at 2:03 PM, Phil Endecott wrote:
Howard Hinnant wrote:
Actually thread cancellation is old. Many (not all) thread API's have some form or another of cancellation. That being said, the compromise reached at the Kona meeting 4 weeks ago was to remove cancellation (or interruption) from the proposal. This removal is reflected in N2411.
That's unfortunate. I'm a little confused about how completely removing it is a "compromise"!
This is something that I need. Does anyone know of an alternative (POSIX-compatible) C++ thread library that offers cancellation? If not I may try to write on.
A partial workaround is to religiously use cv::timed_wait, as opposed to cv::wait, so that you can wake up and discover if you're supposed to quit every once in a while. It should be easy to set up a "cancel" flag that a thread can periodically check. For example you could just pass a ref(bool) as one of the arguments in your thread- main function and reference that argument with thread local data. One will still have trouble waking from blocked system calls, but that's life. -Howard
Howard Hinnant wrote:
On Oct 30, 2007, at 2:03 PM, Phil Endecott wrote:
Howard Hinnant wrote:
Actually thread cancellation is old. Many (not all) thread API's have some form or another of cancellation. That being said, the compromise reached at the Kona meeting 4 weeks ago was to remove cancellation (or interruption) from the proposal. This removal is reflected in N2411.
That's unfortunate. I'm a little confused about how completely removing it is a "compromise"!
This is something that I need. Does anyone know of an alternative (POSIX-compatible) C++ thread library that offers cancellation? If not I may try to write one.
A partial workaround is to religiously use cv::timed_wait, as opposed to cv::wait, so that you can wake up and discover if you're supposed to quit every once in a while.
Hi Howard, Maybe you'll recall a previous message of mine where I commented on the power-efficiency implications of frequent wakeups, and the benefits of aligning them. So no, I'm not keen on this approach. I would like to think that a pthreads cleanup function could be written that would have the effect of invoking destructors from the cancellation point up to the thread entry point. Can you explain the rationale for leaving it out? Regards, Phil.
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Howard Hinnant wrote:
On Oct 30, 2007, at 2:03 PM, Phil Endecott wrote:
Howard Hinnant wrote:
Actually thread cancellation is old. Many (not all) thread API's have some form or another of cancellation. That being said, the compromise reached at the Kona meeting 4 weeks ago was to remove cancellation (or interruption) from the proposal. This removal is reflected in N2411.
That's unfortunate. I'm a little confused about how completely removing it is a "compromise"!
This is something that I need. Does anyone know of an alternative (POSIX-compatible) C++ thread library that offers cancellation? If not I may try to write one.
A partial workaround is to religiously use cv::timed_wait, as opposed to cv::wait, so that you can wake up and discover if you're supposed to quit every once in a while.
Hi Howard,
Maybe you'll recall a previous message of mine where I commented on the power-efficiency implications of frequent wakeups, and the benefits of aligning them. So no, I'm not keen on this approach.
I would like to think that a pthreads cleanup function could be written that would have the effect of invoking destructors from the cancellation point up to the thread entry point. Can you explain the rationale for leaving it out?
How do you propose doing that? That would require knowing at what point the thread was cancelled, and walking up the stack, which is therefore implementation-specific. What about interaction with other cleanup handlers --- how would these get interleaved? In C++, the only ways of unwinding the stack are to throw an exception or break out of a block with a return, break or goto statement. Are you suggesting adding "thread cancellation" as another, or are you suggesting it maps to an exception? If it maps to an exception, how do you handle cleanup handlers in C stack frames on an implementation that doesn't allow C++ exceptions in C stack frames? If it doesn't map to an exception, what about functions that do their cleanup in catch(...) clauses? If it does map to an exception, can you catch it? Does it automatically rethrow? These are some of the issues that have caused cancellation to be a sticking point. I think cancellation should be a normal exception, so it can be caught and discarded, and handles stack unwinding in a C++-like fashion, and that's what boost.thread now does. I also think that interaction with pthread_cancel and C stack-frame cleanup handlers is a "nice-to-have", but not required. Boost.Thread does not interoperate with pthread_cancel. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Hi Anthony, Anthony Williams wrote:
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Howard Hinnant wrote:
On Oct 30, 2007, at 2:03 PM, Phil Endecott wrote:
Howard Hinnant wrote:
Actually thread cancellation is old. Many (not all) thread API's have some form or another of cancellation. That being said, the compromise reached at the Kona meeting 4 weeks ago was to remove cancellation (or interruption) from the proposal. This removal is reflected in N2411.
That's unfortunate. I'm a little confused about how completely removing it is a "compromise"!
This is something that I need. Does anyone know of an alternative (POSIX-compatible) C++ thread library that offers cancellation? If not I may try to write one.
[answering your other mail first]
By "POSIX-compatible" do you mean "works with and uses pthread_cancel", or do you mean something else? It's the interaction with pthread_cancel that caused the biggest sticking point, as I understand things.
Sorry, I was being inaccurate. What I should have said was that I'm writing Linux code, so I need a library that will work on Linux. Being implemented on top of the pthreads functions would be one solution. No, I don't need to mix C++ destructors and C pthread_cleanup. Of course a cross-platform solution would be essential for a standard library and very highly desirable for a Boost library, but I need a solution "real soon now"! Hence my enquiry about any other libraries that people might know about.
A partial workaround is to religiously use cv::timed_wait, as opposed to cv::wait, so that you can wake up and discover if you're supposed to quit every once in a while.
Hi Howard,
Maybe you'll recall a previous message of mine where I commented on the power-efficiency implications of frequent wakeups, and the benefits of aligning them. So no, I'm not keen on this approach.
I would like to think that a pthreads cleanup function could be written that would have the effect of invoking destructors from the cancellation point up to the thread entry point. Can you explain the rationale for leaving it out?
How do you propose doing that? That would require knowing at what point the thread was cancelled, and walking up the stack, which is therefore implementation-specific. What about interaction with other cleanup handlers --- how would these get interleaved?
Yes, you need lots of implementation-specific stuff. My guess is that the pthread_cleanup function is in practice executed with the cancelled thread's stack, so you can find the starting point for stack-unwinding just above the current stack pointer. Maybe you can even just "throw Cancelled();" from a pthread_cleanup function! But that's just a guess.
In C++, the only ways of unwinding the stack are to throw an exception or break out of a block with a return, break or goto statement. Are you suggesting adding "thread cancellation" as another, or are you suggesting it maps to an exception?
If it maps to an exception, how do you handle cleanup handlers in C stack frames on an implementation that doesn't allow C++ exceptions in C stack frames?
If it does map to an exception, can you catch it? Does it automatically rethrow?
I don't care about any of this for my current application, though I have always imagined it mapping to an exception.
These are some of the issues that have caused cancellation to be a sticking point. I think cancellation should be a normal exception, so it can be caught and discarded, and handles stack unwinding in a C++-like fashion, and that's what boost.thread now does. I also think that interaction with pthread_cancel and C stack-frame cleanup handlers is a "nice-to-have", but not required.
Yes, I agree with all that. Clearly there are various choices about how cancellation should behave, with various pros and cons, and many questions about the practicalities of implementation on different platforms. Certainly the standards committee have investigated these issues. I would very much like to have some insight into what they discovered during their investigations, leading them to their decision to not offer any sort of cancellation. Regards, Phil.
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Hi Anthony,
Anthony Williams wrote:
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Howard Hinnant wrote:
On Oct 30, 2007, at 2:03 PM, Phil Endecott wrote:
Howard Hinnant wrote:
Actually thread cancellation is old. Many (not all) thread API's have some form or another of cancellation. That being said, the compromise reached at the Kona meeting 4 weeks ago was to remove cancellation (or interruption) from the proposal. This removal is reflected in N2411.
That's unfortunate. I'm a little confused about how completely removing it is a "compromise"!
This is something that I need. Does anyone know of an alternative (POSIX-compatible) C++ thread library that offers cancellation? If not I may try to write one.
[answering your other mail first]
By "POSIX-compatible" do you mean "works with and uses pthread_cancel", or do you mean something else? It's the interaction with pthread_cancel that caused the biggest sticking point, as I understand things.
Sorry, I was being inaccurate. What I should have said was that I'm writing Linux code, so I need a library that will work on Linux. Being implemented on top of the pthreads functions would be one solution. No, I don't need to mix C++ destructors and C pthread_cleanup. Of course a cross-platform solution would be essential for a standard library and very highly desirable for a Boost library, but I need a solution "real soon now"! Hence my enquiry about any other libraries that people might know about.
I added better cancellation to boost thread yesterday. condition wait+timed_wait, thread join+timed_join, sleep and this_thread::cancellation_point are now all cancellation points on pthreads and windows. This uses a thread_cancelled exception, and does not interoperate with pthread_cancel.
Clearly there are various choices about how cancellation should behave, with various pros and cons, and many questions about the practicalities of implementation on different platforms. Certainly the standards committee have investigated these issues. I would very much like to have some insight into what they discovered during their investigations, leading them to their decision to not offer any sort of cancellation.
N2420 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2420.pdf) covers some of the relevant ground --- it's the minutes of the POSIX/C++ liaison committee. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams wrote: [...]
N2420 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2420.pdf) covers some of the relevant ground --- it's the minutes of the POSIX/C++ liaison committee.
"Gnu gcc and Solaris pthreads: The Gnu gcc and Solaris pthreads implementations are two known implementations that attempt to map POSIX pthread cancellation onto C++ exception handling, but both do so at the cost of breaking the exception model (i.e., they no longer conform to ISO C++) because the alternative appears to be that C++ destructors and catch blocks would not be invoked for cancellation which would mean that resources would be leaked." Uhmm. Don't know about (modern) Solaris, but glibc (presumably that is meant by "Gnu gcc pthreads implementation") does invoke C++ destructors and catch blocks. But instead of using ordinary exceptions, it uses "forced unwinding" for thread cancel and exit. For some unknown reason (other than "I say it must be so"), glibc maintainer Drepper of Red Hat simply refuses to even contemplate changing it to use ordinary exceptions instead of "forced unwinding"... and I suspect that is the reason why WG21 was asked to drop standardization of thread cancel (and exit) as exceptions. regards, alexander.
Alexander Terekhov <terekhov@web.de> writes:
Anthony Williams wrote: [...]
N2420 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2420.pdf) covers some of the relevant ground --- it's the minutes of the POSIX/C++ liaison committee.
"Gnu gcc and Solaris pthreads: The Gnu gcc and Solaris pthreads implementations are two known implementations that attempt to map POSIX pthread cancellation onto C++ exception handling, but both do so at the cost of breaking the exception model (i.e., they no longer conform to ISO C++) because the alternative appears to be that C++ destructors and catch blocks would not be invoked for cancellation which would mean that resources would be leaked."
Uhmm. Don't know about (modern) Solaris, but glibc (presumably that is meant by "Gnu gcc pthreads implementation") does invoke C++ destructors and catch blocks. But instead of using ordinary exceptions, it uses "forced unwinding" for thread cancel and exit. For some unknown reason (other than "I say it must be so"), glibc maintainer Drepper of Red Hat simply refuses to even contemplate changing it to use ordinary exceptions instead of "forced unwinding"... and I suspect that is the reason why WG21 was asked to drop standardization of thread cancel (and exit) as exceptions.
That may be a contributory factor, but it's certainly not the only reason. At the WG21 meeting in Oxford last April, Nick Stoughton made it quite clear that POSIX cancellation is sticky --- you cannot catch it and resume processing (as Dave Abrahams was keen to point out, you could write a cleanup handler that didn't return, and then proceeded to do what the main function would have done, but you couldn't then cancel the thread again). Exceptions are not --- you can catch everything in catch(...), and you don't have to rethrow. Also, Howard pointed out that the C ABI for one of the Apple compilers doesn't handle C++ exceptions in a way that would allow cancellation implemented as a C++ exception to call pthread cleanup handlers in a C stack frame. Breaking the C ABI to support the C++ thread library isn't nice. I still like the exception-based cancel. Though interoperability with pthread_cancel would be nice, it's not a must-have from where I'm sitting. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams:
Also, Howard pointed out that the C ABI for one of the Apple compilers doesn't handle C++ exceptions in a way that would allow cancellation implemented as a C++ exception to call pthread cleanup handlers in a C stack frame. Breaking the C ABI to support the C++ thread library isn't nice.
I believe that Solaris does manage to do a C++ friendly unwind without breaking its C ABI. It apparently does so by looking at the linked list created by pthread_cleanup_push and invoking the registered functions and the destructors in the proper order. I could be wrong though. The source code of the C++ friendly unwind doesn't seem to be available.
Alexander Terekhov wrote:
Anthony Williams wrote: [...]
N2420 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2420.pdf) covers some of the relevant ground --- it's the minutes of the POSIX/C++ liaison committee.
"Gnu gcc and Solaris pthreads: The Gnu gcc and Solaris pthreads implementations are two known implementations that attempt to map POSIX pthread cancellation onto C++ exception handling, but both do so at the cost of breaking the exception model (i.e., they no longer conform to ISO C++) because the alternative appears to be that C++ destructors and catch blocks would not be invoked for cancellation which would mean that resources would be leaked."
Uhmm. Don't know about (modern) Solaris, but glibc (presumably that is meant by "Gnu gcc pthreads implementation") does invoke C++ destructors and catch blocks.
Realy??? Wow! Can you direct me to some documentation? Thanks, Phil.
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Alexander Terekhov wrote:
Anthony Williams wrote: [...]
N2420 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2420.pdf) covers some of the relevant ground --- it's the minutes of the POSIX/C++ liaison committee.
"Gnu gcc and Solaris pthreads: The Gnu gcc and Solaris pthreads implementations are two known implementations that attempt to map POSIX pthread cancellation onto C++ exception handling, but both do so at the cost of breaking the exception model (i.e., they no longer conform to ISO C++) because the alternative appears to be that C++ destructors and catch blocks would not be invoked for cancellation which would mean that resources would be leaked."
Uhmm. Don't know about (modern) Solaris, but glibc (presumably that is meant by "Gnu gcc pthreads implementation") does invoke C++ destructors and catch blocks.
Realy??? Wow! Can you direct me to some documentation?
I can't find it documented anywhere sensible, but I've seen it discussed on mailing lists several times. As I understand it, with gcc on linux, pthread_cancel will run destructors and catch(...) blocks in the cancelled thread, but if the catch(...) block doesn't rethrow the exception, it is automatically rethrown at the end of the block (like function-try-blocks do in constructors), so that the cancel is "sticky". Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams:
As I understand it, with gcc on linux, pthread_cancel will run destructors and catch(...) blocks in the cancelled thread, but if the catch(...) block doesn't rethrow the exception, it is automatically rethrown at the end of the block
No, NPTL calls abort() if the catch block ends without a rethrow. There is a patch (by Jason Mitchell, I believe) that makes it not do that. There is no technical reason to make NPTL cancel appear to C++ code as an ordinary C++ exception, it's politics. Even so, NPTL is much, much better than any of the alternatives (except maybe OpenVMS where cancelation is an OS "structured exception").
Anthony Williams wrote:
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Alexander Terekhov wrote:
Anthony Williams wrote: [...]
N2420 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2420.pdf) covers some of the relevant ground --- it's the minutes of the POSIX/C++ liaison committee.
"Gnu gcc and Solaris pthreads: The Gnu gcc and Solaris pthreads implementations are two known implementations that attempt to map POSIX pthread cancellation onto C++ exception handling, but both do so at the cost of breaking the exception model (i.e., they no longer conform to ISO C++) because the alternative appears to be that C++ destructors and catch blocks would not be invoked for cancellation which would mean that resources would be leaked."
Uhmm. Don't know about (modern) Solaris, but glibc (presumably that is meant by "Gnu gcc pthreads implementation") does invoke C++ destructors and catch blocks.
Realy??? Wow! Can you direct me to some documentation?
I can't find it documented anywhere sensible, but I've seen it discussed on mailing lists several times.
As I understand it, with gcc on linux, pthread_cancel will run destructors and catch(...) blocks in the cancelled thread, but if the catch(...) block doesn't rethrow the exception, it is automatically rethrown at the end of the block (like function-try-blocks do in constructors), so that the cancel is "sticky".
<rant>I can't believe how much time I have wasted trying to sidestep the need for cancellation, and investigating how to hack some approximation to cancellation into my application (e.g. by using processes and shared memory rather than threads), only to discover now that exactly the feature that I need is already present in glibc - just not documented by them.</rant> One of the good features of Boost is the high-quality documentation, so this sort of mistake wouldn't happen here, would it? So now there's a choice: - The pthread_cancel functionality in glibc has cancellation points at blocking system calls, which is essential to me and I imagine to many people. But it doesn't throw catchable exceptions, which some people would like (but I don't need). - The approach that Anthony has described would have difficulty introducing cancellation points at system calls. But it generates catchable exceptions, and it's cross-platform. Having different thread classes with similar interfaces but different cancellation behaviour is not ideal, but neither is it inherently a huge problem. However, if I have some long-running loop that wants to do periodic cancellation tests, or if I'm writing a mutex, it needs to do different things depending on the kind of cancellation. Maybe some sort of per-thread function pointer for testcancel() is needed? Regards, Phil.
on Fri Nov 02 2007, "Phil Endecott" <spam_from_boost_dev-AT-chezphil.org> wrote:
- The pthread_cancel functionality in glibc has cancellation points at blocking system calls, which is essential to me and I imagine to many people. But it doesn't throw catchable exceptions, which some people would like (but I don't need).
For what it's worth, both of the following will break previously exception-safe code: * The introduction of unwinding to (system) calls that are currently specified to be nonthrowing (e.g. fopen). * The introduction of an uncatchable exception. Just something to keep in mind. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
Phil Endecott wrote:
Alexander Terekhov wrote:
"Gnu gcc and Solaris pthreads: The Gnu gcc and Solaris pthreads implementations are two known implementations that attempt to map POSIX pthread cancellation onto C++ exception handling, but both do so at the cost of breaking the exception model (i.e., they no longer conform to ISO C++) because the alternative appears to be that C++ destructors and catch blocks would not be invoked for cancellation which would mean that resources would be leaked."
Uhmm. Don't know about (modern) Solaris, but glibc (presumably that is meant by "Gnu gcc pthreads implementation") does invoke C++ destructors and catch blocks.
<rant>I can't believe how much time I have wasted trying to sidestep the need for cancellation, and investigating how to hack some approximation to cancellation into my application (e.g. by using processes and shared memory rather than threads), only to discover now that exactly the feature that I need is already present in glibc - just not documented by them.</rant>
In case anyone is curious, I have put together a simple thread class that uses the pthread_cancel. You can see it here: http://chezphil.org/tmp/Thread.hh I've done this mainly to prove to myself that pthread_cancel really does invoke destructors. But I've also taken the opportunity to do a couple of other things: * I've written a set of scoped_* classes that change the thread cancellation state, e.g. { scoped_disable_cancellation d; non_cancellation_safe_function(); // As Dave Abrahams points out, you need this around code that relies on catch blocks since // the glibc pthread_cancel doesn't call them. } { scoped_enable_async_cancellation e; for (int i=0; i<1000000000; ++i) { complex_maths(i); } // I imagine that only certain kinds of code can be used safely with async cancellation. // The example that interests me is JPEG image decoding with libjpeg. } { scoped_enable_deferred_cancellation e; code_that_does_blocking_io(); } This would also fit with the approach that Anthony has been using, apart from the async mode. * I've written the thread class with its lifetime tied to the lifetime of the thread, in contrast to Boost.Thread, which is more like a thread-pointer. This makes a bit more sense with cancellation available; the thread is cancelled by deleting the thread object. I'm going to look at whether this makes for more or less convenient coding in my applications. Of course, there's no reason why this can't co-exist with the thread-handle approach, with one implemented in terms of the other. I haven't yet looked into the testcancel() problem (i.e. code that wants to provide an explicit cancellation point needs to do so in a way that's appropriate for the kind of cancellation in use). Anthony, do you have any thoughts about providing a per-thread function pointer for this? Regards, Phil.
Alexander Terekhov:
Anthony Williams wrote: [...]
N2420 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2420.pdf) covers some of the relevant ground --- it's the minutes of the POSIX/C++ liaison committee.
"Gnu gcc and Solaris pthreads: The Gnu gcc and Solaris pthreads implementations are two known implementations that attempt to map POSIX pthread cancellation onto C++ exception handling, but both do so at the cost of breaking the exception model (i.e., they no longer conform to ISO C++) because the alternative appears to be that C++ destructors and catch blocks would not be invoked for cancellation which would mean that resources would be leaked."
Uhmm. Don't know about (modern) Solaris, but glibc (presumably that is meant by "Gnu gcc pthreads implementation") does invoke C++ destructors and catch blocks.
Solaris invokes destructors and does not invoke catch blocks (not because they can't, but because they've decided not to.)
Alexander Terekhov wrote:
Anthony Williams wrote: [...]
N2420 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2420.pdf) covers some of the relevant ground --- it's the minutes of the POSIX/C++ liaison committee.
"Gnu gcc and Solaris pthreads: The Gnu gcc and Solaris pthreads implementations are two known implementations that attempt to map POSIX pthread cancellation onto C++ exception handling, but both do so at the cost of breaking the exception model (i.e., they no longer conform to ISO C++) because the alternative appears to be that C++ destructors and catch blocks would not be invoked for cancellation which would mean that resources would be leaked."
Uhmm. Don't know about (modern) Solaris, but glibc (presumably that is meant by "Gnu gcc pthreads implementation") does invoke C++ destructors and catch blocks. But instead of using ordinary exceptions, it uses "forced unwinding" for thread cancel and exit. For some unknown reason (other than "I say it must be so"), glibc maintainer Drepper of Red Hat simply refuses to even contemplate changing it to use ordinary exceptions instead of "forced unwinding"... and I suspect that is the reason why WG21 was asked to drop standardization of thread cancel (and exit) as exceptions.
Yep, Ulrich Drepper was one of the people involved, but there were others also concerned. I suspect Boost will run into some platforms where we can't rely on some of the out-of-the-box native functionality, and have to do a certain amount of wrapping or reimplementing. But that doesn't mean we shouldn't try. There are a lot of smart people contributing ideas for Boost.Threads, and with enough minds on these issues we have a reasonable chance of resolving these issues. --Beman
On Nov 2, 2007, at 4:05 AM, Anthony Williams wrote
I added better cancellation to boost thread yesterday. condition wait+timed_wait, thread join+timed_join, sleep and this_thread::cancellation_point are now all cancellation points on pthreads and windows. This uses a thread_cancelled exception, and does not interoperate with pthread_cancel.
Thanks Anthony! -Howard
"Phil Endecott" <spam_from_boost_dev@chezphil.org> writes:
Howard Hinnant wrote:
Actually thread cancellation is old. Many (not all) thread API's have some form or another of cancellation. That being said, the compromise reached at the Kona meeting 4 weeks ago was to remove cancellation (or interruption) from the proposal. This removal is reflected in N2411.
That's unfortunate. I'm a little confused about how completely removing it is a "compromise"!
This is something that I need. Does anyone know of an alternative (POSIX-compatible) C++ thread library that offers cancellation? If not I may try to write on.
By "POSIX-compatible" do you mean "works with and uses pthread_cancel", or do you mean something else? It's the interaction with pthread_cancel that caused the biggest sticking point, as I understand things. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Howard Hinnant wrote:
On Oct 28, 2007, at 11:53 AM, Yuval Ronen wrote:
* Thread cancellation is new, and disable_cancellation/restore_cancellation are even newer. They are new for C++ programmers, and maybe new for *all* programmers (I never heard of a language with them). I'm not sure if it's a good idea to standardize them before we get some real-life experience with thread cancellation.
Actually thread cancellation is old. Many (not all) thread API's have some form or another of cancellation. That being said, the compromise reached at the Kona meeting 4 weeks ago was to remove cancellation (or interruption) from the proposal. This removal is reflected in N2411.
I see.
* I see that you chose not go accept a mutex in condition's constructor. If that is so, why make the condition class templated, instead of only the wait() functions, as is in Boost.Thread?
There was an interim paper:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2406.html
which changed the way cv's were handled from N2320. In Kona this was folded into N2411, and also renamed along the way. The condition's constructor still does not take a mutex. There are now two condition variable classes, and neither is templated. The rationale for this design is in N2406.
I'll try to find some time soon to read it...
* If C++0x is going to have lambda support (is it?), then maybe the not-accepting-predicate overloads of condition::wait are no longer needed, and can be removed? I think the major reason why people use that overload is because they are too lazy to write a functor. With core support for lambda, it's a breeze. This will solve the other problem (mentioned in some other thread in this ML) about absolute vs. relative times.
I would be very hesitant to force one programming style over another. Sometimes, for whatever reasons, one might really need an explicit loop (say to handle more complex flow control than just while (!pred) cv.wait(lk)).
Is there any known use case for that? The common practice, and as far as I remember, it's also defined by POSIX, that *all* calls to wait must be like while (!pred) cv.wait(), so there is no other way. Is it?
* Back then we had a discussion about unique_lock, which was never finished. I claim that there is no need for a state where mutex() != 0 && owns() == false, which means that defer_lock is not needed. There is one example in the document using defer_lock, which I think can be done in another way. That example goes like:
Record& Record::operator=(const Record& r) { if (this != &r) { std::unique_lock<std::mutex> this_lock(mut, std::defer_lock); std::unique_lock<std::mutex> that_lock(r.mut, std::defer_lock); std::lock(this_lock, that_lock); // Both source and destination are locked // Safe to assign // ... } return *this; }
If we change the first 3 lines of the 'if' to:
std::lock(mut, r.mut); std::unique_lock<std::mutex> this_lock(mut, std::accept_ownership); std::unique_lock<std::mutex> that_lock(r.mut, std::accept_ownership);
then it works without defer_lock.
<nod> However I'm not comfortable with claiming that defer_lock has no use beyond this use case, especially when defer_lock-support costs nothing (see below). This is especially true given that boost::thread has supported the defer_lock functionality (under a different spelling) for six or seven years and this is the first report I've seen that the functionality is sufficiently complicating that it should be removed.
This allows us to: 1. Make unique_lock's interface simpler. 2. Make sizeof(unique_lock) smaller - no need for bool owns.
Even if we remove defer_lock, unique_lock will still need the internal bool to support try and timed locks. This functionality still retains a reference to the mutex even if the try/timed lock fails (so that further action with the mutex can be easily taken).
This is something I don't understand. Why should the lock retain a reference to the mutex even if the try/timed lock fails?
3. Make unique_lock more similar to unique_ptr, which makes it more intuitive. 4. Make std::lock simpler in the way that it doesn't need to accept locks, only mutexes.
std::lock isn't further complicated by accepting both mutexes and locks. It accepts anything that supports lock(), try_lock() and unlock(). To disallow locks, we would have to make std::lock more complicated by detecting lock template arguments and then actively disabling them. I see no motivation to add this complication.
Oh no, I wasn't suggesting something like that at all. By "make std::lock simpler" I meant simpler in our heads. Make our thinking of it simpler. Thinking about it (and documenting it) as something that deals with mutexes is simpler than if it was dealing with both mutexes and locks.
In general it seems you paid a lot of attention to being able to pass a lock instead of a mutex wherever possible. I think it's absolutely not necessary. It's an undue complication.
Actually I tried to pay attention to what the minimal requirements were on template parameters. It just turned out that some generic code would accept either locks or mutexes.
But you made the effort of mentioning it several times in the document. That implies importance. My claim is that there is no importance to it, because there's no use for it, even if it happens that we named unique_lock::lock the same way we named mutex::lock. We could've have named them differently. And while I'm thinking of it, why is there a lock() method for unique_lock anyway? Isn't it unnecessary?
* Other than that - very nice :)
Thanks. Sorry the paper keeps changing out from under you. There are many people actively working a single document now and such changes are inevitable.
No need to apologize. I much prefer it like that, than the other way around, where the document was forgotten, accumulating dust... :)
On Oct 30, 2007, at 3:43 PM, Yuval Ronen wrote:
* If C++0x is going to have lambda support (is it?), then maybe the not-accepting-predicate overloads of condition::wait are no longer needed, and can be removed? I think the major reason why people use that overload is because they are too lazy to write a functor. With core support for lambda, it's a breeze. This will solve the other problem (mentioned in some other thread in this ML) about absolute vs. relative times.
I would be very hesitant to force one programming style over another. Sometimes, for whatever reasons, one might really need an explicit loop (say to handle more complex flow control than just while (!pred) cv.wait(lk)).
Is there any known use case for that? The common practice, and as far as I remember, it's also defined by POSIX, that *all* calls to wait must be like while (!pred) cv.wait(), so there is no other way. Is it?
I was thinking about "embellished" while loops. Maybe something like: while (!pred()) { cv.timed_wait(lk, std::get_system_time() + std::milliseconds(100)); bool is_canceled = true; if (std::atomic_compare_and_swap(&cancel, &is_canceled, false)) { cv2.notify_one(); while (!exceptional_pred()) cv.wait(lk); throw thread_canceled(); } } All of this could probably be packed into a predicate. But it might be inconvenient. Sometimes it is easier to just write your own loop.
2. Make sizeof(unique_lock) smaller - no need for bool owns.
Even if we remove defer_lock, unique_lock will still need the internal bool to support try and timed locks. This functionality still retains a reference to the mutex even if the try/timed lock fails (so that further action with the mutex can be easily taken).
This is something I don't understand. Why should the lock retain a reference to the mutex even if the try/timed lock fails?
Because if the try_lock fails, I might want to take some corrective action and then do a blocking lock() on the mutex. If the lock has retained the reference to the mutex, I don't have to store that reference elsewhere "just in case" the try_lock fails.
3. Make unique_lock more similar to unique_ptr, which makes it more intuitive. 4. Make std::lock simpler in the way that it doesn't need to accept locks, only mutexes.
std::lock isn't further complicated by accepting both mutexes and locks. It accepts anything that supports lock(), try_lock() and unlock(). To disallow locks, we would have to make std::lock more complicated by detecting lock template arguments and then actively disabling them. I see no motivation to add this complication.
Oh no, I wasn't suggesting something like that at all. By "make std::lock simpler" I meant simpler in our heads. Make our thinking of it simpler. Thinking about it (and documenting it) as something that deals with mutexes is simpler than if it was dealing with both mutexes and locks.
The current spec just says: Each template parameter type must supply the following member functions with semantics corresponding to the Mutex concept, except that try_lock is allowed to throw an exception [Note: The unique_lock class template meets these requirements when suitable instantiated. -- end note] void lock(); bool try_lock(); void unlock(); I suppose we could remove the note. But I really don't think that simplifies anything.
In general it seems you paid a lot of attention to being able to pass a lock instead of a mutex wherever possible. I think it's absolutely not necessary. It's an undue complication.
Actually I tried to pay attention to what the minimal requirements were on template parameters. It just turned out that some generic code would accept either locks or mutexes.
But you made the effort of mentioning it several times in the document. That implies importance. My claim is that there is no importance to it, because there's no use for it, even if it happens that we named unique_lock::lock the same way we named mutex::lock. We could've have named them differently. And while I'm thinking of it, why is there a lock() method for unique_lock anyway? Isn't it unnecessary?
The original answer is because that's the way boost::scoped_lock specified its API. After further study and experience, I find myself in complete agreement with the boost API in this regard, at least for unique_lock. I think the member lock() functions on unique_lock are quite convenient. That being said, please see lock_guard in N2447 which both lacks the defer_lock functionality, and lacks member lock, try_lock and unlock functions. It also lacks the internal bool you would like to get rid of. I think we may already have what you're looking for, just under a different name. -Howard
Howard Hinnant wrote:
On Oct 30, 2007, at 3:43 PM, Yuval Ronen wrote:
2. Make sizeof(unique_lock) smaller - no need for bool owns. Even if we remove defer_lock, unique_lock will still need the internal bool to support try and timed locks. This functionality still retains a reference to the mutex even if the try/timed lock fails (so that further action with the mutex can be easily taken). This is something I don't understand. Why should the lock retain a reference to the mutex even if the try/timed lock fails?
Because if the try_lock fails, I might want to take some corrective action and then do a blocking lock() on the mutex. If the lock has retained the reference to the mutex, I don't have to store that reference elsewhere "just in case" the try_lock fails.
But now you do it (by "it" I mean "retained the reference, just in case") all the time, for everyone, even if unnecessary. I believe this is a rare use case (I never written or read such code), and it's also something that can be easily implemented externally (either as a unique_lock wrapper, or not). It seems pity that everyone should pay that cost.
In general it seems you paid a lot of attention to being able to pass a lock instead of a mutex wherever possible. I think it's absolutely not necessary. It's an undue complication. Actually I tried to pay attention to what the minimal requirements were on template parameters. It just turned out that some generic code would accept either locks or mutexes. But you made the effort of mentioning it several times in the document. That implies importance. My claim is that there is no importance to it, because there's no use for it, even if it happens that we named unique_lock::lock the same way we named mutex::lock. We could've have named them differently. And while I'm thinking of it, why is there a lock() method for unique_lock anyway? Isn't it unnecessary?
The original answer is because that's the way boost::scoped_lock specified its API. After further study and experience, I find myself in complete agreement with the boost API in this regard, at least for unique_lock. I think the member lock() functions on unique_lock are quite convenient. That being said, please see lock_guard in N2447 which both lacks the defer_lock functionality, and lacks member lock, try_lock and unlock functions. It also lacks the internal bool you would like to get rid of. I think we may already have what you're looking for, just under a different name.
What makes member lock() functions on unique_lock convenient? In other words, when are they convenient? I've skimmed N2447 very quickly, and it seems that lock_guard is a rename of scoped_lock, right? The major difference between lock_guard and a stripped unique_lock, is the moveability. There is no vector<lock_guard>, and if we are to support vector<unique_lock>, then we should support it well, which means reducing the used size by half.
Yuval Ronen <ronen_yuval@yahoo.com> writes:
Howard Hinnant wrote:
On Oct 30, 2007, at 3:43 PM, Yuval Ronen wrote:
2. Make sizeof(unique_lock) smaller - no need for bool owns. Even if we remove defer_lock, unique_lock will still need the internal bool to support try and timed locks. This functionality still retains a reference to the mutex even if the try/timed lock fails (so that further action with the mutex can be easily taken). This is something I don't understand. Why should the lock retain a reference to the mutex even if the try/timed lock fails?
Because if the try_lock fails, I might want to take some corrective action and then do a blocking lock() on the mutex. If the lock has retained the reference to the mutex, I don't have to store that reference elsewhere "just in case" the try_lock fails.
But now you do it (by "it" I mean "retained the reference, just in case") all the time, for everyone, even if unnecessary. I believe this is a rare use case (I never written or read such code), and it's also something that can be easily implemented externally (either as a unique_lock wrapper, or not). It seems pity that everyone should pay that cost.
unique_lock needs to retain a reference to the mutex in order to unlock it in the destructor. The cost comes from the fact that retaining the reference means you need a separate bool to indicate locked/unlocked. Depending on the pointer format and alignment requirements of the mutex, this could be stored as the LSB of a pointer to the mutex to save the space (for example). unique_lock is a lock object associated with a mutex. The mutex doesn't change through the life of the lock object, though it may be locked or unlocked many times. Code that uses the lock object doesn't need to know which mutex it refers to: it can just call lock() and unlock() on the lock object. Condition variables have to do that: they call unlock() at the start of the wait, and lock() when done. lock_guard is a simple always-owns-the-lock object, and thus saves the space of the boolean. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams wrote:
Yuval Ronen <ronen_yuval@yahoo.com> writes:
On Oct 30, 2007, at 3:43 PM, Yuval Ronen wrote:
2. Make sizeof(unique_lock) smaller - no need for bool owns. Even if we remove defer_lock, unique_lock will still need the internal bool to support try and timed locks. This functionality still retains a reference to the mutex even if the try/timed lock fails (so that further action with the mutex can be easily taken). This is something I don't understand. Why should the lock retain a reference to the mutex even if the try/timed lock fails? Because if the try_lock fails, I might want to take some corrective action and then do a blocking lock() on the mutex. If the lock has retained the reference to the mutex, I don't have to store that reference elsewhere "just in case" the try_lock fails. But now you do it (by "it" I mean "retained the reference, just in case") all the time, for everyone, even if unnecessary. I believe this is a rare use case (I never written or read such code), and it's also something that can be easily implemented externally (either as a unique_lock wrapper, or not). It seems pity that everyone should pay
Howard Hinnant wrote: that cost.
unique_lock needs to retain a reference to the mutex in order to unlock it in the destructor. The cost comes from the fact that retaining the reference means you need a separate bool to indicate locked/unlocked.
You say that you need both a reference and a bool to implement unique_lock's interface. This is obvious. My claim was that there is no use case that cannot be satisfied with a *simpler* interface, one that can be implemented with a single mutex pointer that is NULL when the mutex is not owned by the lock. And even if there is such a use case, it's very rate (and can be done externally). I haven't seen any yet.
Depending on the pointer format and alignment requirements of the mutex, this could be stored as the LSB of a pointer to the mutex to save the space (for example).
"Depending on the pointer format and alignment requirements" is hardly an argument that there is no cost, you surely know that...
unique_lock is a lock object associated with a mutex. The mutex doesn't change through the life of the lock object, though it may be locked or unlocked many times. Code that uses the lock object doesn't need to know which mutex it refers to: it can just call lock() and unlock() on the lock object. Condition variables have to do that: they call unlock() at the start of the wait, and lock() when done.
Condition variables are still not the use case I'm looking for (anyone remembers the line form Star Wars IV "These are not the men you're looking for"? :) ) The cv code can easily call lock() and unlock() for lk.mutex(), and anyway, it needs intimate knowledge of the mutex, so it probably bypasses the lock. Well, it seems we're not going to convince each other that easily. So anyway it goes, thanks for all the hard work.
Howard Hinnant wrote:
On Oct 28, 2007, at 11:53 AM, Yuval Ronen wrote:
[snip]
* Thread cancellation is new, and disable_cancellation/restore_cancellation are even newer. They are new for C++ programmers, and maybe new for *all* programmers (I never heard of a language with them). I'm not sure if it's a good idea to standardize them before we get some real-life experience with thread cancellation.
Actually thread cancellation is old. Many (not all) thread API's have some form or another of cancellation. That being said, the compromise reached at the Kona meeting 4 weeks ago was to remove cancellation (or interruption) from the proposal. This removal is reflected in N2411.
Oh, no. I think thread cancellation is a must-have in general, and it would be really great to have it within the standard. It's not that it's a big problem to make/use composite cv predicates, where one part holds the "cancel" part and the other the real stuff - but having this in the standard would allow implementations to even cancel e.g. unconditional mutex waits. Sure, C and C++ compatibility would be great - but is this "just" a matter of principle, or is there a large user base which is only waiting for cross-language threading compatibility? I'm certainly not (sorry). / Johan
On Oct 31, 2007, at 12:45 AM, Johan Nilsson wrote:
Howard Hinnant wrote:
On Oct 28, 2007, at 11:53 AM, Yuval Ronen wrote:
[snip]
* Thread cancellation is new, and disable_cancellation/restore_cancellation are even newer. They are new for C++ programmers, and maybe new for *all* programmers (I never heard of a language with them). I'm not sure if it's a good idea to standardize them before we get some real-life experience with thread cancellation.
Actually thread cancellation is old. Many (not all) thread API's have some form or another of cancellation. That being said, the compromise reached at the Kona meeting 4 weeks ago was to remove cancellation (or interruption) from the proposal. This removal is reflected in N2411.
Oh, no. I think thread cancellation is a must-have in general, and it would be really great to have it within the standard.
It's not that it's a big problem to make/use composite cv predicates, where one part holds the "cancel" part and the other the real stuff - but having this in the standard would allow implementations to even cancel e.g. unconditional mutex waits.
Sure, C and C++ compatibility would be great - but is this "just" a matter of principle, or is there a large user base which is only waiting for cross-language threading compatibility? I'm certainly not (sorry).
Here's an interesting read on the subject: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2455.pdf For those of you not familiar with the language of standardization, this is an official letter from the C committee to the C++ committee saying: "Thank you for removing cancellation. Now we want you to promise that you will not even discuss bringing it back." -Howard
Howard Hinnant wrote:
On Oct 31, 2007, at 12:45 AM, Johan Nilsson wrote:
[snip]
Sure, C and C++ compatibility would be great - but is this "just" a matter of principle, or is there a large user base which is only waiting for cross-language threading compatibility? I'm certainly not (sorry).
Here's an interesting read on the subject:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2455.pdf
For those of you not familiar with the language of standardization, this is an official letter from the C committee to the C++ committee saying: "Thank you for removing cancellation. Now we want you to promise that you will not even discuss bringing it back."
I think I can understand why members of the standard committee(s), with an explicit goal of providing cross-language compatibility between C and C++, will want to remove cancellation. Are there any rationales available for the uninitiated? Thanks / Johan
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Howard Hinnant Sent: 01 November 2007 02:16 To: boost@lists.boost.org Subject: Re: [boost] Comments about N2320
Sure, C and C++ compatibility would be great - but is this "just" a matter of principle, or is there a large user base which is only waiting for cross-language threading compatibility? I'm certainly not (sorry).
Here's an interesting read on the subject:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2455.pdf
For those of you not familiar with the language of standardization, this is an official letter from the C committee to the C++ committee saying: "Thank you for removing cancellation. Now we want you to promise that you will not even discuss bringing it back."
Sounds like the tail is trying to wag the dog! Paul --- Paul A Bristow Prizet Farmhouse, Kendal, Cumbria UK LA8 8AB +44 1539561830 & SMS, Mobile +44 7714 330204 & SMS pbristow@hetp.u-net.com
Howard Hinnant <howard.hinnant@gmail.com> writes:
Here's an interesting read on the subject:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2455.pdf
For those of you not familiar with the language of standardization, this is an official letter from the C committee to the C++ committee saying: "Thank you for removing cancellation. Now we want you to promise that you will not even discuss bringing it back."
I just spotted that in the mailing. Ouch. I know we removed cancellation so we could make progress on other stuff, but I really think we ought to bring it back. One of the key things people have said to me when I have mentioned that we're standardizing a thread library for C++ is "does that include cancellation?" Everyone I've spoken to about it has been very glad the answer was "yes." I've added cancellation to the boost thread library (though currently this_thread::cancellation_point is the only cancellation point on pthreads --- something I intend to fix), and I don't intend to remove it. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams wrote:
Howard Hinnant <howard.hinnant@gmail.com> writes:
Here's an interesting read on the subject:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2455.pdf
For those of you not familiar with the language of standardization, this is an official letter from the C committee to the C++ committee saying: "Thank you for removing cancellation. Now we want you to promise that you will not even discuss bringing it back."
I just spotted that in the mailing. Ouch. I know we removed cancellation so we could make progress on other stuff, but I really think we ought to bring it back.
One of the key things people have said to me when I have mentioned that we're standardizing a thread library for C++ is "does that include cancellation?" Everyone I've spoken to about it has been very glad the answer was "yes."
I've added cancellation to the boost thread library (though currently this_thread::cancellation_point is the only cancellation point on pthreads --- something I intend to fix), and I don't intend to remove it.
I think Boost will perform a valuable service to the community if we develop successful cancellation extensions. The name, however, is very contentious, particularly among the POSIX community. It appears to me that any C++ language or library use of the name "cancellation" causes the POSIX community to try to derail all further progress, regardless of technical merit or existing practice. I strongly suggest we use the name "interruption". That appears to be much more acceptable. --Beman
Beman Dawes <bdawes@acm.org> writes:
Anthony Williams wrote:
Howard Hinnant <howard.hinnant@gmail.com> writes:
Here's an interesting read on the subject:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2455.pdf
For those of you not familiar with the language of standardization, this is an official letter from the C committee to the C++ committee saying: "Thank you for removing cancellation. Now we want you to promise that you will not even discuss bringing it back."
I just spotted that in the mailing. Ouch. I know we removed cancellation so we could make progress on other stuff, but I really think we ought to bring it back.
One of the key things people have said to me when I have mentioned that we're standardizing a thread library for C++ is "does that include cancellation?" Everyone I've spoken to about it has been very glad the answer was "yes."
I've added cancellation to the boost thread library (though currently this_thread::cancellation_point is the only cancellation point on pthreads --- something I intend to fix), and I don't intend to remove it.
I think Boost will perform a valuable service to the community if we develop successful cancellation extensions.
Thank you for the support. I have now extended the current thread library on trunk so that it supports all the cancellation points from N2320 on POSIX and Windows.
The name, however, is very contentious, particularly among the POSIX community. It appears to me that any C++ language or library use of the name "cancellation" causes the POSIX community to try to derail all further progress, regardless of technical merit or existing practice.
I strongly suggest we use the name "interruption". That appears to be much more acceptable.
I'll change the name over. It's just a name, after all. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Beman Dawes wrote:
I strongly suggest we use the name "interruption". That appears to be much more acceptable.
I recognized this quite some time ago already and tried to suggest "alert". Because this is what the suggested mechanism really means. "Alerting" a thread means causing it to throw an exception when in an alertable state. While interrupt describes also what the mechanism is about to do, the name is overloaded with HW interrupt semantics already. Personally I think it should be avoided as for this reason. Not trying to mix alertion with cancellation, so was my hope, would reduce the risk of misunderstanding. Of course it is possible to turn alertion into cancellation on the user-side easily. But, my suggestion went by almost unrecognized. Perhaps this is a new chance? Roland aka speedsnail
Roland Schwarz <roland.schwarz@chello.at> writes:
Beman Dawes wrote:
I strongly suggest we use the name "interruption". That appears to be much more acceptable.
I recognized this quite some time ago already and tried to suggest "alert". Because this is what the suggested mechanism really means. "Alerting" a thread means causing it to throw an exception when in an alertable state.
While interrupt describes also what the mechanism is about to do, the name is overloaded with HW interrupt semantics already. Personally I think it should be avoided as for this reason.
Not trying to mix alertion with cancellation, so was my hope, would reduce the risk of misunderstanding. Of course it is possible to turn alertion into cancellation on the user-side easily.
But, my suggestion went by almost unrecognized. Perhaps this is a new chance?
I didn't see your previous suggestion to use "alert" rather than "cancel". However, I do like interruption, and I have renamed cancellation to interruption on trunk. I know there is a potential conflict with the use of the term with hardware interrupts, but it is consistent with Java and .NET Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams wrote:
I didn't see your previous suggestion to use "alert" rather than "cancel". However, I do like interruption, and I have renamed cancellation to interruption on trunk.
Why not even take another step too: throw-at(threadid, exception); instead of interruption? After all it would underpin that interruption is not normal control flow. Roland aka speedsnail
Roland Schwarz <roland.schwarz@chello.at> writes:
Anthony Williams wrote:
I didn't see your previous suggestion to use "alert" rather than "cancel". However, I do like interruption, and I have renamed cancellation to interruption on trunk.
Why not even take another step too:
throw-at(threadid, exception);
instead of interruption? After all it would underpin that interruption is not normal control flow.
That's a whole lot more heavyweight --- you'd have to transport the exception into the thread in a type-safe manner in order to throw it. It's not impossible --- you can copy the passed exception into a buffer along with a function to throw it, and pass that over to the thread --- but it's conceptually a lot more complicated. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Roland Schwarz wrote:
Anthony Williams wrote:
I didn't see your previous suggestion to use "alert" rather than "cancel". However, I do like interruption, and I have renamed cancellation to interruption on trunk.
Why not even take another step too:
throw-at(threadid, exception);
No problem as long as exception IS-A std::thread_cancel_request. :-) regards, alexander.
Howard Hinnant wrote:
Here's an interesting read on the subject:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2455.pdf
For those of you not familiar with the language of standardization, this is an official letter from the C committee to the C++ committee saying: "Thank you for removing cancellation. Now we want you to promise that you will not even discuss bringing it back."
I was pretty surprised by N2455 and figured that there's a missing context, without which it doesn't make sense to the uninformed.
Peter Dimov wrote:
Howard Hinnant wrote:
Here's an interesting read on the subject:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2455.pdf
For those of you not familiar with the language of standardization, this is an official letter from the C committee to the C++ committee saying: "Thank you for removing cancellation. Now we want you to promise that you will not even discuss bringing it back."
I was pretty surprised by N2455 and figured that there's a missing context, without which it doesn't make sense to the uninformed.
_______________________________________________
I was hoping somebody else would ask so that I didn't seem like the only uninformed... but since I have not seen the questions yet: Why not branch C and C++ further? Why try to find a solution that works well for both languages? I am a user of both languages on nearly a daily basis. I already have to adjust my mind set to not think of the two as the same. Simple differences such as where I can declare variables already change the style enough that the two are just completely different languages in my mind. Sure, they can cooperatively use the same libraries if care is taken, but the same can be said about many other languages with bindings. -- ---------------------------------- Michael Caisse Object Modeling Designs www.objectmodelingdesigns.com
Michael Caisse:
I was hoping somebody else would ask so that I didn't seem like the only uninformed... but since I have not seen the questions yet: Why not branch C and C++ further? Why try to find a solution that works well for both languages?
There are two main reasons, one, because a significant portion of the community doesn't want or need such branching, and two, because the C runtime library is (both de facto and de jure) part of C++. So a "C++ cancelation" that doesn't cancel the C blocking waits would be severely limited in practice, unless the C++ library is revamped to offer pure C++ equivalents to all blocking C functions. (It is still much better than nothing, though.)
Peter Dimov wrote:
There are two main reasons, one, because a significant portion of the community doesn't want or need such branching, and two, because the C runtime library is (both de facto and de jure) part of C++. So a "C++ cancelation" that doesn't cancel the C blocking waits would be severely limited in practice, unless the C++ library is revamped to offer pure C++ equivalents to all blocking C functions. (It is still much better than nothing, though.)
Peter - Thank you for taking time to answer what I'm sure was a "silly" question. michael
participants (15)
-
Alexander Terekhov -
Anthony Williams -
Beman Dawes -
David Abrahams -
Howard Hinnant -
Howard Hinnant -
Jeff Garland -
Johan Nilsson -
Michael Caisse -
Paul A Bristow -
Peter Dimov -
Phil Endecott -
Roland Schwarz -
Sebastian Redl -
Yuval Ronen