Future of Boost.Thread, date-time, and C++0x
At the Kona meeting (just finished) the standard committee made very significant progress on threading for C++0x -- voting in a proposal written by Howard H, Beman, Anthony Williams, myself, and others -- subject to wording changes to be made by a small drafting committee. Basically we were in agreement on semantics, but even after several late night drafting sessions people weren't happy with the standardeze. While much of the proposal is based on Boost.thread it has some serious departures in the way mutexes and guard interfaces work, it's use of date-time types for clock/timing interfaces and such. At this point I feel it would be very important to have a reference implementation widely available to Boosters and others to play with so that any issues are shaken out while there's still time tweak things. Howard has been maintaining a reference implementation, but this is a call out to the broader thread development team (not sure who all that is now besides Anthony) to consider how we might formalize this into Boost.threads. Thoughts? Jeff
Jeff Garland <jeff@crystalclearsoftware.com> writes:
At the Kona meeting (just finished) the standard committee made very significant progress on threading for C++0x -- voting in a proposal written by Howard H, Beman, Anthony Williams, myself, and others -- subject to wording changes to be made by a small drafting committee. Basically we were in agreement on semantics, but even after several late night drafting sessions people weren't happy with the standardeze.
That's unfortunate.
While much of the proposal is based on Boost.thread it has some serious departures in the way mutexes and guard interfaces work, it's use of date-time types for clock/timing interfaces and such. At this point I feel it would be very important to have a reference implementation widely available to Boosters and others to play with so that any issues are shaken out while there's still time tweak things. Howard has been maintaining a reference implementation, but this is a call out to the broader thread development team (not sure who all that is now besides Anthony) to consider how we might formalize this into Boost.threads.
Thoughts?
The boost thread team is pretty much just me at the moment, though there's a reasonable level of interest from others (such as Peter Dimov). I'm trying to update trunk to be have more of the C++0x interface at the moment. I've already integrated call_once (though I forgot only MSVC has _ReadWriteBarrier, and I haven't done the variadic arguments yet), and I'm currently working on the mutexes. I'm keen to get everything updated as much as possible before the 1.35 cutoff. I've got a reasonably up-to-date Windows implementation to use as a basis, and I've been working through the POSIX stuff. Also, even though it's been punted to TR2, I want to get the read-write mutex back in --- I have a new implementation, and it's been missing since 1.32, so it would be good to get it back in the relese. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
On Oct 8, 2007, at 10:42 AM, Anthony Williams wrote:
Jeff Garland <jeff@crystalclearsoftware.com> writes:
At the Kona meeting (just finished) the standard committee made very significant progress on threading for C++0x -- voting in a proposal written by Howard H, Beman, Anthony Williams, myself, and others -- subject to wording changes to be made by a small drafting committee. Basically we were in agreement on semantics, but even after several late night drafting sessions people weren't happy with the standardeze.
That's unfortunate.
While much of the proposal is based on Boost.thread it has some serious departures in the way mutexes and guard interfaces work, it's use of date-time types for clock/timing interfaces and such. At this point I feel it would be very important to have a reference implementation widely available to Boosters and others to play with so that any issues are shaken out while there's still time tweak things. Howard has been maintaining a reference implementation, but this is a call out to the broader thread development team (not sure who all that is now besides Anthony) to consider how we might formalize this into Boost.threads.
Thoughts?
I can help with a posix implementation. I don't have windows to develop on though.
The boost thread team is pretty much just me at the moment, though there's a reasonable level of interest from others (such as Peter Dimov).
I'm trying to update trunk to be have more of the C++0x interface at the moment. I've already integrated call_once (though I forgot only MSVC has _ReadWriteBarrier, and I haven't done the variadic arguments yet), and I'm currently working on the mutexes. I'm keen to get everything updated as much as possible before the 1.35 cutoff. I've got a reasonably up-to-date Windows implementation to use as a basis, and I've been working through the POSIX stuff.
Also, even though it's been punted to TR2, I want to get the read- write mutex back in --- I have a new implementation, and it's been missing since 1.32, so it would be good to get it back in the relese.
I can help on posix here too. I'm hoping it will look something like: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2406.html#shared_mu... or at least take some of those design decisions into consideration. The downside is that the above really needs rvalue ref for the upgrade- stuff. Possibilities include just not doing the upgrade stuff if rvalue ref is missing, or simulating move along the lines of what Dave has done. The standard containers are an integral part of the proposed std::thread package. Without move support in the standard containers useful structures such as vector<thread> are going to be missing. The lack of thread factory functions will also be unfortunate until compilers (such as gcc 4.3, boostcon and CodeWarrior 10) support returning move-only types from functions. -Howard
Howard Hinnant wrote:
On Oct 8, 2007, at 10:42 AM, Anthony Williams wrote:
The standard containers are an integral part of the proposed std::thread package. Without move support in the standard containers useful structures such as vector<thread> are going to be missing. The lack of thread factory functions will also be unfortunate until compilers (such as gcc 4.3, boostcon and CodeWarrior 10) support returning move-only types from functions.
That's a good point. FWIW, OSL is running regressions with gcc4_3 in 0x mode so we could take advantage of whatever's in there for now. I assume they will be willing to upgrade as new features become available. Jeff
Howard Hinnant <hinnant@twcny.rr.com> writes:
On Oct 8, 2007, at 10:42 AM, Anthony Williams wrote:
I can help with a posix implementation. I don't have windows to develop on though.
Excellent. I've got both available (Windows + kubuntu), but it's certainly easier to just stick to one platform.
Also, even though it's been punted to TR2, I want to get the read- write mutex back in --- I have a new implementation, and it's been missing since 1.32, so it would be good to get it back in the relese.
I can help on posix here too. I'm hoping it will look something like:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2406.html#shared_mu...
That looks like my prototypes, other than the name.
or at least take some of those design decisions into consideration. The downside is that the above really needs rvalue ref for the upgrade- stuff. Possibilities include just not doing the upgrade stuff if rvalue ref is missing, or simulating move along the lines of what Dave has done.
I've got a simulated move that I've been using.
The standard containers are an integral part of the proposed std::thread package. Without move support in the standard containers useful structures such as vector<thread> are going to be missing. The lack of thread factory functions will also be unfortunate until compilers (such as gcc 4.3, boostcon and CodeWarrior 10) support returning move-only types from functions.
Simulated move can handle thread factory functions, but you can't get vector<thread> without a suitable standard library. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams wrote:
Jeff Garland <jeff@crystalclearsoftware.com> writes:
At the Kona meeting (just finished) the standard committee made very significant progress on threading for C++0x -- voting in a proposal written by Howard H, Beman, Anthony Williams, myself, and others -- subject to wording changes to be made by a small drafting committee. Basically we were in agreement on semantics, but even after several late night drafting sessions people weren't happy with the standardeze.
That's unfortunate.
Not really -- it's just hard. I'd rather have it correct and consistent with the rest of the standard. Point is that unless something goes horribly off-track during the refinements the wording will be in the standard prior to the Feb 08 meeting in Bellevue.
While much of the proposal is based on Boost.thread it has some serious departures in the way mutexes and guard interfaces work, it's use of date-time types for clock/timing interfaces and such. At this point I feel it would be very important to have a reference implementation widely available to Boosters and others to play with so that any issues are shaken out while there's still time tweak things. Howard has been maintaining a reference implementation, but this is a call out to the broader thread development team (not sure who all that is now besides Anthony) to consider how we might formalize this into Boost.threads.
Thoughts?
The boost thread team is pretty much just me at the moment, though there's a reasonable level of interest from others (such as Peter Dimov).
I'm trying to update trunk to be have more of the C++0x interface at the moment. I've already integrated call_once (though I forgot only MSVC has _ReadWriteBarrier, and I haven't done the variadic arguments yet), and I'm currently working on the mutexes. I'm keen to get everything updated as much as possible before the 1.35 cutoff. I've got a reasonably up-to-date Windows implementation to use as a basis, and I've been working through the POSIX stuff.
Also, even though it's been punted to TR2, I want to get the read-write mutex back in --- I have a new implementation, and it's been missing since 1.32, so it would be good to get it back in the relese.
Ok, that sounds good, although are you going to maintain a 'classic' interface for existing users? Given the changes in the interface going straight from the current boost.thread to 0x interfaces would require some client rework. One thought would be to put the new stuff in namespace c++0x or std? I have a draft implementation of the date-time types you need -- I think Howard has been distributing that as well -- it needs some update because we made some name changes. I don't know that I want to incorporate directly into Boost date-time just yet, so maybe we want to put it in thread for now? Anyway, I'll contact you off-list and so we can figure out the details. One other thing I forgot about was the atomics stuff -- that was voted in as is. Is anyone working on a Boost implementation of that stuff? I know there is some of this floating around in shared ptr or something. Jeff
Jeff Garland <jeff@crystalclearsoftware.com> writes:
Anthony Williams wrote:
I'm trying to update trunk to be have more of the C++0x interface at the moment. I've already integrated call_once (though I forgot only MSVC has _ReadWriteBarrier, and I haven't done the variadic arguments yet), and I'm currently working on the mutexes. I'm keen to get everything updated as much as possible before the 1.35 cutoff. I've got a reasonably up-to-date Windows implementation to use as a basis, and I've been working through the POSIX stuff.
Also, even though it's been punted to TR2, I want to get the read-write mutex back in --- I have a new implementation, and it's been missing since 1.32, so it would be good to get it back in the relese.
Ok, that sounds good, although are you going to maintain a 'classic' interface for existing users? Given the changes in the interface going straight from the current boost.thread to 0x interfaces would require some client rework. One thought would be to put the new stuff in namespace c++0x or std?
Well, I've left the boost::mutex::scoped_lock typedefs in for now, even though they're just typedefs to boost::unique_lock<boost::mutex>. The constructor parameters have changed in line with the C++0X proposal, though. I've just added a backwards-compatibility overload for call_once (the order of args has changed).
I have a draft implementation of the date-time types you need -- I think Howard has been distributing that as well -- it needs some update because we made some name changes. I don't know that I want to incorporate directly into Boost date-time just yet, so maybe we want to put it in thread for now? Anyway, I'll contact you off-list and so we can figure out the details.
boost/thread/thread_time.hpp contains a minimal date-time interface for thread (based on boost::datetime). This seems to be sufficient --- is it not?
One other thing I forgot about was the atomics stuff -- that was voted in as is. Is anyone working on a Boost implementation of that stuff? I know there is some of this floating around in shared ptr or something.
I have some partial implementations --- I was waiting to see what the final outcome was before trying to keep up to date. I know Peter Dimov had a more complete implementation, but I don't know whether it's up to date. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Jeff Garland wrote:
At the Kona meeting (just finished) the standard committee made very significant progress on threading for C++0x -- voting in a proposal written by Howard H, Beman, Anthony Williams, myself, and others -- subject to wording changes to be made by a small drafting committee. Basically we were in agreement on semantics, but even after several late night drafting sessions people weren't happy with the standardeze.
While much of the proposal is based on Boost.thread it has some serious departures in the way mutexes and guard interfaces work, it's use of date-time types for clock/timing interfaces and such.
This reminds me: I thought I posted something about this when Howard sent his proposal to this list, but it seems to have been lost in an email black-hole. Howard's code included timed operations that are templated on the time type, which (presumably) allows an implementation to provide versions for each of that platform's standard time types. This reminded me of some presentations about power saving at LinuxConf.EU: apparently reducing the number of timer interrupts is greatly beneficial to increasing energy efficiency, and one way to achieve this is to align all applications' periodic wakeups. Very often, we want to timeout after "about a second" but will actually specify a timeout value of "exactly 1.0000000 seconds"; if the API allowed us to indicate, for example, "between 0.1 and 1.5 seconds", or "1 second +/- 10%", then some hidden power-saving wakeup-alignment algorithm could be applied. (Equally, you could imagine situations where it would be beneficial to spread out wakeups, rather than grouping them.) So does the proposal allow this sort of thing to be done, by supplying a "time range" type? There's also the question of whether the implementation style of a mutex and/or its locks can somehow be chosen, e.g. between a spin-lock (best when the probability of contention is very low) and an OS-supported mutex (best when the probability of contention is greater). Having substantially improved the performance of shared_ptr on ARM-linux by moving from pthreads to an atomic-asm implementation, I'm considering doing the same to Boost.Threads on that platform. However, the distinction between locks around very short critical sections (e.g. a pair of variables that must be updated atomically) and longer-duration locks needs to be made somehow. Any thoughts? Regards, Phil.
On Oct 8, 2007, at 12:51 PM, Phil Endecott wrote:
I thought I posted something about this when Howard sent his proposal to this list, but it seems to have been lost in an email black-hole. Howard's code included timed operations that are templated on the time type, which (presumably) allows an implementation to provide versions for each of that platform's standard time types.
Fwiw, the bits that were approved in Kona have templated timed functions. That document isn't public yet, but will be in about two weeks. The templated functions are still missing template requirements wording, at least in a few places, but that will be fixed up by Bellevue. The shared/upgrade mutex parts were not reviewed in Kona, nor considered for C++0X. Still targeting TR2 with those, and TR2 still has a backseat priority with respect to C++0X.
There's also the question of whether the implementation style of a mutex and/or its locks can somehow be chosen, e.g. between a spin-lock (best when the probability of contention is very low) and an OS-supported mutex (best when the probability of contention is greater). Having substantially improved the performance of shared_ptr on ARM-linux by moving from pthreads to an atomic-asm implementation, I'm considering doing the same to Boost.Threads on that platform. However, the distinction between locks around very short critical sections (e.g. a pair of variables that must be updated atomically) and longer-duration locks needs to be made somehow.
The only suggestion I have here, from a standards point of view, is for the standard to anticipate and support user-defined mutexes/locks in the easiest way possible so that libraries such as boost can easily build upon the standard foundation. This philosophical foundation is in the standard-proposed document. Whether or not we've actually achieved this goal is a separate question. ;-) -Howard
Howard Hinnant wrote:
The only suggestion I have here, from a standards point of view, is for the standard to anticipate and support user-defined mutexes/locks in the easiest way possible so that libraries such as boost can easily build upon the standard foundation. This philosophical foundation is in the standard-proposed document. Whether or not we've actually achieved this goal is a separate question. ;-)
Is there a summary somewhere regarding the extensibility features of the threading library? In particular, is this threading library you're standardizing going to create threading objects that hide their thread handle such that they can't be extended to work with custom multiplexers? [I am horribly concerned by the ever-increasing trend to write concrete-style components that support only least-common-denominator system features, with no possibly way to extend them due to inability to 'get at' some inner handle. Boost filesystem, Boost thread, iostream all do this, making it impossible for even a completely normal vanilla single-threaded GUI program to be able to safely use them (due to the fact they might block, and there's no way to multiplex them with the GUI, despite the fact the OS would be able to do this). The net effect is that these style of components are mostly useless to most modern single-threaded applications writers. I am hoping this threading library you are working on is something I will actually be able to use, and not just something made for terminal-based POSIX systems.]
On Oct 10, 2007, at 7:23 PM, Aaron W. LaFramboise wrote:
Howard Hinnant wrote:
The only suggestion I have here, from a standards point of view, is for the standard to anticipate and support user-defined mutexes/locks in the easiest way possible so that libraries such as boost can easily build upon the standard foundation. This philosophical foundation is in the standard-proposed document. Whether or not we've actually achieved this goal is a separate question. ;-)
Is there a summary somewhere regarding the extensibility features of the threading library?
In particular, is this threading library you're standardizing going to create threading objects that hide their thread handle such that they can't be extended to work with custom multiplexers?
[I am horribly concerned by the ever-increasing trend to write concrete-style components that support only least-common-denominator system features, with no possibly way to extend them due to inability to 'get at' some inner handle. Boost filesystem, Boost thread, iostream all do this, making it impossible for even a completely normal vanilla single-threaded GUI program to be able to safely use them (due to the fact they might block, and there's no way to multiplex them with the GUI, despite the fact the OS would be able to do this).
The net effect is that these style of components are mostly useless to most modern single-threaded applications writers. I am hoping this threading library you are working on is something I will actually be able to use, and not just something made for terminal-based POSIX systems.]
I hope you'll be able to use it too. The current wording has: class thread { ... native_handle_type native_handle(); }; which I believe may address your concerns (not 100% positive). On a POSIX system this might return a pthread_t. The idea is to give you sufficient rope (hang or save, your choice). That being said, I seem to remember some concerns over this feature and can not at this point firmly establish its presence in the final draft (just too many balls in the air). -Howard
on Mon Oct 08 2007, "Phil Endecott" <spam_from_boost_dev-AT-chezphil.org> wrote:
Jeff Garland wrote:
At the Kona meeting (just finished) the standard committee made very significant progress on threading for C++0x -- voting in a proposal written by Howard H, Beman, Anthony Williams, myself, and others -- subject to wording changes to be made by a small drafting committee. Basically we were in agreement on semantics, but even after several late night drafting sessions people weren't happy with the standardeze.
While much of the proposal is based on Boost.thread it has some serious departures in the way mutexes and guard interfaces work, it's use of date-time types for clock/timing interfaces and such.
This reminds me: I thought I posted something about this when Howard sent his proposal to this list, but it seems to have been lost in an email black-hole. Howard's code included timed operations that are templated on the time type, which (presumably) allows an implementation to provide versions for each of that platform's standard time types. This reminded me of some presentations about power saving at LinuxConf.EU: apparently reducing the number of timer interrupts is greatly beneficial to increasing energy efficiency, and one way to achieve this is to align all applications' periodic wakeups. Very often, we want to timeout after "about a second" but will actually specify a timeout value of "exactly 1.0000000 seconds"; if the API allowed us to indicate, for example, "between 0.1 and 1.5 seconds", or "1 second +/- 10%", then some hidden power-saving wakeup-alignment algorithm could be applied.
Agreed, this is hugely important. However, I think the standardese may be easier than that: unless we think some programs need both exact and flexible timeouts, we can leave it up to the implementation to interpret the timeout value. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
Jeff Garland wrote:
At the Kona meeting (just finished) the standard committee made very significant progress on threading for C++0x -- voting in a proposal written by Howard H, Beman, Anthony Williams, myself, and others -- subject to wording changes to be made by a small drafting committee. Basically we were in agreement on semantics, but even after several late night drafting sessions people weren't happy with the standardeze.
Will you please post here when a document describing what has been agreed is available (not the standardize, something I can understand :) )?
Yuval Ronen wrote:
Jeff Garland wrote:
At the Kona meeting (just finished) the standard committee made very significant progress on threading for C++0x -- voting in a proposal written by Howard H, Beman, Anthony Williams, myself, and others -- subject to wording changes to be made by a small drafting committee. Basically we were in agreement on semantics, but even after several late night drafting sessions people weren't happy with the standardeze.
Will you please post here when a document describing what has been agreed is available (not the standardize, something I can understand :) )?
Everybody involved in this is working really hard to meet post-meeting deadlines. You probably won't see much until the post-meeting mailing appears about two weeks from now, and maybe not even then. This has been a death-march, particularly for Howard, so please be patient... Thanks, --Beman
On Oct 10, 2007, at 12:04 PM, Beman Dawes wrote:
Yuval Ronen wrote:
Jeff Garland wrote:
At the Kona meeting (just finished) the standard committee made very significant progress on threading for C++0x -- voting in a proposal written by Howard H, Beman, Anthony Williams, myself, and others -- subject to wording changes to be made by a small drafting committee. Basically we were in agreement on semantics, but even after several late night drafting sessions people weren't happy with the standardeze.
Will you please post here when a document describing what has been agreed is available (not the standardize, something I can understand :) )?
Everybody involved in this is working really hard to meet post-meeting deadlines. You probably won't see much until the post-meeting mailing appears about two weeks from now, and maybe not even then. This has been a death-march, particularly for Howard, so please be patient...
No, no! Not dead yet, <gasp> still breathing... <gasp> ;-) Beman's right. But we're hanging in there and will endeavor to keep everyone informed. We know this is an important issue to lots of people. And thanks for your continued interest. -Howard
Howard Hinnant wrote:
On Oct 10, 2007, at 12:04 PM, Beman Dawes wrote:
Yuval Ronen wrote:
Jeff Garland wrote:
At the Kona meeting (just finished) the standard committee made very significant progress on threading for C++0x -- voting in a proposal written by Howard H, Beman, Anthony Williams, myself, and others -- subject to wording changes to be made by a small drafting committee. Basically we were in agreement on semantics, but even after several late night drafting sessions people weren't happy with the standardeze. Will you please post here when a document describing what has been agreed is available (not the standardize, something I can understand :) )? Everybody involved in this is working really hard to meet post-meeting deadlines. You probably won't see much until the post-meeting mailing appears about two weeks from now, and maybe not even then. This has been a death-march, particularly for Howard, so please be patient...
No, no! Not dead yet, <gasp> still breathing... <gasp> ;-)
Beman's right. But we're hanging in there and will endeavor to keep everyone informed. We know this is an important issue to lots of people. And thanks for your continued interest.
Thank *you* for the enormous effort you have put in this, for the benefit of all of us.
participants (8)
-
Aaron W. LaFramboise -
Anthony Williams -
Beman Dawes -
David Abrahams -
Howard Hinnant -
Jeff Garland -
Phil Endecott -
Yuval Ronen