[move] New version with move-aware containers library

Hi to all, Sorry for the delay, but finally I've found some time to uploaded a new version of Boost.Move: Sandbox: https://svn.boost.org/trac/boost/browser/sandbox/move ZIP file with docs: http://www.drivehq.com/web/igaztanaga/move.zip Online docs: http://www.drivehq.com/web/igaztanaga/libs/move/ Changes: -> Changed folder from boost/move_semantics to boost/move. -> I've extracted all the containers from Boost.Interprocess, put them in boost/container and they only depend on official Boost SVN code (I haven't tested it, but it should be also nice with Boost 1.38). There is no documentation for these containers. -> Fixed a bug with EDG-based compilers. -> Note: I haven't applied yet the patch for non-sfinae compilers. With this changes, I think move emulation is quite portable (at least MSVC, EDG and GCC). Now that containers are independent from Boost.Interprocess, I would like to ask about the next step. If the move library is added, where should we put those containers? I would like to just drop common (vector, list, ...) Boost.Interprocess containers and just maintain these move-aware ones. If move emulation is considered mature enough, we should also plan when could we port all custom move-emulations present in several boost libraries to the "official" move library. Best, Ion

Hi Ion, ----- Original Message ----- From: "Ion Gaztañaga" <igaztanaga@gmail.com> To: <boost@lists.boost.org> Sent: Monday, April 13, 2009 1:21 AM Subject: [boost] [move] New version with move-aware containers library
Hi to all,
Sorry for the delay, but finally I've found some time to uploaded a new version of Boost.Move:
Sandbox:
https://svn.boost.org/trac/boost/browser/sandbox/move
ZIP file with docs:
http://www.drivehq.com/web/igaztanaga/move.zip
Online docs:
http://www.drivehq.com/web/igaztanaga/libs/move/
Changes:
-> Changed folder from boost/move_semantics to boost/move.
-> I've extracted all the containers from Boost.Interprocess, put them in boost/container and they only depend on official Boost SVN code (I haven't tested it, but it should be also nice with Boost 1.38). There is no documentation for these containers.
This is a major feature that Boost needs to have. Excelent!
-> Fixed a bug with EDG-based compilers. -> Note: I haven't applied yet the patch for non-sfinae compilers.
With this changes, I think move emulation is quite portable (at least MSVC, EDG and GCC).
Now that containers are independent from Boost.Interprocess, I would like to ask about the next step. If the move library is added, where should we put those containers? I would like to just drop common (vector, list, ...) Boost.Interprocess containers and just maintain these move-aware ones.
If move emulation is considered mature enough, we should also plan when could we port all custom move-emulations present in several boost libraries to the "official" move library.
I think the first thing to do is purpose the Boost.Move library for review. And then ask for a mini review of the move-aware Boost.Containers. I'm sure you will find quicky a review manager. BTW, how far are the interface of these move-aware containers from the C++0x STL ones? Best, Vicente

vicente.botet wrote:
Hi Ion,
I think the first thing to do is purpose the Boost.Move library for review. And then ask for a mini review of the move-aware Boost.Containers. I'm sure you will find quicky a review manager.
Yes, but I would like to reach a consensus because usually boost reviews require a lot of time and effort and my intention was to provide a very simple and basic support for move semantics. I sincerely don't have time to add more features to the library (I want to work again Interprocess & Intrusive) so that it might be disappointing to many (e.g.: we don't have move-aware algorithms). Any help on this issue (anyone wants to contribute move-aware algorithms?) would be appreciated. An informal discussion between boosters interested in move semantics to reach a consensus about the minimum features of the library would be a good way to focus on those minimum features and introduce as soon as possible move-semantics into boost.
BTW, how far are the interface of these move-aware containers from the C++0x STL ones?
I'm not sure but they should be quite similar. They don't have concepts or initializer lists but I've tried to follow the C++0x draft(e.g. they include placement insertion, resize() no longer require copy-constructible, just default constructible, ...). I would need to review all of them to see how far they are. They also don't include support for scoped allocators (...yet). boost/container currently contains more containers than C++0x: we have the ordered vector flat_xxx family, the good old slist (not forward_list), the shared-memory compatible string and and the move-aware, shared-memory compatible and largely undocumented version of Joaquín's stable_vector. list, set-map are based on Boost.Intrusive. I don't want to push all of them to the official containers library, but I've put them there so that people can play a bit. But the final version would have just the containers boosters consider essential.
Best, Vicente
Best, Ion

Ion Gaztañaga wrote:
BTW, how far are the interface of these move-aware containers from the C++0x STL ones?
I'm not sure but they should be quite similar. They don't have concepts or initializer lists but I've tried to follow the C++0x draft(e.g. they include placement insertion, resize() no longer require copy-constructible, just default constructible, ...). I would need to review all of them to see how far they are. They also don't include support for scoped allocators (...yet).
People are used to the standard containers, so I think it's important to provide that same interface. Feel free to extend it however, or even to differ when there is peculiar reason to. From my personal experience, the find function of sets is fairly limited for certain semi-intrusive scenarios. I seem to recall boost.intrusive allows for a find with any key type, maybe that could be added to your containers too.

----- Original Message ----- From: "Ion Gaztañaga" <igaztanaga@gmail.com> To: <boost@lists.boost.org> Sent: Monday, April 13, 2009 1:21 AM Subject: [boost] [move] New version with move-aware containers library
-> I've extracted all the containers from Boost.Interprocess, put them in boost/container and they only depend on official Boost SVN code (I haven't tested it, but it should be also nice with Boost 1.38). There is no documentation for these containers.
Hi, I don't find boost/container, please could you give the address where you have put boost/container? Vicente

Forget this post, I have found it in move/boost/container Vicente ----- Original Message ----- From: "vicente.botet" <vicente.botet@wanadoo.fr> To: <boost@lists.boost.org> Sent: Monday, April 13, 2009 12:23 PM Subject: Re: [boost] [move] New version with move-aware containers library ----- Original Message ----- From: "Ion Gaztañaga" <igaztanaga@gmail.com> To: <boost@lists.boost.org> Sent: Monday, April 13, 2009 1:21 AM Subject: [boost] [move] New version with move-aware containers library
-> I've extracted all the containers from Boost.Interprocess, put them in boost/container and they only depend on official Boost SVN code (I haven't tested it, but it should be also nice with Boost 1.38). There is no documentation for these containers.
Hi, I don't find boost/container, please could you give the address where you have put boost/container? Vicente _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

vicente.botet wrote:
Hi,
I don't find boost/container, please could you give the address where you have put boost/container?
https://svn.boost.org/trac/boost/browser/sandbox/move/boost and in the zip file Best, Ion

2009/4/13 Ion Gaztañaga <igaztanaga@gmail.com>:
-> I've extracted all the containers from Boost.Interprocess, put them in boost/container and they only depend on official Boost SVN code (I haven't tested it, but it should be also nice with Boost 1.38). There is no documentation for these containers.
I'm trying to convert the unordered containers, is it okay it I add them to your library in the sandbox? I'm getting an error for one of my tests, but that might be at my end. A small note on your tutorial. In the first example of using the library should mention the header to be included, probably just by putting '#include <boost/move/move.hpp>' at the beginning of the example. And it might help if you add 'public:' somewhere before 'BOOST_ENABLE_MOVE_EMULATION(clone_ptr)' to make it clear that it isn't private. Daniel

Daniel James wrote:
I'm trying to convert the unordered containers, is it okay it I add them to your library in the sandbox? I'm getting an error for one of my tests, but that might be at my end.
That would be nice.
A small note on your tutorial. In the first example of using the library should mention the header to be included, probably just by putting '#include <boost/move/move.hpp>' at the beginning of the example. And it might help if you add 'public:' somewhere before 'BOOST_ENABLE_MOVE_EMULATION(clone_ptr)' to make it clear that it isn't private.
Ok. Thanks.
Daniel
Best, Ion

on Sun Apr 12 2009, Ion Gaztañaga <igaztanaga-AT-gmail.com> wrote:
Hi to all,
Sorry for the delay, but finally I've found some time to uploaded a new version of Boost.Move:
Sandbox:
https://svn.boost.org/trac/boost/browser/sandbox/move
ZIP file with docs:
http://www.drivehq.com/web/igaztanaga/move.zip
Online docs:
Nice! Here's one thing I don't love: Derived(BOOST_RV_REF(Derived) x) // Move ctor : Base(boost::move(static_cast<Base&>(x))), mem_(boost::move(x.mem_)) { } is there a way to get rid of the static_cast and make Derived(BOOST_RV_REF(Derived) x) // Move ctor : Base(boost::move(x)), mem_(boost::move(x.mem_)) { } work?
Changes:
-> Changed folder from boost/move_semantics to boost/move. -> I've extracted all the containers from Boost.Interprocess, put them in boost/container and they only depend on official Boost SVN code (I haven't tested it, but it should be also nice with Boost 1.38). There is no documentation for these containers.
w00t!
-> Fixed a bug with EDG-based compilers. -> Note: I haven't applied yet the patch for non-sfinae compilers.
With this changes, I think move emulation is quite portable (at least MSVC, EDG and GCC).
Super!
Now that containers are independent from Boost.Interprocess, I would like to ask about the next step. If the move library is added, where should we put those containers? I would like to just drop common (vector, list, ...) Boost.Interprocess containers and just maintain these move-aware ones.
If these were originally just implementation details of another library, then they would go in a detail/ directory until they are reviewed and accepted. On the other hand, since *these* are simply evolutionary modifications of the containers in Interprocess, which /were/ reviewed and accepted. So this really becomes a question of how to go about breaking an existing, accepted library into multiple libraries. And I don't really know the answer to that one. I'm open to suggestions.
If move emulation is considered mature enough, we should also plan when could we port all custom move-emulations present in several boost libraries to the "official" move library.
Yep. -- Dave Abrahams BoostPro Computing http://www.boostpro.com

AMDG David Abrahams wrote:
Here's one thing I don't love:
Derived(BOOST_RV_REF(Derived) x) // Move ctor : Base(boost::move(static_cast<Base&>(x))), mem_(boost::move(x.mem_)) { }
is there a way to get rid of the static_cast and make
Derived(BOOST_RV_REF(Derived) x) // Move ctor : Base(boost::move(x)), mem_(boost::move(x.mem_)) { }
work?
FWIW, I always write copy constructors using the cast, after having been bitten by templated base class constructors a few times. In Christ, Steven Watanabe

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday 15 April 2009, Steven Watanabe wrote:
AMDG
David Abrahams wrote:
Here's one thing I don't love:
Derived(BOOST_RV_REF(Derived) x) // Move ctor
: Base(boost::move(static_cast<Base&>(x))),
mem_(boost::move(x.mem_)) { }
is there a way to get rid of the static_cast and make
Derived(BOOST_RV_REF(Derived) x) // Move ctor
: Base(boost::move(x)),
mem_(boost::move(x.mem_)) { }
work?
FWIW, I always write copy constructors using the cast, after having been bitten by templated base class constructors a few times.
What about a boost::forward<Base>(x) instead of boost::move(static_cast<Base&>(x)) Isn't forward supposed to be some kind of safe and movable-friendly static_cast? I admit I'm a bit hazy on exactly what std::forward is supposed to accomplish. I see it used in perfect forwarding examples, but they seem to work just as well without it. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAknoei4ACgkQ5vihyNWuA4X4MwCgloI4NsYcHsS47CP3rkPaZq8D FtcAoLtj0KvXdc+AvSkZQRqB3N86Gh34 =YPu0 -----END PGP SIGNATURE-----

on Wed Apr 15 2009, Steven Watanabe <watanabesj-AT-gmail.com> wrote:
AMDG
David Abrahams wrote:
Here's one thing I don't love:
Derived(BOOST_RV_REF(Derived) x) // Move ctor : Base(boost::move(static_cast<Base&>(x))), mem_(boost::move(x.mem_)) { }
is there a way to get rid of the static_cast and make
Derived(BOOST_RV_REF(Derived) x) // Move ctor : Base(boost::move(x)), mem_(boost::move(x.mem_)) { }
work?
FWIW, I always write copy constructors using the cast, after having been bitten by templated base class constructors a few times.
Hmm, good point. -- Dave Abrahams BoostPro Computing http://www.boostpro.com

David Abrahams wrote:
Here's one thing I don't love:
Derived(BOOST_RV_REF(Derived) x) // Move ctor : Base(boost::move(static_cast<Base&>(x))), mem_(boost::move(x.mem_)) { }
is there a way to get rid of the static_cast and make
Derived(BOOST_RV_REF(Derived) x) // Move ctor : Base(boost::move(x)), mem_(boost::move(x.mem_)) { }
work?
I tried but the second form ends calling the copy constructor because rv<Derived> & is upcasted to Derived & and then to Base &. Maybe we could enable some additional conversion operator to a base class.
If these were originally just implementation details of another library, then they would go in a detail/ directory until they are reviewed and accepted. On the other hand, since *these* are simply evolutionary modifications of the containers in Interprocess, which /were/ reviewed and accepted.
So this really becomes a question of how to go about breaking an existing, accepted library into multiple libraries. And I don't really know the answer to that one. I'm open to suggestions.
So am I ;-)
If move emulation is considered mature enough, we should also plan when could we port all custom move-emulations present in several boost libraries to the "official" move library.
Yep.
I think porting should be quite fast. One option is to set a Boost release as a milestone (Boost 1.40, Boost 1.41?) and fill a bug for every library with move semantics. Best, Ion

on Thu Apr 16 2009, Ion Gaztañaga <igaztanaga-AT-gmail.com> wrote:
So this really becomes a question of how to go about breaking an existing, accepted library into multiple libraries. And I don't really know the answer to that one. I'm open to suggestions.
So am I ;-)
I *think* this change can be done by consensus. Maybe we should just see if there are any reasonable objections to that path. If there are, we could fall back to a mini-review. -- Dave Abrahams BoostPro Computing http://www.boostpro.com

on Sun Apr 12 2009, Ion Gaztañaga <igaztanaga-AT-gmail.com> wrote:
Hi to all,
Sorry for the delay, but finally I've found some time to uploaded a new version of Boost.Move:
Sandbox:
https://svn.boost.org/trac/boost/browser/sandbox/move
ZIP file with docs:
http://www.drivehq.com/web/igaztanaga/move.zip
Online docs:
Hi Ion, are you familiar with http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2009/n2855.html ? Until the committee decides exactly how to handle it and compilers catch up with the feature they add, we need a workaround for that problem, both for C++03 and C++0x. The simplest approach requires a trait called has_nothrow_move<T> which can be used to SFINAE-out pair's move constructor unless its members all have a nonthrowing move constructor. It would be reasonable to say template <class T> struct has_nothrow_move : has_move_constructor {}; as a default, since move constructors shouldn't throw. But still, we need an implementation of has_move_constructor. Obviously, that will require compiler support to be optimal, but in the meantime it can be specialized. Now, the rub: I'd like to have has_nothrow_move in place in the Boost SVN by the time BoostCon starts (Monday), so we can use it for http://www.boostcon.com/program/sessions#abrahams-boost0x-1-hands-on-rvalue-... What do you think; can we get this integrated with your library and checked in? I think we'll all be doing our BoostCon work on a branch anyway, so it's reasonable to consider just checking the move constructor stuff in on a branch of the Boost trunk. Thanks, -- Dave Abrahams BoostPro Computing http://www.boostpro.com

David Abrahams wrote:
Hi Ion,
are you familiar with http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2009/n2855.html ?
Until the committee decides exactly how to handle it and compilers catch up with the feature they add, we need a workaround for that problem, both for C++03 and C++0x. The simplest approach requires a trait called has_nothrow_move<T> which can be used to SFINAE-out pair's move constructor unless its members all have a nonthrowing move constructor. It would be reasonable to say
template <class T> struct has_nothrow_move : has_move_constructor {};
as a default, since move constructors shouldn't throw. But still, we need an implementation of has_move_constructor. Obviously, that will require compiler support to be optimal, but in the meantime it can be specialized.
What if we say template <class T> struct move_is_nothrow : mpl::true_type {}; and have it defined whether there actually is a move constructor or not? Would that work? Or would it interact badly with copy-only types? Sebastian

on Thu Apr 30 2009, Sebastian Redl <sebastian.redl-AT-getdesigned.at> wrote:
David Abrahams wrote:
Hi Ion,
are you familiar with http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2009/n2855.html ?
Until the committee decides exactly how to handle it and compilers catch up with the feature they add, we need a workaround for that problem, both for C++03 and C++0x. The simplest approach requires a trait called has_nothrow_move<T> which can be used to SFINAE-out pair's move constructor unless its members all have a nonthrowing move constructor. It would be reasonable to say
template <class T> struct has_nothrow_move : has_move_constructor {};
as a default, since move constructors shouldn't throw. But still, we need an implementation of has_move_constructor. Obviously, that will require compiler support to be optimal, but in the meantime it can be specialized.
What if we say
template <class T> struct move_is_nothrow : mpl::true_type {};
and have it defined whether there actually is a move constructor or not? Would that work? Or would it interact badly with copy-only types?
The latter. We can probably count on users not to write throwing move ctors, but we can't count on nonthrowing copy ctors. It needs to be conservative, i.e. something like: template <class T> struct has_nothrow_move : mpl::or_< has_nothrow_copy<T> // handles all builtins , has_move_constructor<T> // handle UDTs > {}; template <class T> struct has_move_constructor // Must assume UDTs don't have a move ctor : mpl::false_ {}; template <class T> struct has_nothrow_copy // Must assume UDTs have a copy ctor : has_trivial_copy<T> {}; otherwise, pair<movable_type, type_with_throwing_copy_and_no_move_ctor> has a throwing move constructor. -- Dave Abrahams BoostPro Computing http://www.boostpro.com

David Abrahams wrote:
are you familiar with http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2009/n2855.html ?
I've just skimmed part of the discussion, just to see what the problem was and to think that noexcept is a great idea that can also help performance and code size.
Until the committee decides exactly how to handle it and compilers catch up with the feature they add, we need a workaround for that problem, both for C++03 and C++0x. The simplest approach requires a trait called has_nothrow_move<T> which can be used to SFINAE-out pair's move constructor unless its members all have a nonthrowing move constructor. It would be reasonable to say
template <class T> struct has_nothrow_move : has_move_constructor {};
What does has_nothrow_move exactly mean? It means that type has a nothrow T(T &&x) overload or that conversion from rvalue ref to a type does not throw (including copy constructor if the type has not overloaded the constructor with both const & and &&)? In the working draft there are some "move" traits that are confusing to me regarding this issue (MoveConstructible requires RvalueOf<T> && Constructible<T, RvalueOf<T>::type> which I think is true for C++03 types). I think has_move_constructor could be just "is_movable" (in the emulation library "is_movable" means "this type has activated move emulation", the T(rv<T> &) overload). For compilers with rvalue references, I'm not a language expert so I need someone that could write such a trait (can we easily detect if a type has the T(T&&t) constructor?). For compilers with rvalue reference, if all types in the course are going to use the portable syntax (that is, they are going to define BOOST_HAS_MOVE_EMULATION macro) BOOST_HAS_MOVE_EMULATION can be used to put a typedef that will mark those types as having a move constructor. Do you also want me to tweak container/detail/pair.hpp implementation and apply SFINAE for move constructors? I can do some work tomorrow (the trait), and commit it to sandbox. Best, Ion

on Thu Apr 30 2009, Ion Gaztañaga <igaztanaga-AT-gmail.com> wrote:
David Abrahams wrote:
are you familiar with http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2009/n2855.html ?
I've just skimmed part of the discussion, just to see what the problem was and to think that noexcept is a great idea that can also help performance and code size.
Until the committee decides exactly how to handle it and compilers catch up with the feature they add, we need a workaround for that problem, both for C++03 and C++0x. The simplest approach requires a trait called has_nothrow_move<T> which can be used to SFINAE-out pair's move constructor unless its members all have a nonthrowing move constructor. It would be reasonable to say
template <class T> struct has_nothrow_move : has_move_constructor {};
What does has_nothrow_move exactly mean? It means that type has a nothrow T(T &&x) overload or that conversion from rvalue ref to a type does not throw (including copy constructor if the type has not overloaded the constructor with both const & and &&)?
It means that construction from an rvalue of the same type doesn't throw.
In the working draft there are some "move" traits that are confusing to me regarding this issue (MoveConstructible requires RvalueOf<T> && Constructible<T, RvalueOf<T>::type> which I think is true for C++03 types).
Yes, anything that is CopyConstructible is MoveConstructible. It's not great terminology, but you have to think of the T(move(x)) (where x is of type T) as "the move operation" regardless of whether it actually moves or copies. Then MoveConstructible means you can perform the move operation, not (necessarily) that T has a move constructor.
I think has_move_constructor could be just "is_movable" (in the emulation library "is_movable" means "this type has activated move emulation", the T(rv<T> &) overload).
I expected something like that to be the case for emulated move.
For compilers with rvalue references, I'm not a language expert so I need someone that could write such a trait (can we easily detect if a type has the T(T&&t) constructor?).
There's no known way to do it (even with C++0x as currently defined). Unless someone comes up with an amazing hack, we need a new language feature.
For compilers with rvalue reference, if all types in the course are going to use the portable syntax (that is, they are going to define BOOST_HAS_MOVE_EMULATION macro) BOOST_HAS_MOVE_EMULATION can be used to put a typedef that will mark those types as having a move constructor.
Yep, that sounds like a great idea. And, failing that, the trait can be specialized.
Do you also want me to tweak container/detail/pair.hpp implementation and apply SFINAE for move constructors?
If possible. I suspect it applies not just to pair, but a bunch of others, too. Consider containers, which contain allocators.
I can do some work tomorrow (the trait), and commit it to sandbox.
Anything you can do would be much appreciated! -- Dave Abrahams BoostPro Computing http://www.boostpro.com

I've committed to sandbox/move changes to add has_nothrow_move and some tweaks to containers (revision 52699). I haven't touched pair because I have no time to test anythying from now to the end of the next week, but I will try to keep an eye on this issue. I need time to see implications on allocators, which are copied (for non scoped allocators) on move construction and that would require no-throw guarantee to make the container nothrowmovable. I hope you find changes useful for BoostCon. Best, Ion
participants (8)
-
Daniel James
-
David Abrahams
-
Frank Mori Hess
-
Ion Gaztañaga
-
Mathias Gaunard
-
Sebastian Redl
-
Steven Watanabe
-
vicente.botet