[move] new rvalue reference emulation
I've recently discovered (well, probably rediscovered) an alternate rvalue reference emulation, one which allows rvalues ***of any arbitrary (move-emulation-enabled) type*** to be captured in C++03 and identified as rvalues, rather than references-to-const (there is a drawback, though, which I'll get to below; as usual, there's no "free lunch"). This is in contrast to the current move emulation framework Ion has created in Boost.Move which, at best, allows you to capture rvalues of specific types. That is, if you can enumerate the types for which you want to enable rvalue capture, then you're good. For example, if you want to create a generic unary function some_fn, and you know you want to at least capture rvalues of type X, you could do: template< class T > void some_fn(T&); // captures (lvalue) references-to-non-const, explicitly created emulated rvalue references, *and* (lvalue) references-to-(X const) template< class T > typename boost::disable_if< boost::is_same<T,X> >::type void some_fn(T const &); // captures (lvalue) references-to-const *except* (lvalue) references-to-(X const) void some_fn(rv<X>&); // captures rvalues of X as emulated rvalue references However, if Y is any type other than X, the above overloads will capture an rvalue of Y as an (lvalue) reference-to-(Y const). This is one of the shortcomings of using the emulated rvalue reference rv<T>& as a function argument: it can't bind to rvalues in a context where the template parameter T is deduced. Now I don't know if the following proposition is a good idea or not, but I think it is worth considering. Again, we would like the emulated rvalue reference rv<T>& to bind to rvalues, but the problem is the deduction of the T template parameter. Instead of using rv<T>& to capture arbitrary rvalues, I'll use a kind of type-erased emulated rvalue reference, generic_rv<...> (for lack of a better name at the moment), which will just store a void pointer to the original, pre-type-erased object. Of course, we need some way to recover the object back from the void pointer, and that's what the "..." template parameters are for. This is where the (unfortunate) runtime overhead comes in, as some dispatching mechanism (e.g., function pointers) needs to be used to effect the recovery of the original, pre-type-erased object. The important difference between rv<T> and generic_rv<...> is that there's no T in the template parameter list "..." of generic_rv; more to the point, its template parameters are not deduced within the context of binding function arguments of some_fn, hence the compiler will be allowed to look for conversions from arbitrary rvalues to generic_rv<...>. Hopefully that gives enough of an introduction to the idea. Further details are probably best left to a code snippet. Comments? Specifically, should this be considered as an addition to Boost.Move's current emulation framework? Has anyone seen this technique before? ---------------- #include <iostream> #include <boost/type_traits/is_convertible.hpp> #include <boost/utility/enable_if.hpp> // rv<T>& emulates T&& template< class T > class rv : public T { rv(); rv(rv const &); rv& operator=(rv const &); }; template< class T > struct has_move_emulation : boost::is_convertible< T, rv<T>& > { }; // *** NEW EMULATED RVALUE REFERENCE *** // generic_rv< F, Result > also emulates an rvalue reference, but in a kind of // type-erased way. The actual object referred to is referenced by a void // pointer. To be able to cast the void pointer back to a pointer-to-object, we // need to package the void pointer together with some kind of dispatching // mechanism. We use a function pointer to an instantiation of cast_forward<T>. template< class F, class Result = void > struct generic_rv { typedef Result (*p_cast_forward_type)(F, void*); generic_rv(p_cast_forward_type p_cast_forward, void* p) : mp_cast_forward(p_cast_forward), m_p(p) { } Result cast_forward(F f) const { return (*mp_cast_forward)(f, m_p); } template< class T > static Result cast_forward(F f, void* const p) { return f(static_cast< rv<T>& >(*static_cast< T* >(p))); } private: p_cast_forward_type mp_cast_forward; void* m_p; }; // X is just a typical move-emulation-enabled class. struct X { X() { std::cout << "X::X()" << std::endl; } X(X const &) { std::cout << "X::X(X const &)" << std::endl; } X(rv<X>&) { std::cout << "X::X(rv<X>&)" << std::endl; } X& operator=(X) { std::cout << "X::operator=(X)" << std::endl; return *this; } X& operator=(rv<X>&) { std::cout << "X::operator=(rv<X>&)" << std::endl; return *this; } operator rv<X>&() { std::cout << "X::operator rv<X>&()" << std::endl; return *static_cast< rv<X>* >(this); } operator rv<X> const &() const { std::cout << "X::operator rv<X> const &() const" << std::endl; return *static_cast< rv<X> const * >(this); } // *** NEW CONVERSION OPERATOR *** // X provides an implicit conversion to generic_rv. template< class F, class Result > operator generic_rv< F, Result >() { std::cout << "X::operator generic_rv< F, Result >()" << std::endl; return generic_rv< F, Result >( &generic_rv< F, Result >::template cast_forward<X>, static_cast< void* >(this) ); } }; struct some_fn { template< class T > void operator()(T&) const { std::cout << "some_fn::operator()(T&) const" << std::endl; } template< class T > typename boost::disable_if< has_move_emulation<T> >::type operator()(T const &) const { std::cout << "some_fn::operator()(T const &) const" << std::endl; } // Notice that this overload requires template parameter deduction, hence // the compiler cannot apply an implicit conversion to rv<T>& from rvalues // of move-emulation-enabled types. In other words, this overload can only // bind to *explicitly* created emulated rvalue references, not to "real" // rvalues :( template< class T > void operator()(rv<T>&) const { std::cout << "some_fn::operator()(rv<T>&) const" << std::endl; } // *** NEW FUNCTION OVERLOAD TO CAPTURE RVALUES *** // Since this overload requires no template parameter deduction, the // compiler *can* apply an implicit conversion to generic_rv<...> from // rvalues of move-emulation-enabled types. Yes, there will be some runtime // overhead from the function pointer dispatching, but that will often be // preferable to copying x! void operator()(generic_rv< some_fn > const x) const { std::cout << "some_fn::operator()(generic_rv< some_fn >) const" << std::endl; return x.cast_forward(*this); } }; template< class T > T make() { return T(); } int main(int argc, char* argv[]) { int a = 0; int const b = 0; some_fn()(a); // some_fn::operator()(T&) const some_fn()(b); // some_fn::operator()(T const &) const some_fn()(make< int >()); // some_fn::operator()(T const &) const X x; // X::X() X const y; // X::X() some_fn()(x); // some_fn::operator()(T&) const some_fn()(y); // some_fn::operator()(T&) const some_fn()(make<X>()); // X::X() // X::operator generic_rv< F, Result >() // some_fn::operator()(generic_rv< some_fn >) const // some_fn::operator()(rv<T>&) const return 0; } ---------------- - Jeff
At Mon, 4 Apr 2011 10:36:29 -0700, Jeffrey Lee Hellrung, Jr. wrote:
I've recently discovered (well, probably rediscovered)
Whoa; this is cool, and I don't believe I've seen anything like it anywhere before.
an alternate rvalue reference emulation, one which allows rvalues ***of any arbitrary (move-emulation-enabled) type*** to be captured in C++03 and identified as rvalues, rather than references-to-const (there is a drawback, though, which I'll get to below; as usual, there's no "free lunch").
I'm actually not sure there's no free lunch...
Instead of using rv<T>& to capture arbitrary rvalues, I'll use a kind of type-erased emulated rvalue reference, generic_rv<...> (for
I think you should spell out "..." as "F, Result" in any descriptions or it just gets confusing. "Is he talking about variadic templates," I wondered?
lack of a better name at the moment), which will just store a void pointer to the original, pre-type-erased object. Of course, we need some way to recover the object back from the void pointer, and that's what the "..." template parameters are for. This is where the (unfortunate) runtime overhead comes in, as some dispatching mechanism (e.g., function pointers) needs to be used to effect the recovery of the original, pre-type-erased object.
First, did you try this with function references instead of function pointers? Next, which compilers did you try this on? I've found that when everything is inlined and flow analysis allows the compiler to "see" the source of function pointers/references, many compilers can turn an indirect call into a direct one.
The important difference between rv<T> and generic_rv<...> is that there's no T in the template parameter list "..." of generic_rv; more to the point, its template parameters are not deduced within the context of binding function arguments of some_fn, hence the compiler will be allowed to look for conversions from arbitrary rvalues to generic_rv<...>. Hopefully that gives enough of an introduction to the idea. Further details are probably best left to a code snippet.
Comments? Specifically, should this be considered as an addition to Boost.Move's current emulation framework?
Yes! -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Mon, Apr 4, 2011 at 3:33 PM, Dave Abrahams <dave@boostpro.com> wrote:
At Mon, 4 Apr 2011 10:36:29 -0700, Jeffrey Lee Hellrung, Jr. wrote:
[...]
an alternate rvalue reference emulation, one which allows rvalues ***of any arbitrary (move-emulation-enabled) type*** to be captured in C++03 and identified as rvalues, rather than references-to-const (there is a drawback, though, which I'll get to below; as usual, there's no "free lunch").
I'm actually not sure there's no free lunch...
I also forgot to mention that (obviously) this precludes polymorphic return types :( At least, I don't see how they can be supported...
Instead of using rv<T>& to capture arbitrary rvalues, I'll use a
kind of type-erased emulated rvalue reference, generic_rv<...> (for
I think you should spell out "..." as "F, Result" in any descriptions or it just gets confusing. "Is he talking about variadic templates," I wondered?
Ai, good suggestion...I have yet to move to a compiler supporting variadic templates.
lack of a better name at the moment), which will just store a void
pointer to the original, pre-type-erased object. Of course, we need some way to recover the object back from the void pointer, and that's what the "..." template parameters are for. This is where the (unfortunate) runtime overhead comes in, as some dispatching mechanism (e.g., function pointers) needs to be used to effect the recovery of the original, pre-type-erased object.
First, did you try this with function references instead of function pointers?
Now I did. Will provide assembly listing below. It looks like it doesn't get inlined :( Next, which compilers did you try this on?
MSVC9 only. I've found that when everything is inlined and flow analysis allows
the compiler to "see" the source of function pointers/references, many compilers can turn an indirect call into a direct one.
Perhaps GCC will do better. I'll try to generate the assembly output in that case...
The important
difference between rv<T> and generic_rv<...> is that there's no T in the template parameter list "..." of generic_rv; more to the point, its template parameters are not deduced within the context of binding function arguments of some_fn, hence the compiler will be allowed to look for conversions from arbitrary rvalues to generic_rv<...>. Hopefully that gives enough of an introduction to the idea. Further details are probably best left to a code snippet.
Comments? Specifically, should this be considered as an addition to Boost.Move's current emulation framework?
Yes!
I appreciate your enthusiasm, Dave :) I've attached generic_rv_test.cpp and generic_rv_test.asm. I've also reproduced the relevant assembly listings below, as generated by MSVC9. First, the assembly listing for the member function generic_rv<some_fn,void>::cast_forward<X>: ; COMDAT ??$cast_forward@UX@@@?$generic_rv@Usome_fn@@X@@SAXUsome_fn@ @QAX@Z _TEXT SEGMENT _f$ = 8 ; size = 1 _p$ = 12 ; size = 4 ??$cast_forward@UX@@@?$generic_rv@Usome_fn@@X@@SAXUsome_fn@@QAX@Z PROC ; generic_rv<some_fn,void>::cast_forward<X>, COMDAT ; 41 : static Result cast_forward(F f, void* const p) mov eax, DWORD PTR __imp_?endl@std@@YAAAV?$basic_ostream@DU ?$char_traits@D@std@@@1@AAV21@@Z mov ecx, DWORD PTR __imp_?cout@std@@3V?$basic_ostream@DU ?$char_traits@D@std@@@1@A push eax push OFFSET ??_C@_0CC@IOIEODGA @some_fn?3?3operator?$CI?$CJ?$CIrv?$DMT?$DO?$CG?$CJ?5cons@ push ecx call ??$?6U?$char_traits@D@std@@@std@@YAAAV?$basic_ostream@DU ?$char_traits@D@std@@@0@AAV10@PBD@Z ; std::operator<<<std::char_traits<char>
add esp, 8 mov ecx, eax call DWORD PTR __imp_??6?$basic_ostream@DU?$char_traits@D@std@@@std@ @QAEAAV01@P6AAAV01@AAV01@@Z@Z ; 42 : { return f(static_cast< rv<T>& >(*static_cast< T* >(p))); } ret 0 ??$cast_forward@UX@@@?$generic_rv@Usome_fn@@X@@SAXUsome_fn@@QAX@Z ENDP ; generic_rv<some_fn,void>::cast_forward<X> ; Function compile flags: /Ogtpy _TEXT ENDS Looks like a direct to call to printing to std::cout. Next, the assembly listing for the expression some_fn()(make<X>()): lea ecx, DWORD PTR $T25928[esp+8] mov BYTE PTR $T25927[esp+8], 0 mov edx, DWORD PTR $T25927[esp+8] push ecx push edx call ??$cast_forward@UX@@@?$generic_rv@Usome_fn@@X@@SAXUsome_fn@ @QAX@Z ; generic_rv<some_fn,void>::cast_forward<X> This appears to call generic_rv<some_fn,void>::cast_forward<X>. I don't know, is this basically the code that invokes the function reference? (Might have to dig through the asm file to answer that.) Lastly, the assembly listing for the expression some_fn()(static_cast< rv<X>& >(make<X>())): mov eax, DWORD PTR __imp_?endl@std@@YAAAV?$basic_ostream@DU ?$char_traits@D@std@@@1@AAV21@@Z add esp, 8 push eax push ecx mov ecx, DWORD PTR __imp_?cout@std@@3V?$basic_ostream@DU ?$char_traits@D@std@@@1@A push OFFSET ??_C@_0CC@IOIEODGA @some_fn?3?3operator?$CI?$CJ?$CIrv?$DMT?$DO?$CG?$CJ?5cons@ push ecx call ??$?6U?$char_traits@D@std@@@std@@YAAAV?$basic_ostream@DU ?$char_traits@D@std@@@0@AAV10@PBD@Z ; std::operator<<<std::char_traits<char>
add esp, 12 ; 0000000cH mov ecx, eax call DWORD PTR __imp_??6?$basic_ostream@DU?$char_traits@D@std@@@std@ @QAEAAV01@P6AAAV01@AAV01@@Z@Z Ouch...that gets completely inlined, unlike the previous case... - Jeff
On Mon, Apr 4, 2011 at 10:14 PM, Jeffrey Lee Hellrung, Jr. < jeffrey.hellrung@gmail.com> wrote:
On Mon, Apr 4, 2011 at 3:33 PM, Dave Abrahams <dave@boostpro.com> wrote:
I've found that when everything is inlined and flow analysis allows the compiler to "see" the source of function pointers/references, many compilers can turn an indirect call into a direct one.
Perhaps GCC will do better. I'll try to generate the assembly output in that case...
It looks to me like GCC likewise does not inline the indirection, based on a study of the output of passing -fdump-tree-all (found this option from a stackoverflow question). The assembly output I was able to generate with g++ was basically incomprehensible to me, but I didn't spend very much time on it and I'm sure there's a way to make it at least as readable as MSVC's assembly output, if anyone wants to try (see previous email for attached cpp file). - Jeff
At Mon, 4 Apr 2011 22:14:29 -0700, Jeffrey Lee Hellrung, Jr. wrote:
I also forgot to mention that (obviously) this precludes polymorphic return types :( At least, I don't see how they can be supported...
Sorry, can you show an example of what's precluded, using C++0x notation? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Tue, Apr 5, 2011 at 7:37 AM, Dave Abrahams <dave@boostpro.com> wrote:
At Mon, 4 Apr 2011 22:14:29 -0700, Jeffrey Lee Hellrung, Jr. wrote:
I also forgot to mention that (obviously) this precludes polymorphic
return
types :( At least, I don't see how they can be supported...
Sorry, can you show an example of what's precluded, using C++0x notation?
There's no way to make the return type dependent on the argument type, since the argument type has been erased: // C++0x, with rvalue references template< class T > typename result_of::some_fn<T>::type // result type is polymorphic; it depends on the argument type some_fn(T&& x); // C++03, with emulated "typeless" rvalue reference typename result_of::some_fn< /* what goes here? */ >::type // result type cannot depend on the argument type, since we don't know the argument type! some_fn(generic_rv< ???, ??? > x); Perhaps my use of the term "polymorphic" here is wrong; if so, let me know! - Jeff
At Tue, 5 Apr 2011 07:44:02 -0700, Jeffrey Lee Hellrung, Jr. wrote:
On Tue, Apr 5, 2011 at 7:37 AM, Dave Abrahams <dave@boostpro.com> wrote:
At Mon, 4 Apr 2011 22:14:29 -0700, Jeffrey Lee Hellrung, Jr. wrote:
I also forgot to mention that (obviously) this precludes polymorphic
return
types :( At least, I don't see how they can be supported...
Sorry, can you show an example of what's precluded, using C++0x notation?
There's no way to make the return type dependent on the argument type, since the argument type has been erased:
// C++0x, with rvalue references template< class T > typename result_of::some_fn<T>::type // result type is polymorphic; it depends on the argument type some_fn(T&& x);
// C++03, with emulated "typeless" rvalue reference typename result_of::some_fn< /* what goes here? */ >::type // result type cannot depend on the argument type, since we don't know the argument type! some_fn(generic_rv< ???, ??? > x);
Perhaps my use of the term "polymorphic" here is wrong; if so, let me know!
No, that makes perfect sense. Thanks for explaining. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Mon, Apr 4, 2011 at 10:36 AM, Jeffrey Lee Hellrung, Jr. < jeffrey.hellrung@gmail.com> wrote: [...snipping original description...] I have a follow-up to my original message regarding a typeless emulated rvalue reference. I was disappointed in the lack of feedback (other than Dave) on the original message, which might mean that those who took the time to parse the message didn't think it was worth pursuing; maybe C++03 is already too outdated and we should forget about it? However, at the moment, I prefer to think that everyone (except, again, Dave; thanks Dave!) was simply too busy to delve into what I was trying to convey. So let me try again, with a longer, more detailed, and more leisurely explanation, and this time I additionally present a partial solution to extend the technique to quasi-polymorphic result types (I'll explain what I mean below). If you're already plenty familiar with the difficulty of capturing arbitrary rvalues in C++03, you can skip to "TYPELESS EMULATED RVALUE REFERENCES". The "new" stuff dealing with quasi-polymorphic result types is even further down, at "QUASI-POLYMORPHIC RESULT TYPES". [In the exposition below, whenever I say "lvalue-to-const" or "lvalue-to-non-const", I usually mean "lvalue-reference-to-const" and "lvalue-reference-to-non-const", respectively.] First, the big picture: In C++03, it is notoriously difficult (and, until I discovered this use of typeless emulated rvalue references, I thought it was impossible) to capture *arbitrary* rvalues *and* distinguish between captured rvalues and lvalues-to-const. To be specific, one generally uses a "template< class T > void foo(T const &)" overload to capture arbitrary rvalues, but unfortunately this overload also binds to lvalues-to-const. This makes it impossible to distinguish between a captured rvalue and an lvalue-to-const, hence the function body must go the safe route and assume the argument is const, precluding any kind of move optimizations. Now, if we restrict the problem to capturing *specific* rvalues instead of *arbitrary* rvalues, then things look better. Boost.Move defines a class template rv<T> and introduces the convention that rv<T>& represents an (emulated) rvalue reference. A move-emulation-enabled class X defines a non-const conversion operator to rv<X>&, which basically just amounts to a static cast: "return static_cast< rv<X>& >(*this);" (see the Boost.Move documentation, either in trunk or the sandbox; or past mailing list messages; if this is unfamiliar). This means that overloads with argument rv<X>& are candidates to capture rvalues of type X, and leads to defining the following 3 overloads for foo: template< class T > void foo(T&); // 1 template< class T > typename boost::disable_if< boost::is_convertible< T&, rv<X>& > > void foo(T const &); // 2 void foo(rv<X>&); // 3 If one goes through all the various argument binding possibilities, one finds that: the "foo(T&); // 1" overload captures lvalues-to-non-const, X const & (due to the use of SFINAE via disable_if in the "foo(T const &); // 2" overload), and explicitly created emulated rvalue references *except* rv<X>& (these are captured by the "foo(rv<X>&); // 3" overload); the "foo(T const &); // 2" overload captures lvalues-to-const *except* X const & (these are captured by the "foo(T&); // 1" overload due to disable_if) and rvalues *except* X-rvalues (again, thanks to disable_if); finally, the "foo(rv<X>&); // 3" overload captures X-rvalues and explicitly created emulated-rvalue-references-to-X. To be absolutely clear: template< class T > T make(); // ... X x; X const cx; Y y; // Y != X Y const cy; foo(x); // 1, T = X foo(cx); // 1, T = X const foo(make<X>()); // 3 (*not* 2 !!!), as desired :) foo(y); // 1, T = Y foo(cy); // 2, T = Y foo(make<Y>()); // 2, T = Y, since we didn't tell the foo overloads about Y :( // and if Y is move-emulation-enabled... foo(static_cast< rv<Y>& >(y)); // 3 Thus, X-rvalues are *accurately* captured, meaning that an X-rvalue argument to foo is bound to an emulated-rvalue-reference-to-X, rv<X>&, allowing the argument to be moved rather than copied. This is better than using only the first two foo overloads, especially if the class X plays a prominent role in the specific application of the foo overload set. For example, the push_back member function of vector<X> can accurately capture X-rvalues using this technique, which is highly desirable. Clearly, this technique can be extended to any finite, compile-time-known set of "special" move-emulation-enabled types X0, ..., X[n-1] such that all Xi-rvalues (i = 0, ..., n-1) are accurately captured. This would be applicable to, say, variant< X0, ..., X[n-1] > constructor and assignment overloads. The ability to accurately capture any rvalue of a type within a specific, compile-time-known set of "special" types is quite powerful, and is, indeed, sufficient for many applications. However, many other applications don't have such a set of special types; instead, it is desirable to accurately capture arbitrary rvalues of a priori unknown types. At first, one might think that the emulated rvalue reference rv<T>& might come to the rescue and try to use a "template< class T > void foo2(rv<T>&)" overload. However, this overload cannot bind to rvalues: template< class T > void foo2(rv<T>&); // ... foo2(make<X>()); // will not bind to foo2<T>(rv<T>&)! :( Unfortunately, the "T" in the "rv<T>&" argument is a deduced template parameter, so there's no way for the compiler to know what T should be when looking for conversions from X to rv<T>&. It might look obvious that T should be X, especially considering that X provides an implicit conversion to rv<X>& and this fact is plainly available to the compiler, but, trust me, things just don't (and can't) work that way. The problem appears to lie in the template parameter deduction, and this motivates the introduction of a TYPELESS EMULATED RVALUE REFERENCE What we're going to try to do is capture arbitrary rvalues in a typeless fashion, thereby obviating the need for any template parameter deduction and allowing the compiler to locate a conversion operator from the rvalue to the typeless emulated rvalue reference: template< class T > void foo3(genrv< ??? > const x) { /* ... */ } genrv< ??? > is basically just a wrapper around a void pointer which points to the type-erased object ("gen" being short for "generic"). Of course, we have to be able to recover the original type of the object prior to type-erasure, which we effect via dynamic dispatch through a function reference. We use the function reference together with the visitor pattern to be able to "do something" with caught object. A basic formulation of the pieces looks as follows: // New typeless emulated rvalue reference class. template< class Visitor > struct genrv { // Adding support for non-void result types is easy, but we'll use void for now to simplify exposition. typedef void result_type; template< class T > explicit genrv(T* const p) // Set the function reference to a specific instantiation of the static apply member function. // This way, we can recover the type of the object pointed to by mp later. : m_apply(apply<T>), mp(static_cast< void* >(p)) { } result_type operator()(Visitor visitor) const { return m_apply(visitor, mp); } private: template< class T > static result_type apply(Visitor visitor, void* const p) // Explicitly cast the object to an emulated rvalue reference to ensure the right overload is selected. { return visitor(static_cast< rv<T>& >(*static_cast< T* >(p))); } result_type (&m_apply)(Visitor, void* const); void* const mp; }; // ... // Assume X is just a typical move-emulation-enabled class... struct X { // ... // The usual conversion to rv<X>&, as introduced by Boost.Move. operator rv<X>&() { return static_cast< rv<X>& >(*this); } // New conversion operator to genrv<V>! template< class V > operator genrv<V>() { return genrv<V>(this); } }; // ... // Typical function object... struct foo4 { template< class T > void operator()(T&) const; // 1 typedef genrv< foo4 > genrv_type; template< class T > typename boost::disable_if< boost::is_convertible< T&, genrv_type >
::type operator()(T const &) const; // 2
// The disable_if in "operator()(T const &) const; // 2" forces rvalues of move-emulation-enabled classes to bind here. void operator()(genrv_type const x) const // 3 // Dispatch to overload "operator()(T&) const; // 1", where T = rv< [whatever type x is wrapping] >. { return x(*this); } }; Let's go through what's happening here. Suppose foo4 is passed an X-rvalue as an argument. This rvalue cannot bind to the "operator()(T&) const; // 1" overload; the fact that X has an implicit conversion to foo4::genrv_type means that it cannot bind to "operator()(T const &) const; // 2" due to the disable_if *and* that it *can* bind to "void operator()(genrv_type const x) const // 3". So overload 3 is chosen, requiring the implicit conversion from the X-rvalue to a genrv_type. The genrv_type is constructed such that its m_apply member variable references genrv_type::apply<X>(Visitor, void* const) and its mp member variable points to the X-rvalue. Then we *visit* the genrv_type object with the foo4 function object (this is the "x(*this)" expression), causing a runtime dispatch to genrv_type::apply<X>(Visitor, void* const) via the m_apply function reference. genrv_type::apply<X>(Visitor, void* const) "knows" the "real" type of the object pointed to be the void pointer is X, so it can safely cast up to the emulated rvalue reference rv<X>& and pass this to the visitor object. Execution thus passes to the "void operator()(T&) const; // 1" overload, with T = rv<X>, which is the desired outcome. The X-rvalue is (eventually) bound to an emulated rvalue reference, without foo4 possessing any special knowledge of X itself! One can also verify that passing an lvalue or a non-move-emulation-enabled rvalue as an argument to foo4 will bind to one of the first two overloads, as desired. That's the basic idea. Now, the drawbacks. First, of course, there's the dynamic dispatch through the function reference to recover the original type of the object. Unfortunately, it looks like MSVC9 and the version of GCC I tried (a relatively recent version) were unable to eliminate the dispatch (default release mode settings for MSVC9; only added -O3 to GCC), implying some runtime overhead. However, the cost of such an indirection would likely be compensated many times over by through the avoidance of unnecessary copying of movable rvalues. In other words, this is still better than capturing by "T const &". The second (and, in my opinion, more serious) drawback is a pretty severe restriction on the result type. Fully polymorphic result types (that is, result types that depend arbitrarily on the argument type(s)) are out of the question. In other words, there's no way to do the equivalent of the following C++0x code in C++03 without result_of::foo5 possessing some kind of almost trivial "structure", like being a constant metafunction. namespace result_of { template< class T > struct foo5 { typedef /* ... */ type; }; } template< class T > typename result_of::foo5<T>::type foo5(T&& x); One can support non-void monomorphic result types fairly easily by only a slight adjustment of the genrv class template and the genrv conversion operator: // Adding a Result template parameter...not much else changes. template< class Visitor, class Result = void > struct genrv { typedef Result result_type; template< class T > explicit genrv(T* const p) : m_apply(apply<T>), mp(static_cast< void* >(p)) { } result_type operator()(Visitor visitor) const { return m_apply(visitor, mp); } private: template< class T > static result_type apply(Visitor visitor, void* const p) { return visitor(static_cast< rv<T>& >(*static_cast< T* >(p))); } result_type (&m_apply)(Visitor, void* const); void* const mp; }; // ... struct X { // ... operator rv<X>&() { return static_cast< rv<X>& >(*this); } template< class V, class R > operator genrv<V,R>() { return genrv<V,R>(this); } }; // ... struct foo6 { typedef /*...*/ result_type; template< class T > result_type operator()(T&) const; // 1 // Just need to specify the Result template parameter in genrv... typedef genrv< foo6, result_type > genrv_type; template< class T > typename boost::disable_if< boost::is_convertible< T&, genrv_type >, result_type >::type operator()(T const &) const; // 2 result_type operator()(genrv_type const x) const // 3 { return x(*this); } }; This was as far as I had gotten before, in the original message. Today, I got one step further, figuring out how to support QUASI-POLYMORPHIC RESULT TYPES By a quasi-polymorphic result type, I mean that the result type *can* depend on the argument type, but only in a very limited way. Specifically, the following technique allows one to support any finite, compile-time-known set of result types. The basic idea here is to be able to "tag" certain genrv instantiations and limit the implicit conversions to these instantiations. Let's use a concrete example with just two result types. Suppose we partition our argument type space up into 2 sets, those which satisfy predA and those which satisfy predB, where pred[A,B] are Boost.MPL metafunctions. Thus, every argument type satisfies either predA or predB, but never both simultaneously (so, informally, predB(T) = !predA(T)). Let resultA and resultB be the result types after application on arguments types satisfying predA and predB, respectively. In other words: struct foo7 { template<class> struct result; // Boost.ResultOf protocol template< class This, class T > struct result< This ( T ) > : boost::mpl::if_< boost::mpl::apply1< predA, T >, resultA, resultB > { }; // ... }; To support this result structure for foo7, we add a couple extra template parameters to genrv: template< class Visitor, class Result = void, // Add a couple more template parameters to aid the suppression of certain implicit conversions, as we'll shortly see... class Cond = boost::mpl::always< boost::true_type >, bool Enable = true
struct genrv { typedef Result result_type; template< class T > explicit genrv(T* const p) : m_apply(apply<T>), mp(static_cast< void* >(p)) // Make sure we didn't f*** up. { BOOST_MPL_ASSERT((boost::mpl::apply1< Cond, T >)); } result_type operator()(Visitor visitor) const { return m_apply(visitor, mp); } private: template< class T > static result_type apply(Visitor visitor, void* const p) { return visitor(static_cast< rv<T>& >(*static_cast< T* >(p))); } result_type (&m_apply)(Visitor, void* const); void* const mp; }; We likewise supplement the genrv conversion operator in X: // MSVC9 ICE's without this helper struct :( template< class Cond, class T > struct apply_cond { static const bool value = boost::mpl::apply1< Cond, T >::type::value; }; struct X { // ... template< class V, class R, class C > operator genrv<V,R,C,apply_cond<C,X>::value>() { return genrv<V,R,C,apply_cond<C,X>::value>(this); } }; Now, when the compiler is looking to convert an X to a genrv<V,R,C,true>, that conversion operator will only exist if X satisfies the Boost.MPL predicate C. If X does *not* satisfy C, then X can only be converted to genrv<V,R,C,false>. Now, our foo7 overloads with genrv arguments will always use the default Enable parameter, which is true, thus ensuring that anything that binds to genrv<V,R,C> really does satisfy C. Thus, the foo7 overloads look like the following: struct foo7 { template<class> struct result; // Boost.ResultOf protocol template< class This, class T > struct result< This ( T ) > : boost::mpl::if_< boost::mpl::apply1< predA, T >, resultA, resultB > { }; // Add a specialization to transform rv<T>& to plain T. template< class This, class T > struct result< This ( rv<T>& ) > : result< This ( T ) > { }; template< class T > typename result< foo7 ( T& ) >::type operator()(T&) const; // 1 typedef genrv< foo7, resultA, predA > genrvA_type; typedef genrv< foo7, resultB, predB > genrvB_type; template< class T > typename boost::lazy_disable_if< boost::mpl::or_< boost::is_convertible< T&, genrvA_type >, boost::is_convertible< T&, genrvB_type > >, result< foo7 ( T const & ) > >::type operator()(T const &) const; // 2 // Captures move-emulation-enabled rvalues satisfying predA. resultA operator()(genrvA_type const x) const // 3A { return x(*this); } // Captures move-emulation-enabled rvalues satisfying predB. resultB operator()(genrvB_type const x) const // 3B { return x(*this); } }; This is still pretty far from supporting fully polymorphic result types, but it is still (maybe only slightly) better than monomorphic result types, and likely the best we can do with this technique, given that none of the genrv overloads can have deduced template parameters. CONCLUSION Despite the drawbacks of this technique (the probable runtime overhead, limitations on the result type, and additional complexity), I think this would be a worthwhile addition to the Boost.Move library. I look forward to hearing what others think about it... - Jeff
On 4/10/2011 9:22 PM, Jeffrey Lee Hellrung, Jr. wrote:
On Mon, Apr 4, 2011 at 10:36 AM, Jeffrey Lee Hellrung, Jr.< jeffrey.hellrung@gmail.com> wrote: [...snipping original description...]
I have a follow-up to my original message regarding a typeless emulated rvalue reference. I was disappointed in the lack of feedback (other than Dave) on the original message, which might mean that those who took the time to parse the message didn't think it was worth pursuing; maybe C++03 is already too outdated and we should forget about it? However, at the moment, I prefer to think that everyone (except, again, Dave; thanks Dave!) was simply too busy to delve into what I was trying to convey. So let me try again, with a longer, more detailed, and more leisurely explanation, and this time I additionally present a partial solution to extend the technique to quasi-polymorphic result types (I'll explain what I mean below).
This is no doubt a disappointing response to your wonderful post, but I believe I can tell you why you received so little feedback. 'Move' is a C++0x concept which current C++ programmers view as an optimization, not a core programming issue, and is not supported by many current C++ compilers except in some sort of C++0x mode. Boost.Move may have brought the idea to programmers but it is not yet in Boost itself for some reason. So as disappointing as it may seem, very probably many programmers, like myself, know very little about 'Move', So it is not that C++03 is outdated but exactly the opposite, which is that C++0x is not here yet nor is Boost.Move and therefore the issues which 'Move' entails is not something of which most C++ programmers are not aware.
On 4/10/2011 9:22 PM, Jeffrey Lee Hellrung, Jr. wrote:
I have a follow-up to my original message regarding a typeless emulated rvalue reference. I was disappointed in the lack of feedback (other than Dave) on the original message, which might mean that those who took the time to parse the message didn't think it was worth pursuing;
Not at all. I am very interested in improved move emulation in C++03. Speaking solely from myself, I just don't have the time to do the technical investigation that it would take to respond intelligently to your mail. I expect some others are in the same position. Please don't be discouraged! -- Eric Niebler BoostPro Computing http://www.boostpro.com
On Mon, Apr 11, 2011 at 5:46 AM, Eric Niebler <eric@boostpro.com> wrote:
On 4/10/2011 9:22 PM, Jeffrey Lee Hellrung, Jr. wrote:
I have a follow-up to my original message regarding a typeless emulated rvalue reference. I was disappointed in the lack of feedback (other than Dave) on the original message, which might mean that those who took the time to parse the message didn't think it was worth pursuing;
Not at all. I am very interested in improved move emulation in C++03. Speaking solely from myself, I just don't have the time to do the technical investigation that it would take to respond intelligently to your mail. I expect some others are in the same position. Please don't be discouraged!
Totally understandable. Perhaps a short summary will be useful, with full details provided in [1]. I'm assuming familiarity with C++0x rvalue references. For those unfamiliar, I recommend the "RValue References: Moving Forward" sequence of articles by Dave Abrahams, e.g., [2]. I also recommend familiarity with Boost.Move [3,4], which basically provides the current state-of-the-art move and rvalue reference emulation within C++03. I propose to supplement Boost.Move to very closely approximate perfect forwarding. To be a little bit more specific... One of the problems that rvalue references solves is the forwarding problem [5]. To the best of my knowledge (until now!), the best approximation to generic forwarding in C++03 turns rvalues into lvalues ([5], indeed, says as much), but preserving the rvalueness of a function argument is important to identify when one can safely move from the argument. I have devised a technique (let's call it the "typeless emulated rvalue reference (TERR) technique") that, when combined with the existing Boost.Move emulation framework, can actually preserve rvalueness for *arbitrary* (move-emulation-enabled) types. To get an idea of what such a forwarding overload set might look like, let's modify the first example, unary_function_wrapper, from [5] to use a TERR to achieve perfect forwarding: template< class F > struct unary_function_wrapper { unary_function_wrapper(F f) : m_f(f) { } template< class T > void operator()(T& x) { m_f(x); } // genrv< Visitor > is a TERR typedef genrv< unary_function_wrapper > genrv_type; template< class T > typename boost::disable_if< boost::is_convertible< T&, genrv_type >
::type operator()(T const & x) { m_f(x); }
void operator()(genrv_type x) { x(*this); } private: F m_f; }; Now, if X is *any* move-emulation-enabled class, unary_function_wrapper<F> will capture X-rvalues by emulated-rvalue-reference rv<X>& and forward this onto the wrapper function object m_f, thus preserving rvalueness. Of course, it's far from obvious how that happens, exactly, just given the above code, so for those interested in *how* that's achieved, I again refer you to [1]. Hopefully this summary helps determine whether one should investigate [1] in detail or not. - Jeff [1] http://thread.gmane.org/gmane.comp.lib.boost.devel/217478/focus=217772 [2] http://cpp-next.com/archive/2009/09/move-it-with-rvalue-references/ [3] http://igaztanaga.drivehq.com/libs/move/doc/html/index.html [4] http://svn.boost.org/svn/boost/sandbox/move/ [5] http://cpp-next.com/archive/2009/12/onward-forward/
Message du 11/04/11 22:02 De : "Jeffrey Lee Hellrung, Jr." A : boost@lists.boost.org Copie à : Objet : Re: [boost] [move] new rvalue reference emulation
On Mon, Apr 11, 2011 at 5:46 AM, Eric Niebler wrote:
On 4/10/2011 9:22 PM, Jeffrey Lee Hellrung, Jr. wrote:
I have a follow-up to my original message regarding a typeless emulated rvalue reference. I was disappointed in the lack of feedback (other than Dave) on the original message, which might mean that those who took the time to parse the message didn't think it was worth pursuing;
Yes sure. Not everyone is able to see as quickly as him when there is something really valuable ;-) What about renaming genrv to any_rv? Have you already thought about how this enhanced emulation could be used in a portable way on C++98 and C++0x compilers? Best, Vicente
On Mon, Apr 11, 2011 at 3:03 PM, Vicente BOTET <vicente.botet@wanadoo.fr>wrote:
Message du 11/04/11 22:02 De : "Jeffrey Lee Hellrung, Jr." A : boost@lists.boost.org Copie à : Objet : Re: [boost] [move] new rvalue reference emulation
On Mon, Apr 11, 2011 at 5:46 AM, Eric Niebler wrote:
On 4/10/2011 9:22 PM, Jeffrey Lee Hellrung, Jr. wrote:
I have a follow-up to my original message regarding a typeless emulated rvalue reference. I was disappointed in the lack of feedback (other than Dave) on the original message, which might mean that those who took the time to parse the message didn't think it was worth pursuing;
Yes sure. Not everyone is able to see as quickly as him when there is something really valuable ;-)
It's probably more likely that not everyone has as much time as Dave does ;) What about renaming genrv to any_rv?
Yeah, that's probably a better name.
Have you already thought about how this enhanced emulation could be used in a portable way on C++98 and C++0x compilers?
No, I haven't :( At this point, the easiest way to write portable code seems to be to do an ugly "#ifndef BOOST_NO_RVALUE_REFERENCES / #else / #endif" dance :( Which I would expect would make it largely inaccessible to general users, mostly relegating this technique to libraries that want to provide a more performance-friendly interface. - Jeff
On 4/11/2011 8:03 AM, Edward Diener wrote:
On 4/10/2011 9:22 PM, Jeffrey Lee Hellrung, Jr. wrote:
On Mon, Apr 4, 2011 at 10:36 AM, Jeffrey Lee Hellrung, Jr.< jeffrey.hellrung@gmail.com> wrote: [...snipping original description...]
I have a follow-up to my original message regarding a typeless emulated rvalue reference. I was disappointed in the lack of feedback (other than Dave) on the original message, which might mean that those who took the time to parse the message didn't think it was worth pursuing; maybe C++03 is already too outdated and we should forget about it? However, at the moment, I prefer to think that everyone (except, again, Dave; thanks Dave!) was simply too busy to delve into what I was trying to convey. So let me try again, with a longer, more detailed, and more leisurely explanation, and this time I additionally present a partial solution to extend the technique to quasi-polymorphic result types (I'll explain what I mean below).
This is no doubt a disappointing response to your wonderful post, but I believe I can tell you why you received so little feedback. 'Move' is a C++0x concept which current C++ programmers view as an optimization, not a core programming issue, and is not supported by many current C++ compilers except in some sort of C++0x mode. Boost.Move may have brought the idea to programmers but it is not yet in Boost itself for some reason. So as disappointing as it may seem, very probably many programmers, like myself, know very little about 'Move', So it is not that C++03 is outdated but exactly the opposite, which is that C++0x is not here yet nor is Boost.Move and therefore the issues which 'Move' entails is not something of which most C++ programmers are not aware.
Correction. I meant to end with "which most C++ programmers are aware."
El 11/04/2011 3:22, Jeffrey Lee Hellrung, Jr. escribió:
On Mon, Apr 4, 2011 at 10:36 AM, Jeffrey Lee Hellrung, Jr.< jeffrey.hellrung@gmail.com> wrote: [...snipping original description...]
Sorry for the lack of feedaback! I tried to your example in some compilers and checked that MSVC7.1 does not like pointer to function storage in the generic rv (it's a msvc bug, but annoying, nevertheless). I've been very busy with several interprocess issues and I couldn't take time to study this but I promise to dedicate some time soon. I was a bit confused as you use a function object in your example. Can't we take rid of that function object and apply it to free functions? How can we apply this to the constructor of an object? Best, Ion
2011/4/11 Ion Gaztañaga <igaztanaga@gmail.com>
El 11/04/2011 3:22, Jeffrey Lee Hellrung, Jr. escribió:
On Mon, Apr 4, 2011 at 10:36 AM, Jeffrey Lee Hellrung, Jr.< jeffrey.hellrung@gmail.com> wrote: [...snipping original description...]
Sorry for the lack of feedaback! I tried to your example in some compilers and checked that MSVC7.1 does not like pointer to function storage in the generic rv (it's a msvc bug, but annoying, nevertheless).
It sounds like that could be easily worked around without too much obfuscation.
I've been very busy with several interprocess issues and I couldn't take time to study this but I promise to dedicate some time soon.
Great, thanks; as long as I know that you've seen the message, I'm happy :)
I was a bit confused as you use a function object in your example. Can't we take rid of that function object and apply it to free functions?
You can, but you would still need a function object to execute the visitor pattern on the genrv object, since the visitor must be polymorphic (it must accept many different argument types). Making the function object *both* the visitor *and* the "housing" of the overload set is merely a convenience.
How can we apply this to the constructor of an object?
Unfortunately, here the technique can get quite cumbersome. After thinking a little about this, I think you'd have to be in one of the following situations: - Use no initializer lists, and perform the construction in an "init" method. This should be okay for, e.g., pimpl classes. - All member object(s) and base class(es) that receive the to-be-forwarded argument can accept a genrv object. - Use some kind of "typeless opaque storage" for all member object(s) that receive the to-be-forwarded argument to delay initialization until the constructor body. Should be strongly discouraged, but sometimes you gotta do what you gotta do... - Jeff
Jeffrey Lee Hellrung, Jr.-2 wrote
2011/4/11 Ion Gaztañaga <igaztanaga@>
El 11/04/2011 3:22, Jeffrey Lee Hellrung, Jr. escribió:
On Mon, Apr 4, 2011 at 10:36 AM, Jeffrey Lee Hellrung, Jr.< jeffrey.hellrung@> wrote: [...snipping original description...]
Sorry for the lack of feedaback! I tried to your example in some compilers and checked that MSVC7.1 does not like pointer to function storage in the generic rv (it's a msvc bug, but annoying, nevertheless).
It sounds like that could be easily worked around without too much obfuscation.
I've been very busy with several interprocess issues and I couldn't take time to study this but I promise to dedicate some time soon.
Great, thanks; as long as I know that you've seen the message, I'm happy :)
I was a bit confused as you use a function object in your example. Can't we take rid of that function object and apply it to free functions?
You can, but you would still need a function object to execute the visitor pattern on the genrv object, since the visitor must be polymorphic (it must accept many different argument types). Making the function object *both* the visitor *and* the "housing" of the overload set is merely a convenience.
How can we apply this to the constructor of an object?
Unfortunately, here the technique can get quite cumbersome. After thinking a little about this, I think you'd have to be in one of the following situations: - Use no initializer lists, and perform the construction in an "init" method. This should be okay for, e.g., pimpl classes. - All member object(s) and base class(es) that receive the to-be-forwarded argument can accept a genrv object. - Use some kind of "typeless opaque storage" for all member object(s) that receive the to-be-forwarded argument to delay initialization until the constructor body. Should be strongly discouraged, but sometimes you gotta do what you gotta do...
Hi, Are there any news about this Move improvements? Best, Vicente -- View this message in context: http://boost.2283326.n4.nabble.com/move-new-rvalue-reference-emulation-tp342... Sent from the Boost - Dev mailing list archive at Nabble.com.
On Sat, Jan 21, 2012 at 4:22 PM, Vicente Botet <vicente.botet@wanadoo.fr>wrote:
Jeffrey Lee Hellrung, Jr.-2 wrote
2011/4/11 Ion Gaztañaga <igaztanaga@>
El 11/04/2011 3:22, Jeffrey Lee Hellrung, Jr. escribió:
On Mon, Apr 4, 2011 at 10:36 AM, Jeffrey Lee Hellrung, Jr.< jeffrey.hellrung@> wrote: [...snipping original description...]
Sorry for the lack of feedaback! I tried to your example in some compilers and checked that MSVC7.1 does not like pointer to function storage in
generic rv (it's a msvc bug, but annoying, nevertheless).
It sounds like that could be easily worked around without too much obfuscation.
I've been very busy with several interprocess issues and I couldn't take time to study this but I promise to dedicate some time soon.
Great, thanks; as long as I know that you've seen the message, I'm happy :)
I was a bit confused as you use a function object in your example. Can't we take rid of that function object and apply it to free functions?
You can, but you would still need a function object to execute the visitor pattern on the genrv object, since the visitor must be polymorphic (it must accept many different argument types). Making the function object *both* the visitor *and* the "housing" of the overload set is merely a convenience.
How can we apply this to the constructor of an object?
Unfortunately, here the technique can get quite cumbersome. After thinking a little about this, I think you'd have to be in one of the following situations: - Use no initializer lists, and perform the construction in an "init" method. This should be okay for, e.g., pimpl classes. - All member object(s) and base class(es) that receive the to-be-forwarded argument can accept a genrv object. - Use some kind of "typeless opaque storage" for all member object(s)
the that
receive the to-be-forwarded argument to delay initialization until the constructor body. Should be strongly discouraged, but sometimes you gotta do what you gotta do...
Hi,
Are there any news about this Move improvements?
Best, Vicente
None from my end. I've thought about it a bit since, and I *think* you can get by using this with any change to the existing Boost.Move infrastructure (only additions), so, if desired, it could be added to an "advanced techniques in C++03" section or something like that. Do you think it's worth adding to Boost.Move? - Jeff
Le 23/01/12 17:55, Jeffrey Lee Hellrung, Jr. a écrit :
On Sat, Jan 21, 2012 at 4:22 PM, Vicente Botet<vicente.botet@wanadoo.fr>wrote:
Hi,
Are there any news about this Move improvements?
Best, Vicente
None from my end.
I've thought about it a bit since, and I *think* you can get by using this with any change to the existing Boost.Move infrastructure (only additions), so, if desired, it could be added to an "advanced techniques in C++03" section or something like that.
Do you think it's worth adding to Boost.Move?
I need to emulate the thread constructor template <class F, class ...Args> explicit thread(F&& f, Args&&... args); at least for 2 args. If I have understood, there are some constraints when applying your technique on constructors, but guess that this is better than nothing. So yes, I would like to see a clear description on how to emulate this kind of functions with Boost.Move+ your additions Best, Vicente
On Mon, Jan 23, 2012 at 10:16 AM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 23/01/12 17:55, Jeffrey Lee Hellrung, Jr. a écrit :
On Sat, Jan 21, 2012 at 4:22 PM, Vicente Botet<vicente.botet@wanadoo.fr**
wrote:
Hi,
Are there any news about this Move improvements?
Best, Vicente
None from my end.
I've thought about it a bit since, and I *think* you can get by using this with any change to the existing Boost.Move infrastructure (only additions), so, if desired, it could be added to an "advanced techniques in C++03" section or something like that.
Do you think it's worth adding to Boost.Move?
I need to emulate the thread constructor
template <class F, class ...Args> explicit thread(F&& f, Args&&... args);
at least for 2 args.
If I have understood, there are some constraints when applying your technique on constructors, but guess that this is better than nothing.
So yes, I would like to see a clear description on how to emulate this kind of functions with Boost.Move+ your additions
Best, Vicente
Yikes. That could be challenging (in C++03). Though it's made easier by thread being pimpl'ed (at least, I'm guessing thread is pimpl'ed). Let me see if I can come up with a full example illustrating the technique for such a case. Give me a day or two :) - Jeff
on Mon Jan 23 2012, "Jeffrey Lee Hellrung, Jr." <jeffrey.hellrung-AT-gmail.com> wrote:
On Mon, Jan 23, 2012 at 10:16 AM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
at least, I'm guessing thread is pimpl'ed
Reading this surprised me so much that I had to look for myself, and indeed it does seem to be pimpl'ed. My understanding was that the move-only threading components were designed so that they could be implemented as efficiently as possible, without dynamic allocation, and I wonder why Boost's implementation would be less efficient. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 30/01/12 18:44, Dave Abrahams wrote:
on Mon Jan 23 2012, "Jeffrey Lee Hellrung, Jr."<jeffrey.hellrung-AT-gmail.com> wrote:
On Mon, Jan 23, 2012 at 10:16 AM, Vicente J. Botet Escriba< vicente.botet@wanadoo.fr> wrote:
at least, I'm guessing thread is pimpl'ed
Reading this surprised me so much that I had to look for myself, and indeed it does seem to be pimpl'ed. My understanding was that the move-only threading components were designed so that they could be implemented as efficiently as possible, without dynamic allocation, and I wonder why Boost's implementation would be less efficient.
"Efficiency" is in the eye of the beholder. The thread function supplied to the OS has to have various bits of data from the boost::thread constructor, not least of which the function (object) to call and its parameters. It has to store these somewhere that persists (at a fixed address) for the life of the thread, even if the boost::thread object is moved, or the thread is detached. One option is to allocate it dynamically, and pass in the pointer to the newly-allocated object to the thread function. The advantage here is that the boost::thread constructor doesn't have to wait for the thread to actually be scheduled by the OS, but the disadvantage is that it uses dynamic allocation. The second option is to wait for the thread to start, have the thread function allocate some memory off its own stack and pass the address back to the boost::thread constructor, which can then populate the memory with the necessary data before signalling the thread function to continue. The advantage here is that there is no dynamic allocation. The disadvantage is that the boost::thread constructor has to wait for the thread to be scheduled by the OS, and then there is back-and-forth communication. There may be a third option but I cannot think of one just now. Boost.Thread uses the first option because it returns control to the code that started the thread sooner, and does not wait for the OS to schedule the thread. It's a trade-off, so reasonable people may disagree with the choice. Anthony -- Author of C++ Concurrency in Action http://www.stdthread.co.uk/book/ just::thread C++11 thread library http://www.stdthread.co.uk Just Software Solutions Ltd http://www.justsoftwaresolutions.co.uk 15 Carrallack Mews, St Just, Cornwall, TR19 7UL, UK. Company No. 5478976
on Tue Jan 31 2012, Anthony Williams <anthony.ajw-AT-gmail.com> wrote:
On 30/01/12 18:44, Dave Abrahams wrote:
on Mon Jan 23 2012, "Jeffrey Lee Hellrung, Jr."<jeffrey.hellrung-AT-gmail.com> wrote:
On Mon, Jan 23, 2012 at 10:16 AM, Vicente J. Botet Escriba< vicente.botet@wanadoo.fr> wrote:
at least, I'm guessing thread is pimpl'ed
Reading this surprised me so much that I had to look for myself, and indeed it does seem to be pimpl'ed. My understanding was that the move-only threading components were designed so that they could be implemented as efficiently as possible, without dynamic allocation, and I wonder why Boost's implementation would be less efficient.
"Efficiency" is in the eye of the beholder. The thread function supplied to the OS has to have various bits of data from the boost::thread constructor, not least of which the function (object) to call and its parameters. It has to store these somewhere that persists (at a fixed address) for the life of the thread, even if the boost::thread object is moved, or the thread is detached.
Oh, silly me. That makes sense. Thanks for the explanation. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Mon, Jan 23, 2012 at 10:16 AM, Vicente J. Botet Escriba < vicente.botet@wanadoo.fr> wrote:
Le 23/01/12 17:55, Jeffrey Lee Hellrung, Jr. a écrit :
On Sat, Jan 21, 2012 at 4:22 PM, Vicente Botet<vicente.botet@wanadoo.fr**
wrote:
Hi,
Are there any news about this Move improvements?
Best, Vicente
None from my end.
I've thought about it a bit since, and I *think* you can get by using this with any change to the existing Boost.Move infrastructure (only additions), so, if desired, it could be added to an "advanced techniques in C++03" section or something like that.
Do you think it's worth adding to Boost.Move?
I need to emulate the thread constructor
template <class F, class ...Args> explicit thread(F&& f, Args&&... args);
at least for 2 args.
If I have understood, there are some constraints when applying your technique on constructors, but guess that this is better than nothing.
So yes, I would like to see a clear description on how to emulate this kind of functions with Boost.Move+ your additions
Apologies for the long delay. I finally got a minimal example program out to illustrate the idea. ---------------- #include <iostream> #include <typeinfo> #include <boost/move/move.hpp> #include <boost/mpl/always.hpp> #include <boost/mpl/and.hpp> #include <boost/mpl/apply.hpp> #include <boost/type_traits/integral_constant.hpp> #include <boost/type_traits/is_const.hpp> #include <boost/utility/addressof.hpp> #include <boost/utility/enable_if.hpp> namespace { // Necessary to workaround an MSVC9 ICE. template< class Pred, class T, bool = boost::has_move_emulation_enabled<T>::value
struct rv_sink_enable_ctor; template< class Pred, class T> struct rv_sink_enable_ctor< Pred, T, false > { }; template< class Pred, class T> struct rv_sink_enable_ctor< Pred, T, true > : boost::enable_if_c< boost::mpl::apply1< Pred, T >::type::value > { }; template< class Visitor, class Result = void, class Pred = boost::mpl::always< boost::true_type >
struct rv_sink { typedef Result result_type; // implicit on purpose template< class T > rv_sink(T const & x, typename rv_sink_enable_ctor< Pred, T >::type* = 0) : m_apply(apply<T>), mp(static_cast< void* >(boost::addressof(const_cast< T& >(x)))) { } result_type operator()(Visitor visitor) const { return m_apply(visitor, mp); } private: template< class T > static result_type apply(Visitor visitor, void* p) { return visitor(boost::move(*static_cast< T* >(p))); } result_type (&m_apply)(Visitor, void*); void* const mp; }; struct A { private: struct ctor_rv_sink { struct binder { explicit binder(A& this_) : m_this(this_) { } typedef void result_type; template< class T > void operator()(T& x) const { return m_this.init(x); } private: A& m_this; }; typedef rv_sink< binder > type; }; public: // lvalues template< class T > explicit A(T& x) { init(x); } // movable rvalues explicit A(ctor_rv_sink::type const x) { x(ctor_rv_sink::binder(*this)); } // const lvalues + non-movable rvalues template< class T > explicit A(T const & x, typename boost::disable_if_c< boost::has_move_emulation_enabled<T>::value >::type* = 0) { init(x); } private: template< class T > void init(T&) { std::cout << "A::init(T&) with T = " << typeid( T ).name() << (boost::is_const<T>::value ? " (const)" : "") << std::endl; } }; struct X { X() { std::cout << "X::X()" << std::endl; } X(X const &) { std::cout << "X::X(X const &)" << std::endl; } X& operator=(X const &) { std::cout << "X::operator=(X const &)" << std::endl; return *this; } static X make() { return X(); } }; struct Y { BOOST_COPYABLE_AND_MOVABLE( Y ) public: Y() { std::cout << "Y::Y()" << std::endl; } Y(Y const &) { std::cout << "Y::Y(Y const &)" << std::endl; } Y(BOOST_RV_REF( Y )) { std::cout << "Y::Y(Y&&)" << std::endl; } Y& operator=(BOOST_COPY_ASSIGN_REF( Y )) { std::cout << "Y::operator=(Y const &)" << std::endl; return *this; } Y& operator=(BOOST_RV_REF( Y )) { std::cout << "Y::operator=(Y&&)" << std::endl; return *this; } static Y make() { return Y(); } }; } // namespace int main(int argc, char* argv[]) { std::cout << "Constructing x, cx, y, cy..." << std::endl; X x; X const cx; Y y; Y const cy; std::cout << "Constructing ax..." << std::endl; A ax(x); std::cout << "Constructing acx..." << std::endl; A acx(cx); std::cout << "Constructing arx..." << std::endl; A arx(X::make()); std::cout << "Constructing ay..." << std::endl; A ay(y); std::cout << "Constructing acy..." << std::endl; A acy(cy); std::cout << "Constructing ary..." << std::endl; A ary(Y::make()); return 0; } ---------------- (The print statements are just to verify expected behavior.) So, with a 1 argument constructor, you can catch rvalues of move-emulation-enabled types using the above technique. You could extend this to 2 or 3 arguments, but it looks like it would take some binding and overload acrobatics to cover all the possibilities. If this looks like an acceptable solution, I can help with adding the requisite boilerplate (maybe some Boost.PP magic can make this manageable) to Boost.Thread. Note: This isn't as slick, efficient, or easy-to-implement as Dan's solution, but it doesn't rely on non-standard behavior and, perhaps, is safer regarding volatile move-emulation-enabled types. - Jeff
On Sun, Jan 22, 2012 at 2:22 AM, Vicente Botet <vicente.botet@wanadoo.fr> wrote:
Hi,
Are there any news about this Move improvements?
Best, Vicente
Hi Following the boost.move thread from a couple of weeks ago, I played a little with spin-offs on the technique illustrated there to see how far can it be pushed, and it turns out that quite a bit, actually. Judging from the relatively little interest that that other thread generated, I wasn't sure if it's worth the while to push it further, but I can share what I came up with if people are interested.
On Mon, Jan 23, 2012 at 1:40 PM, Dan Ivy <danivy.mail@gmail.com> wrote:
On Sun, Jan 22, 2012 at 2:22 AM, Vicente Botet <vicente.botet@wanadoo.fr> wrote:
Hi,
Are there any news about this Move improvements?
Best, Vicente
Hi
Following the boost.move thread from a couple of weeks ago, I played a little with spin-offs on the technique illustrated there to see how far can it be pushed, and it turns out that quite a bit, actually. Judging from the relatively little interest that that other thread generated, I wasn't sure if it's worth the while to push it further, but I can share what I came up with if people are interested.
I am! - Jeff
Le 23/01/12 22:40, Dan Ivy a écrit :
On Sun, Jan 22, 2012 at 2:22 AM, Vicente Botet<vicente.botet@wanadoo.fr> wrote:
Hi,
Are there any news about this Move improvements?
Best, Vicente Hi
Following the boost.move thread from a couple of weeks ago, I played a little with spin-offs on the technique illustrated there to see how far can it be pushed, and it turns out that quite a bit, actually. Judging from the relatively little interest that that other thread generated, I wasn't sure if it's worth the while to push it further, but I can share what I came up with if people are interested.
Hi, please share whatever could help. Best, Vicente
Sorry for taking so long to respond, I'm a little short in time lately. Unfortunately, I didn't get to this yet, so I'm just going to describe more-or-less where it's standing and, in particaular, what the outstanding issues are, so people could bring up comments or suggestions. First of all, in order to hide the implementation details and to make things portable across 03 an 11, function declarations inevitably have to be written inside a macro, much like boost.local. I guess Jeffrey's method would have to end up like that too, to be practical. So, I assume that we're cool with that. The range of scenarios I am trying to address is a bit wider than what discuessed on this thread. It seems that the focus here is on forwarding functions (which I'll refer to as completely-generic functions, ones whose parameters are of the form T&, const T&, etc... in contrast to partially-generic functions (i.e. f(const vector<T>&) ) and simple-functions (i.e. f(const vector<int>&) ). I'm looking for implicit rvalue detection for the other kinds too, which I suspect are the more common ones. For simple functions, I belive I have a non-intrusive solution wroking. Non-intrusive is to say that it works with any type, regardless of whether it declares itself as movable or not, much like true rv-refs are. To stress the usefulness of a non-intrusive solution, consider the std::string operator+ temporary hell described in [http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2002/n1377.htm#string Motivation]. Note that you don't actually need to access any of string's internals to implement the proposed optimization, just the ability to catch string rvalues, so a non-intrusive solution let's you write something like this on top of the existing std::string. I've tried it, and the performance boost is pretty sweet. Rvalue detection for completely-generic and partially-generic functions only works for types that inherit from movable<T>, as illustrated on the other thread (should I provide a link?). IIRC, Jeffrey was concerned that the inheritance can't be easily disabled for C++11, but I don't think it should be. Inheriting from movable<T> "for nothing" is a victimless crime (well *almost*. I'm thinking about MSVC and EBO). For partially-generic functions, there's a slight issue with the lvalue overloads. The basic set of overload for vector<T>, for instance, looks something like: template <typename T> void f(T& lvalue); template <typename T> void f(const movable<vector<T> >& rvalue) The point is that the lvalue overload has to take completely-generic parameter, so that it won't catch rvalues. Other than making the actual implementation work extra-hours to make OR work correctly, it makes calculating the return type tricky (because the deduced T in the lvalue overload isn't really the T we're looking for). If the arguments are an exact-match, this is rather simple. Otherwise, it must use BOOST_TYPEOF for that, which is eh. On some compilers, it's possible to do better. Instead of using T& params for lvalues, use const volatile vector<T>&. Temporaries can't bind to cv-refs, but they will take lvalues of any cv-qualification. Not all compilers adhere to the rule that temporaries can't bind to cv-refs, though. (and we can take advantage of this case too! see later on) Luckily, completely-generic functions are always an exact match. Unluckily, they have other issues. In particular, they don't play nicely with inheritance heirarchies where more than one type inherits from movable<T>: struct A : movable<A> {}; struct B : movable<B>, A {}; template <typename T> void f(const movable<T>&); If you call f with an A instance, everything's fine. If you call it with a B instance, though, type deduction is going to fail because T can be deduced to either A or B, and the language doesn't prefer to shorter path. Sadly, I can't see a simple way around this. I have somewhat of a solution, but it scales so badly that I don't want to propose it. The best that can be done here is to fall-back to the lvalue overload in case type-deduction fails. That being said, there are plenty of examples of top-level types (that you wouldn't noramlly inherit from) that have move semantics and should work fine with this (i.e. vector, string, etc...). Lastly, here's my favorite trick for non-intrusive rvalue detection, that relies on broken compilers that bind temporaries to cv-refs: template <typename T> void f(T& lvalue); template <typename T> void f(const volatile T& rvalue); As simple as that! It's only symptom is that it considers const-volatile lvalues as rvalues. However, const-volatile non-POD objects, which have special move semnatics, is something I haven't seen, so I'm not sure it's a real issue (no harm in considering a volatile int an rvalue, because it has no consequences. It will lose it's volatility in the context of the function, though). I've tried it on a few compilers. The interesting ones are those that don't already have true rv-refs: it works on MSVC 9 and Sun (latest), and fails on GCC 3.4 (and since this is the correct behavior, I suppose that on any newer GCC too). It would be nice if people with access to other compilers without rv-refs give it a go and report the results. Besides implictly catching rvalues, hiding the functions behind a macro solves another safety issue with "raw" BOOST_RV_REFs (which at the very least should be mentioned in the documentation, I don't think it currently is). BOOST_RV_REF parameters "maintain their rvalue-ness" in the context of the function. For example: void danger(const X&); void danger(BOOST_RV_REF(X)); void naive(BOOST_RV_REF(X) x) { danger(x); } Will call the rv-ref overload on emulation mode, where it should really call the lvalue one. This is dangerous, and it doesn't happen with the macro-ed functions. I hope this isn't too vague and unhelpful :) that's all I have time for ATM. I'm planning to post some actual code, but this might take a while.
On Thu, Jan 26, 2012 at 9:40 AM, Dan Ivy <danivy.mail@gmail.com> wrote:
Sorry for taking so long to respond, I'm a little short in time lately. Unfortunately, I didn't get to this yet, so I'm just going to describe more-or-less where it's standing and, in particaular, what the outstanding issues are, so people could bring up comments or suggestions.
[...] This is all pretty interesting, and much different from the solution I have in mind. I, too, haven't had a convenient block of time to hash out a full example, but it's on my TODO list :/
Besides implictly catching rvalues, hiding the functions behind a macro solves another safety issue with "raw" BOOST_RV_REFs (which at the very least should be mentioned in the documentation, I don't think it currently is). BOOST_RV_REF parameters "maintain their rvalue-ness" in the context of the function. For example:
void danger(const X&); void danger(BOOST_RV_REF(X));
void naive(BOOST_RV_REF(X) x) { danger(x); }
Will call the rv-ref overload on emulation mode, where it should really call the lvalue one. This is dangerous, and it doesn't happen with the macro-ed functions.
I *thought* this had been mentioned somewhere, but I must be misremembering because a quick glance through the documentation didn't yield any hits :/ I agree that it should probably be listed among the other limitations. - Jeff
I've attached a sketch implementation of the idea for simple functions. It doesn't deal with function-templates yet, but might still be interesting. The tar contains two sample programs: 'or_test.cpp' is an overload-resolution test, and 'string_cat.cpp' is a small (and pretty contrived :) benchmark of std::string's operator+ against a rvalue-aware version of it, operator&. Both programs should just compile and run, no need to link against anything unusual. As far as the implementation goes, the interesting stuff are in 'move/move.hpp'. It's quite unparsable, however, as much of the code is burried under heaps of PP junk. Below is a brief description of the additions. Don't get too picky about the macro syntax, I'm sure it can be improved. Any comments are welcome. The main addition is the BOOST_MOVE_FUNCTION macro. It has the form: BOOST_MOVE_FUNCTION(return_type, name, params) If 'return_type' contains any commas, or begins with a double-colon, it should be wrapped in parantheses. 'name' should be a simple identifier (that is, not an operator), and 'params' is a PP sequence of elements of the form: kind (type) name, where 'kind' is either nothing for simple parameters (i.e., an int), 'lv' for lvalue-refs, 'const' for const-lvalue-refs, or 'rv' for rvalue-refs. 'type' is the type of the parameter, possibly containing commas, and 'name' is it's name. There's no support for default arguments ATM: BOOST_MOVE_FUNCTION(void, f, ((int) x)) === void f(int x) BOOST_MOVE_FUNCTION(void, f, (lv (int) x)) === void f(int& x) BOOST_MOVE_FUNCTION(void, f, (const (int) x)) === void f(const int& x) BOOST_MOVE_FUNCTION(void, f, (rv (int) x)) === void f(int&& x) The main thing to keep in mind is that overload sets defined using BOOST_MOVE_FUNCTION need to be "complete": For every overload that takes a 'const' parameter, there has to be an equivalent overload with the same parameter as an 'rv', and vice-versa. 'lv' overloads are optional. If you want to take a parameter by ref-to-const without having to write a rv-ref overload, specify it as a simple parameter (that is, '(const int&) x' instead of 'const (int) x'). BOOST_MOVE_FUNCTION has additional forms that take up to 3 extra parameters, and are distinguished by their suffixes: N for nickname, P for prefix and S for suffix. Any combination of suffixes is possible but they have to be in the above order. The placement of the extra parameters with respect to the rest is the same for all forms, so I'm just going to list the most general one: BOOST_MOVE_FUNCTION_NPS(nickname, prefix, return_type, name, params, suffix) 'nickname' should be used when name is not an identifier (that is, an operator), and should itself be a simple identifier. Functions with different names should have different nicknames. Furthermore, the forms of BOOST_MOVE_FUNCTION that don't take a nickname internally use the function's name as the nickname, so it should be unique with respect to names of these functions as well. If 'name' contains commas (operator ,) it should be wrapped in parantheses. 'prefix' is added before the return type (i.e. inline, static, etc...). The only requirement is that if 'inline' is one of the specifiers in 'prefix', it should come first. 'suffix' comes after the parameters list (i.e. const, throw(), etc...). BOOST_MOVE_FUNCTION should only be used for free-functions. Member-functions should use BOOST_MOVE_MEMBER_FUNCTION, which is otherwise identical. Constructors are not supported ATM. For separate declaration and definition, use BOOST_MOVE_FUNCTION (or MEMBER_FUNCTION) for the declaration, followed by a semi-colon. Then use BOOST_MOVE_FUNCTION_DEFINITION for the definition, which has the same form of BOOST_MOVE_FUNCTION. BOOST_MOVE_MEMBER_FUNCTION_DEFINITION takes an aditional 'class_name' parameter (possibly wrapped in parantheses) before 'name'. BOOST_MOVE_FUNCTION, as a declaration, can appear at most once for each overload, even at namespace scope. User-defined conversions work as expected. Furthermore, they're dispatched to the correct overload (to the lvalue overload if they yield a lvalue, and to the rvalue one if they yield a rvalue). However, the ability to correctly determine the l/r-valueness of the result of a user-defined conversion requires the compiler to be conformant enough. In case the compiler isn't capable of that, all user-defined conversions are considered to yield lvalues.
On Wed, Feb 1, 2012 at 6:24 PM, Dan Ivy <danivy.mail@gmail.com> wrote:
I've attached a sketch implementation of the idea for simple functions. It doesn't deal with function-templates yet, but might still be interesting. [...]
Ping... By the lack of response, I take it that there's no interest in this?
on Fri Feb 10 2012, Dan Ivy <danivy.mail-AT-gmail.com> wrote:
On Wed, Feb 1, 2012 at 6:24 PM, Dan Ivy <danivy.mail@gmail.com> wrote:
I've attached a sketch implementation of the idea for simple functions. It doesn't deal with function-templates yet, but might still be interesting. [...]
Ping... By the lack of response, I take it that there's no interest in this?
I was interested, but IIRC it was TL;DR If you can boil it down, maybe it's worth a second look. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Sun, Feb 12, 2012 at 3:34 PM, Dave Abrahams <dave@boostpro.com> wrote:
on Fri Feb 10 2012, Dan Ivy <danivy.mail-AT-gmail.com> wrote:
On Wed, Feb 1, 2012 at 6:24 PM, Dan Ivy <danivy.mail@gmail.com> wrote:
I've attached a sketch implementation of the idea for simple functions. It doesn't deal with function-templates yet, but might still be interesting. [...]
Ping... By the lack of response, I take it that there's no interest in this?
I was interested, but IIRC it was TL;DR If you can boil it down, maybe it's worth a second look.
-- Dave Abrahams BoostPro Computing http://www.boostpro.com
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Well, that message contained a short attached sample program, which is pretty much as to-the-point as it gets. Maybe the attachment didn't get through?
on Sun Feb 12 2012, Dan Ivy <danivy.mail-AT-gmail.com> wrote:
On Sun, Feb 12, 2012 at 3:34 PM, Dave Abrahams <dave@boostpro.com> wrote:
on Fri Feb 10 2012, Dan Ivy <danivy.mail-AT-gmail.com> wrote:
On Wed, Feb 1, 2012 at 6:24 PM, Dan Ivy <danivy.mail@gmail.com> wrote:
I've attached a sketch implementation of the idea for simple
functions. It doesn't deal with function-templates yet, but might still be interesting. [...]
Ping... By the lack of response, I take it that there's no interest in this?
I was interested, but IIRC it was TL;DR If you can boil it down, maybe it's worth a second look.
-- Dave Abrahams BoostPro Computing http://www.boostpro.com
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Well, that message contained a short attached sample program, which is pretty much as to-the-point as it gets. Maybe the attachment didn't get through?
A. I didn't notice the attachment B. Also TL;DR. There are lots of files there; I don't know where to look -- Dave Abrahams BoostPro Computing http://www.boostpro.com
participants (10)
-
Anthony Williams -
Dan Ivy -
Dave Abrahams -
Edward Diener -
Eric Niebler -
Ion Gaztañaga -
Jeffrey Lee Hellrung, Jr. -
Vicente Botet -
Vicente BOTET
-
Vicente J. Botet Escriba