Andrea Torsello <torsello@dsi.unive.it> wrote:

...

Can you please re-run the tests? Thank you.

Here you have. Best regards. B. MSVC71 =============================================== Constructors -- Copy non-const lvalue non-const lvalue copy contructor -- Copy const lvalue const copy constructor -- Copy non-const rvalue non-const rvalue move constructor non-const rvalue move constructor -- Copy const rvalue non-const rvalue move constructor const copy constructor Pass by const ref -- Pass non-const lvalue -- Pass const lvalue -- Pass non-const rvalue non-const rvalue move constructor -- Pass const rvalue non-const rvalue move constructor Pass by value -- Pass non-const lvalue non-const lvalue copy contructor -- Pass const lvalue explicit copy constructor -- Pass non-const rvalue non-const rvalue move constructor -- Pass const rvalue non-const rvalue move constructor Discriminate -- Pass non-const lvalue discriminate: non-const lvalue -- Pass const lvalue discriminate: const value -- Pass non-const rvalue -- Pass const rvalue non-const rvalue move constructor discriminate: const value Comeau 4.3.3 relaxed mode (backend=MSVC71) =============================================== Constructors -- Copy non-const lvalue non-const lvalue copy contructor -- Copy const lvalue explicit copy constructor -- Copy non-const rvalue -- Copy const rvalue Pass by const ref -- Pass non-const lvalue -- Pass const lvalue -- Pass non-const rvalue -- Pass const rvalue Pass by value -- Pass non-const lvalue non-const lvalue copy contructor -- Pass const lvalue const copy constructor -- Pass non-const rvalue -- Pass const rvalue const copy constructor Discriminate -- Pass non-const lvalue discriminate: non-const lvalue -- Pass const lvalue discriminate: const value -- Pass non-const rvalue -- Pass const rvalue discriminate: const value Comeau 4.3.3 strict mode =============================================== Constructors -- Copy non-const lvalue non-const lvalue copy contructor -- Copy const lvalue const copy constructor -- Copy non-const rvalue non-const rvalue move constructor non-const rvalue move constructor -- Copy const rvalue non-const rvalue move constructor const copy constructor Pass by const ref -- Pass non-const lvalue -- Pass const lvalue -- Pass non-const rvalue non-const rvalue move constructor -- Pass const rvalue non-const rvalue move constructor Pass by value -- Pass non-const lvalue non-const lvalue copy contructor -- Pass const lvalue const copy constructor -- Pass non-const rvalue non-const rvalue move constructor non-const rvalue move constructor -- Pass const rvalue non-const rvalue move constructor const copy constructor Discriminate -- Pass non-const lvalue discriminate: non-const lvalue -- Pass const lvalue discriminate: const value -- Pass non-const rvalue -- Pass const rvalue non-const rvalue move constructor discriminate: const value

"Andrea Torsello" <torsello@dsi.unive.it> writes:

I have attached a new version with an easyer output to parse. It also changes the constructors from the form X x2(x1); to X x2=x1;

You can't "just do that". Both kinds of ctors are significant. One is direct initialization and the other is copy initialization, and they interact differently with the explicit keyword. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

David Abrahams wrote:

"Andrea Torsello" <torsello@dsi.unive.it> writes:

...

Hi, I have been investigating move semantics and think that I came up with a solution that is strongly influenced by both David Abrahams' move approach and Andrei Alexandrescu's mojo, but solves some of the problems I have encountered with them. The basic idea is to use the explicit keyword to force the compiler to use the X(X const &) constructor only in two cases. 1) When explicitly copying a const object i.e. X const x; X y(x); //use explicit copy constructor

Not when passing a non-const lvalue?

X x; X y(x); // implicitly uses a move ctor?? That's highly undesirable!

Actually, as described in a folowup, the approach works only with copy initialization, not with direct initialization. Direct initialization always uses the X(X const&) copy constructor. Not a major drawback in my use-cases, but your milage may vary. When using the copy initialization X x; X y=x; the X(X&) constructoor is used.

...

2) When passing by const ref void f(X const &); X makeX(); void foo() { f(makeX()); //use explicit copy constructor } In all other cases the other constructor take precedence. Which "other constructor"?

Should have been other constructorS. i.e. X(X&) for non-const lvalues, X(X::constant) for constants and X(X::temporary) for non-const rvalues.

...

Note that the template trick David Abrahams uses in his move approach would lead to an error on a conforming compiler when confronted with case #2 (unless more recent versions corrected this).

I'm not sure what you mean, but AFAICT case #2 is handled just fine by conforming compilers. Did you read http://anubis.dkuug.dk/jtc1/sc22/wg21/docs/papers/2004/n1610.html? It's a question of interpretation of a particular clause of the standard.

Not sure about the standard's wording, but Comeau in strict mode fails to compile the test you provided on this list some time ago. http://lists.boost.org/MailArchives/boost/msg59826.php Furthermore in the text of that test you added a comment after test 10, and I quote "// This one fails in Comeau's strict mode due to 12.2/1. GCC 3.3.1 passes it erroneously." Finally the need for X(X const &) to pass a temporary by const ref was the reason that forced Alexandrescu to add the special return type to mojo.

...

I chose to keep the temporary and constant subtypes from mojo because they allow me to discriminate between const values, non-const rvalues and non-const rvalues.

const and non-const rvalues are pretty much interchangeable, since the compiler can turn the former into the latter. If they weren't interchangeable, I don't see any reason you'd want to treat a const rvalue differently from a const lvalue.

...

The lack of this capability was the major problem I had with David Abrahams' move.

Why?

Passing by value is fine if you always move from temporaries and copy from non-temporaries: you pass by value and move from the passed object. If, like in the case of my vector-like container, you don't want to change the backing store unless you have to, the pass-by-value-and-move approach is not feasible

My solution allows discrimination between temporaries and non-temporaries.

It certainly does on copying, but does it allow to pick different overloaded functions when passing a temporary and a non-temporary? The code snipped you sent on the quoted message doesn't appear to allow it, but as I said, I am not sure about newer versions.

...

My solution fullfills almost all the requirements I had: 1) passing by value moves non-const rvalues and copies everithing else. 2) its is transparent to the user of the class. Alexandrescu's Mojo fails this because of the special return type, while Abrahams' move fails it because of the pass-by-const-ref problem described above.

Please describe the problem in more detail.

...

3) it allows to discriminate between constants, non-const lvalues, and non-const rvalues. Alexandrescu's Mojo allows it, while Abrahams' move does not.

Why is that an advantage?

I Hope I explained the points clearly enough, but please contact me if you have any doubt criticism or enquiry. I value all the feedback you can provide. Thanks a lot Andrea Torsello

"David Abrahams" <dave@boost-consulting.com> writes:

...

...

I'm not sure what you mean, but AFAICT case #2 is handled just fine by conforming compilers. Did you read http://anubis.dkuug.dk/jtc1/sc22/wg21/docs/papers/2004/n1610.html?

So did you?

Yes.

The comment was wrong; either behavior could be interpreted as correct if you stretch, but Comeau's is arguably too restrictive. Unfortunately there's no macro we can use to detect strict mode on EDG; it might mean that the implicit move optimization needs to be turned off for certain cases with those compilers.

Correct me if I am wrong, but if both behaviour could be interpreted as correct based on the current standard, it means that relying on one interpretation is undefine behaviour. At least untill a TC is issued. Besides it is not the only problem. I didn't just start to create my own version for pleasure. I started using your approach but the templated constructor had weird interactions with other 2 parameter constructors. I cannot remember the case off hand, but I will look for it and post it.

There is never a need to pass a temporary by const ref if you can move it.

What if the user of the library wants to pass the object to a function that has that signature? One of the main goal is for the move infrastructure to be as transparent as possible.

Yes.

template <class T> enable_if_same<T const, X const, void> f(T&); // lvalues

void f(move_from<X>); // temporaries

Right, didn't realize that you could use the same trick here, good stuff! Anyway, using enable_if_same as a result type convertible to the required type might cause problem with template type deduction if the result is passed to a templated function, while if it is used as a second paramater with default value (as in the constructor) it changes the arity of the function, which might cause problem with functional programming (this one is probably not a real problem). Anyway forcing an otherwise non-templated fiunction to be templated, while not a real problem is not really optimal, especially considering the difficulty most compilers have with the export keyword. Of corse, the templated class might just delegate all work to a non templated class, but still.

...

The code snipped you sent on the quoted message doesn't appear to allow it, but as I said, I am not sure about newer versions.

Why don't you do some research? It's all in the sandbox.

I did, and I proposed an alternative which has some advantages and some disadvantages. If you like i,t please use it, if you don't, don't! I was looking for feedback and constructive criticism, not forcing my approach to anyone.

I still don't understand why you think it's important to distinguish const temporaries from const lvalues.

As you said in another post, it is not a problem to move from a const temporary (the compiler might just copy the const temporary to a non const temporary and optimize the copy away), hece you want to move from const temporaries. On the other hand, you should certainly not move from lvalues. Best regards Andrea Torsello

Andrea Torsello wrote:

template<class U> enable_if_different<Tp, U, X<Tp>&> operator = (X<U> const &);

where enable_if_different<Tp, U, X<Tp>&> converts to X<Tp>&, but then it would interfere with template type deduction when passing the result to a templated function.

No it wouldn't. It would be written: template<class U> typename enable_if_different<Tp, U, X<Tp>&>::type operator = (X<U> const &); -- Daniel Wallin

"Andrea Torsello" <torsello@dsi.unive.it> writes:

Correct me if I am wrong, but if both behaviour could be interpreted as correct based on the current standard, it means that relying on one interpretation is undefine behaviour. At least untill a TC is issued.

Well, it's not undefined behavior. At best it's unspecified. IMO with a careful reading of the standard, the interpretation taken by Comeau is just wrong. What the standard says is really irrelevant, though. What matters are the status quo today and what will happen in the future. Today, compilers all act differently. In the future we have the assurance of the CWG that my move example from N1610 will work, via one of several possible routes. By the way, if you are willing to give up on move semantics from direct initialization, my technique will work in Comeau strict mode just as well as yours. Nothing about what you did will get around that problem; it seems to be a fact of current EDG compilers that in strict mode direct initialization from an rvalue will only ever use an ordinary (T const&) copy ctor. The current version in the sandbox turns off the move optimization for that case on current EDG in non-microsoft mode. If you're not compiling in strict mode you can turn it back on with -DBOOST_IMPLICIT_MOVE_CTOR_FOR_COPYABLE_TYPES. In fact, compilers in general are good enough at copy elision that the move ctor optimization only turns out to be useful for one other compiler I can find: Intel 8.0 for Windows, where it works silently and is enabled by default. I haven't tried /Za mode but apparently that's useless for building real software on Windows.

Besides it is not the only problem. I didn't just start to create my own version for pleasure. I started using your approach but the templated constructor had weird interactions with other 2 parameter constructors. I cannot remember the case off hand, but I will look for it and post it.

Please do. Also please use the latest version of the code in the sandbox to test against.

...

There is never a need to pass a temporary by const ref if you can move it.

What if the user of the library wants to pass the object to a function that has that signature? One of the main goal is for the move infrastructure to be as transparent as possible.

I'm sorry I was unclear. My library allows you to pass a temporary by const ref transparently. You said:

Finally the need for X(X const &) to pass a temporary by const ref was the reason that forced Alexandrescu to add the special return type to mojo.

Actually I'm not sure what I was trying to say anymore. I can state that my library already allows sink(source()); to work transparently with move semantics on all compilers I've tested on.

...

Yes. template <class T> enable_if_same<T const, X const, void> f(T&); // lvalues void f(move_from<X>); // temporaries

Right, didn't realize that you could use the same trick here, good stuff! Anyway, using enable_if_same as a result type convertible to the required type

Sorry, that wans just a mistake; it isn't convertible to anything. I meant: template <class T> // lvalues typename enable_if_same<T const, X const, void>::type f(T&); void f(move_from<X>); // temporaries

Anyway forcing an otherwise non-templated fiunction to be templated, while not a real problem is not really optimal, especially considering the difficulty most compilers have with the export keyword. Of corse, the templated class might just delegate all work to a non templated class

Exactly. Or you could just use explicit instantiation. BTW I assume you mean "templated function".

, but still.

It's no worse a problem than replacing what should be a reference paramter with another type that can be constructed by implicit conversion. You still cut off the possibility of user-defined conversions.

...

...

The code snipped you sent on the quoted message doesn't appear to allow it, but as I said, I am not sure about newer versions.

Why don't you do some research? It's all in the sandbox.

I did

I meant "research into newer versions". If you had done that, you could be sure of it. If you run the latest tests with --verbose-test on the bjam command-line, it will show you exactly what's allowed and how many unneccessary copies are made with your compiler.

, and I proposed an alternative which has some advantages and some disadvantages.

I don't see any advantages yet.

If you like i,t please use it, if you don't, don't! I was looking for feedback and constructive criticism, not forcing my approach to anyone.

I didn't take it that way. I'm not arguing with you out of defensiveness. If you really think what you did had some advantages, I want to be convinced of them so I can change my library to be the best possible.

...

I still don't understand why you think it's important to distinguish const temporaries from const lvalues.

As you said in another post, it is not a problem to move from a const temporary (the compiler might just copy the const temporary to a non const temporary and optimize the copy away), hece you want to move from const temporaries.

Which my library does.

On the other hand, you should certainly not move from lvalues.

OK, but in that case I don't see why you want to distinguish const temporaries from non-const temporaries. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

"Andrea Torsello" <torsello@dsi.unive.it> writes:

"David Abrahams" <dave@boost-consulting.com> writes:

...

By the way, if you are willing to give up on move semantics from direct initialization, my technique will work in Comeau strict mode just as well as yours. Nothing about what you did will get around that problem; it seems to be a fact of current EDG compilers that in strict mode direct initialization from an rvalue will only ever use an ordinary (T const&) copy ctor.

tested void sink(X const &); X source(); void foo() { sink(source()); } on Comeau strict does not compile... everywhere else it does.

My point is that if you disable the library mechanisms that make direct initialization move instead of copy from rvalues: X a(source()); // a case you're explicitly not handling then the sink(source()) example compiles in strict mode. The default setting of my library is now configured that way for comeau.

...

Please do. Also please use the latest version of the code in the sandbox to test against.

Last test was a couple of months ago when I was working activly on this, will do on most recent version I can find

IIRC I hadn't even posted my thing a couple of months ago(?)

As I said on another message, my moving class was parametrized on templated type Tp and had a converting constructor and operator =

template<class U> explicit X(X<U> const &);

there is the solution

template<class U> explicit X(X<U> const &, enable_if_different<U,Tp,char>::type=0);

but adds extra requirements on unrelatd features. The same applies to the enable_if_same<T const, X const, void>::type return type for non temporary overloaded functions. It looks like my approach doesn't do anything your approach cannot do (not surprisingly since the only difference is in the technique I use to make X(X const &) less "priviledged" than the other constructors), but it appears to have less interactions with other language features. And hence less requirements on the library user.

Can you please show a succinct example of "less interactions" so that I can do a comparison?

...

Actually I'm not sure what I was trying to say anymore. I can state that my library already allows

sink(source());

to work transparently with move semantics on all compilers I've tested on.

as stated before if sink is "void sink(X const &);" Comeau in strict mode complains that there is no X(X const &) constructor.

As stated before, no it doesn't. Please test my code first and make claims about it later.

...

Sorry, that wans just a mistake; it isn't convertible to anything. I meant:

template <class T> // lvalues typename enable_if_same<T const, X const, void>::type f(T&);

void f(move_from<X>); // temporaries

Oh, OK that would work with operators as well, still with my approach it would be:

void f(X&); void f(X::constant); void f(X::temporary);

which, to me, seems easier from the user perspective. Clearly, this is just a matter of taste.

Not entirely. Your approach entails some code duplication between the first two overloadsq. I guess the first f can be: void f(X& x) { X::constant(x); } But then what happens when f takes two or three arguments, both of which might be movable rvalues? I see an exponential explosion of overloads coming.

...

It's no worse a problem than replacing what should be a reference paramter with another type that can be constructed by implicit conversion. You still cut off the possibility of user-defined conversions.

Yes. I was not claiming that forcing a templated function was an unsurmoutable problem, obviously there are ways aroud it. But think from the perspective of the user: even if you hide the details of the function definition with macro tricks, suddenly the user cannot keep the function on a separate compilation unit unless the macro manages to perform the explicit instantiation as well, which would be hard since you should explicitly instantiate two specializations: one for type X and one for type X const.

What macro tricks? I only use those for the copy ctor and assignment operator because those may otherwise entail duplicating the function body.

...

I don't see any advantages yet.

Nothing your approach doesn't allow with some work (except Comeau not allowing to pass by const reference),

I'll ignore that one.

but less interactions with orthogonal language features the user should know about.

I'm still interested in seeing how that happens.

The only problem I have encountered so far comes from X x(makeX()); calling the explicit copy constructor (X x=makeX(); does not). While in this case the approach fails to move, it is by far the easyest case for compiles to perform return value optimization.

I'm afraid not...

The only compiler I have seen not performing the optimization is Comeau in strict mode.

...and hence, all the EDG-based compilers do the same thing unless they're trying to emulate microsoft.

...

...

...

I still don't understand why you think it's important to distinguish const temporaries from const lvalues.

As you said in another post, it is not a problem to move from a const temporary (the compiler might just copy the const temporary to a non const temporary and optimize the copy away), hece you want to move from const temporaries.

Which my library does.

Never claimed it does not.

Sorry, you wrote: "I chose to keep the temporary and constant subtypes from mojo because they allow me to discriminate between const values, non-const rvalues and non-const rvalues. The lack of this capability was the major problem I had with David Abrahams' move." I'm still trying to understand why it's a problem. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

"David Abrahams" <dave@boost-consulting.com> writes:

...

"David Abrahams" <dave@boost-consulting.com> writes:

My point is that if you disable the library mechanisms that make direct initialization move instead of copy from rvalues:

X a(source()); // a case you're explicitly not handling

then the sink(source()) example compiles in strict mode. The default setting of my library is now configured that way for comeau.

OK

...

...

Please do. Also please use the latest version of the code in the sandbox to test against.

Last test was a couple of months ago when I was working activly on this, will do on most recent version I can find

IIRC I hadn't even posted my thing a couple of months ago(?)

Your post was Feb 6th, I was working on this first half of February.

Can you please show a succinct example of "less interactions" so that I can do a comparison?

1) You need to treat the templated conversion-copy-constructor I was talking about differently: add a second default parameter with enable_if_different or something like that. 2) When overloading a function between temporaries and non-temporaries you have to use the enable_if_equal trick on return values while the changes with my approach are restricted to the passed types 3) Non temporaries overloads must be templated. If you want to provide explicit overload on another compilation unit you must explicitly overload for both X and X const. While all these are doable, they expose a lot of details of the internals of the library to the user. With my approach The only requirement for #1 is that the conversion-copy-constructor be explicit (something that you better do anyway). If you really want to keep the converting constructor implicit, then you have to use the enable_if_different trick.

...

...

Actually I'm not sure what I was trying to say anymore. I can state that my library already allows

sink(source());

to work transparently with move semantics on all compilers I've tested on.

as stated before if sink is "void sink(X const &);" Comeau in strict mode complains that there is no X(X const &) constructor.

As stated before, no it doesn't. Please test my code first and make claims about it later.

...

...

Sorry, that wans just a mistake; it isn't convertible to anything. I meant:

template <class T> // lvalues typename enable_if_same<T const, X const, void>::type f(T&);

void f(move_from<X>); // temporaries

Oh, OK that would work with operators as well, still with my approach it would be:

void f(X&); void f(X::constant); void f(X::temporary);

which, to me, seems easier from the user perspective. Clearly, this is just a matter of taste.

Not entirely. Your approach entails some code duplication between the first two overloadsq. I guess the first f can be:

void f(X& x) { X::constant(x); }

Actually "void f(X& x) { f(X::constant(x)); }", but yes.

But then what happens when f takes two or three arguments, both of which might be movable rvalues? I see an exponential explosion of overloads coming.

So does your approach. base 2 exponential explosion instead of base 3, but exponential explosion all the same. Furthermore, if you want to provide the explicit instantiation on a separate compilation unit, you wold have a base 3 explosion since you wold have 2 instantiation for each non-temporary type. Furthermore, the higher the number of arguments that need to be differentiated, the longer and uglier the return type would be. Note that if you don't need to differentiate, you can just pass by value and trust the copy constructor to move temporaries.

...

Yes. I was not claiming that forcing a templated function was an unsurmoutable problem, obviously there are ways aroud it. But think from the perspective of the user: even if you hide the details of the function definition with macro tricks, suddenly the user cannot keep the function on a separate compilation unit unless the macro manages to perform the explicit instantiation as well, which would be hard since you should explicitly instantiate two specializations: one for type X and one for type X const.

What macro tricks? I only use those for the copy ctor and assignment operator because those may otherwise entail duplicating the function body.

I am not saying that you are using macro tricks. I am saying that the way you differentiate between temporaries and non-temporaries on function overload is a bit cumbersome in the non-temporary case and exposes a lot of details of the approach (templated code, enable_if_equal on the return type etc..). You might try to hide all this with macro trickery to make it easier on the user, but then you wold have to figure out how to explicitly instantiate the function for both X and X const. The mechanism behind the move abstraction leaks out to the user in a lot of cases.

Sorry, you wrote:

"I chose to keep the temporary and constant subtypes from mojo because they allow me to discriminate between const values, non-const rvalues and non-const rvalues. The lack of this capability was the major problem I had with David Abrahams' move."

I'm still trying to understand why it's a problem.

As a mater of fact I realized that with explicit instantiation of the non-temporary function overload your approach can still differentiate between const and non-const lvalues: non-const lvalues would pick the X overload, while the const lvalues would pick the X const overload. So this point is mute. Sorry. Regards Andrea Torsello

David Abrahams <dave@boost-consulting.com> writes:

So I'm wondering if there's a way to combine the advantages of your approach with those of mine.

The code in the sandbox now appears to be supporting both approaches. I've also added disable_if_same. Use the BOOST_LVALUE_COPY_CTOR2 and BOOST_LVALUE_ASSIGN2 macros to ^ ^ auto-generate constructor/assignment overloads taking const_lvalue<T> instead of using the template technique. Let's call that "technique 2". Findings -------- Aside from the easier time you'll have getting partial ordering of constructors and assignment operators with technique 2, I noticed the following: The use of the 'explicit' keyword for a T const& ctor overload seems to be completely unneccessary. All the optimizations you can get in intel's strict mode are available using an ordinary non-explicit copy ctor. Am I missing something? In non-strict mode (and on Intel 8/win32) technique 2 seems to generate copies when constructing from const rvalues whereas technique 1 does not. Technique 2 seems to require most compilers to go through move constructions in situations where technique 1 allows the compiler to elide copies. That could bring a significant speed penalty for some types, unless optimizers are smart enough to eliminate the code later. Not having a real T const& parameter for const lvalues makes writing overloads tricky. I was able to make the macros transparent for operator=, but because of initializer lists there's no way to do the same thing for constructors. const_lvalue<T> now has a conversion to T const&, so you can use boost::implicit_cast<T const&>(rhs) to get a reference to the argument with uniform syntax. Borland doesn't seem to be able to cope with technique 2. It complains that operator= is ambiguous. I didn't try very hard to work around the problem, though. Results ------- This code has been tested on win32 with the following compilers. Where the techniques differ, results are shown as #technique1/#technique2. como-win32 3 suboptimal copies in strict mode. With -DBOOST_IMPLICIT_MOVE_CTOR_FOR_COPYABLE_TYPES in non-strict mode, 0/1 suboptimal copies with 1/3 warnings. cwpro8 0/1 suboptimal copies gcc 2.95.3, 3.2, 3.3.1 0 suboptimal copies bcc564 2/? suboptimal copies <<< can't handle technique 2 yet >>> msvc 6, 7, 7.1 0 suboptimal copies (**) vc7 issues a couple of bogus warnings (C4927) intel8 0/1 suboptimal copies (**) intel5 intel6 intel7 2 suboptimal copies (**) (**) On these compilers, non-copyability isn't automatically inherited from movable<T>, i.e. the boost::noncopyable trick doesn't work. Rvalues are silently bound to non-const references. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Also tested now on Linux: gcc 3.2.2, 3.4.0-20031231(experimental) - 0 suboptimal copies intel 7, 8 - 0/1 suboptimal copies David Abrahams <dave@boost-consulting.com> writes:

Results -------

This code has been tested on win32 with the following compilers. Where the techniques differ, results are shown as #technique1/#technique2.

como-win32 3 suboptimal copies in strict mode. With -DBOOST_IMPLICIT_MOVE_CTOR_FOR_COPYABLE_TYPES in non-strict mode, 0/1 suboptimal copies with 1/3 warnings.

cwpro8 0/1 suboptimal copies

gcc 2.95.3, 3.2, 3.3.1 0 suboptimal copies

bcc564 2/? suboptimal copies <<< can't handle technique 2 yet >>>

msvc 6, 7, 7.1 0 suboptimal copies (**)

vc7 issues a couple of bogus warnings (C4927)

intel8 0/1 suboptimal copies (**)

intel5 intel6 intel7 2 suboptimal copies (**)

(**) On these compilers, non-copyability isn't automatically inherited from movable<T>, i.e. the boost::noncopyable trick doesn't work. Rvalues are silently bound to non-const references.

-- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Andrea Torsello <torsello@dsi.unive.it> writes:

No, in some implementations you can just not provide the explicit constructor at all (at least gcc works well without it). If you do that you do not have the direct initialization problem anymore. I used the explicit keyword to use my approach with compliers like Comeau that do not like if you do not provide a X(X const &) constructor. I actually omitted to say this on the first iteration of the sample code because I though I needed to focus on the general approach. In the library I use with gcc I simply do not provide the explicit copy ctor at all.

I really understood that's what you were saying. My point was that if you just leave out the fancy "move ctor" overloads and provide the X(X const&) ctor with Comeau, the RVOs kick in and you don't get any suboptimal copies... except when using direct initialization. Since your technique punts on moving in that case anyway, it seems like there's really zero advantage to using the explicit copy ctor at all.

...

In non-strict mode (and on Intel 8/win32) technique 2 seems to generate copies when constructing from const rvalues whereas technique 1 does not.

Yes, actually different compilers seem to act differently here. Since the compiler is free to copy const rvalues to non-const rvalues and optimize the copy away, I would consider returning a const temporary an odd thing to do. I am curious: does anyone have a use-case for it?

I doubt it; it doesn't seem like a very important case. It could be used to have some effects on overload resolution I suppose, but it's hard to imagine that being useful in real code.

...

Not having a real T const& parameter for const lvalues makes writing overloads tricky. I was able to make the macros transparent for operator=, but because of initializer lists there's no way to do the same thing for constructors. const_lvalue<T> now has a conversion to T const&, so you can use boost::implicit_cast<T const&>(rhs) to get a reference to the argument with uniform syntax.

I am not sure I understood what you are saying here. Can you rephrase it please?

The point is that if you want to use macros (or copy-and-paste) to generate the two overloads for (T& rhs) and (const_lvalue<T> rhs), you can have a problem trying to re-use the initializer list and body of the function because T& and const_lvalue<T> have different interfaces. With the appropriate conversion operator on const_lvalue, implicit_cast<T const&>(rhs) is the same thing in both cases and can be used to make initializer lists and bodies that can be reused. Technique 1 essentially uses (T& rhs) and (T const& rhs) signatures, so that's not a problem.

...

Results ------- [...]

We should probably try and not provide the explicit copy ctor with compilers that do not complain about it. I feel that this way some of the suboptimal copies will go away.

The suboptimal copies in my measurements for "technique 2" appear without any use of "explicit" on the copy ctor. Please, though, take a close look at what the code is doing to see if I've misinterpreted your intention. In particular, run the move.cpp test, then turn on -DBOOST_IMPLICIT_MOVE_CTOR_FOR_COPYABLE_TYPES and edit the code to make the ctor explicit. I think you'll see that it puts us back where we started. P.S. My tests are really the best part of what I've done on this; they reveal so much about what the compilers are doing, and tend to find problem cases by testing move-only types. I really want to encourage you to try whatever move ideas you may have in the testing framework I've set up. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

"Fredrik Blomqvist" <fredrik_blomqvist@home.se> writes:

David Abrahams wrote:

...

Andrea Torsello <torsello@dsi.unive.it> writes:

[snip]

...

...

Yes, actually different compilers seem to act differently here. Since the compiler is free to copy const rvalues to non-const rvalues and optimize the copy away, I would consider returning a const temporary an odd thing to do. I am curious: does anyone have a use-case for it?

I doubt it; it doesn't seem like a very important case. It could be used to have some effects on overload resolution I suppose, but it's hard to imagine that being useful in real code.

Just a comment on the 'const temporary return' issue. I'd say it's not really an issue of whether it's useful per se, but how it is _actually used_ in real code as a "safety" feature. I.e to prevent: (a*b) = c; Scott Meyers' Effective C++, item 21, labels this "The Right Thing To Do". I've seen similar recommendations in a couple of coding-guidelines and several times in real code, most likely inspired by EC++ or similar. Thus it will appear in real code if nothing else for a matter of style.

Whoops; good point! -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Andrea Torsello <torsello@dsi.unive.it> writes:

David Abrahams wrote:

...

Andrea Torsello <torsello@dsi.unive.it> writes:

...

No, in some implementations you can just not provide the explicit constructor at all (at least gcc works well without it). If you do that you do not have the direct initialization problem anymore. I used the explicit keyword to use my approach with compliers like Comeau that do not like if you do not provide a X(X const &) constructor. I actually omitted to say this on the first iteration of the sample code because I though I needed to focus on the general approach. In the library I use with gcc I simply do not provide the explicit copy ctor at all.

I really understood that's what you were saying. My point was that if you just leave out the fancy "move ctor" overloads and provide the X(X const&) ctor with Comeau, the RVOs kick in and you don't get any suboptimal copies... except when using direct initialization. Since your technique punts on moving in that case anyway, it seems like there's really zero advantage to using the explicit copy ctor at all.

Oh, OK, if you assume that the compiler can always perform return value optimization, you are right. But I feel there is a difference in requiring the compiler to perform RVO on direct initialization from temporaries and requiring the compiler to perform it every time it is possible.

You appear to be making an argument about what's theoretically going to work in correct, portable C++, based on the behavior of a particular compiler implementation. I am not requiring RVO unless you take what EDG accepts in strict mode as the defintion of "standards conforming". I do not. EDG is erroneous, IMO, in issuing that error, so we either turn off strict mode or work around it by allowing its native RVO to work. GCC has a much better example of what I consider a correct interpretation of the standard in requiring that it be possible to construct a new temporary rvalue from an existing one that will be bound to a reference (and not "copy construct", as I point out in N1610).

Using the explicit constructor technique #2 works even with compilers that do not support RVO, as long as you use the assignment constructor instead of the direct copy constructor.

Well, I challenge you to find a compiler that doesn't support at least some RVO. But if you did find one, technique #1 is as good as technique #2 in that respect. If you try the test with GCC and -DBOOST_IMPLICIT_MOVE_CTOR_FOR_COPYABLE_TYPES you'll see that technique 1 does 0 suboptimal copies and RVOs are suppressed, using move construction instead. If you try it with technique 2, but no explicit, and the same command-line you get 1 suboptimal copy (i.e. it works without explicit). If you then add explicit you get 4 suboptimal copies because direct initialization can no longer move. But who would want to suppress RVOs in favor of move construction, anyway?

You still need to convert to temporary in initializer lists, since you cannot use the assigment constructor there, but most likely you would have to do it anyway.

Is that just speculation? I can't understand why you'd make that claim, since the CWG thinks that a prohibition on copying rvalues is possible.

...

The point is that if you want to use macros (or copy-and-paste) to generate the two overloads for (T& rhs) and (const_lvalue<T> rhs), you can have a problem trying to re-use the initializer list and body of the function because T& and const_lvalue<T> have different interfaces. With the appropriate conversion operator on const_lvalue, implicit_cast<T const&>(rhs) is the same thing in both cases and can be used to make initializer lists and bodies that can be reused.

Oh, right! There is probably no way around explicitly writing both constructors

I just got finished describing how it could be done using implicit_cast. Did you not understand that?

but you can avoid duplication by putting the constructor's body in a private member function and forward from both constructors to it. At least that is what I have been doing.

If it were that easy you could do it with macros by inserting a surrounding brace pair and initializing a T const& there. Unfortunately, it doesn't handle the initializer list and I almost never have code in the ctor body. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Andrea Torsello <torsello@dsi.unive.it> writes:

David Abrahams wrote:

...

...

Oh, OK, if you assume that the compiler can always perform return value optimization, you are right. But I feel there is a difference in requiring the compiler to perform RVO on direct initialization from temporaries and requiring the compiler to perform it every time it is possible.

You appear to be making an argument about what's theoretically going to work in correct, portable C++, based on the behavior of a particular compiler implementation.

I am not requiring RVO unless you take what EDG accepts in strict mode as the defintion of "standards conforming". I do not. EDG is erroneous, IMO, in issuing that error, so we either turn off strict mode or work around it by allowing its native RVO to work. GCC has a much better example of what I consider a correct interpretation of the standard in requiring that it be possible to construct a new temporary rvalue from an existing one that will be bound to a reference (and not "copy construct", as I point out in N1610).

What I was saying is that if you provide a non-explicit X(X const &) constructor when passing a temporary by value you are performing a copy. So if you provide a non explicit copy ctor, you are requiring the compiler to perform RVO or you would have an extra copy. If you can assume the compiler can peform RVO in all cases why have the move construct at all?

Right. That said, for Intel C++ I am not providing any X(X const&) ctor, because it acts like comeau in non-strict mode (except that it doesn't warn either), and thus will optimize direct initializations when I am using the fancy move ctors.

...

But who would want to suppress RVOs in favor of move construction, anyway?

No-one, clearly. And Yes, all modern compilers support at least some RVO. I am just not sure whether this SOME is enough in all cases. I'd love to believe it is, but I am a bit scheptical.

My library is not relying on it being enough. It's just turning off the mechanisms needed for non-RVO'ing compilers where appropriate.

...

...

You still need to convert to temporary in initializer lists, since you cannot use the assigment constructor there, but most likely you would have to do it anyway.

Is that just speculation? I can't understand why you'd make that claim, since the CWG thinks that a prohibition on copying rvalues is possible.

I believe we are talking about two different things here. I was thinking about cases where you have a movable class with movable members.

class X : public movable<X> { ... };

class Y : public movable<y> { X x; public: Y(move_from<Y> y) : x(move_from<X>(y.x)) {} ^^^^^^^^^^^^^^^^^

clearly, "Y(move_from<Y> y) : x(y.x) {}" would not be able to move from x since in this context y.x is an lvalue.

That's what the explicit "move(x)" function is for: class Y : public movable<y> { X x; public: Y(move_from<Y> y) : x(move(y->x)) {}

...

...

Oh, right! There is probably no way around explicitly writing both constructors

I just got finished describing how it could be done using implicit_cast. Did you not understand that?

Sorry, didn't parse the last part correctly. You are right, if "const_lvalue" is convertible to T&, you can force the conversion in both initializer lists and still use a single macro. But it would still be the user's duty to explicitly force the conversion in initializer list, wouldn't it?

When required. Sometimes you can rely on the implicit conversion.

I am not sure wether it is easier on the user to have him force a conversion in a situation that he normally would not than just tell him that he needs two separate constructors, one for const and one non-const lvalues.

Considering that most/all of my ctors' logic is in the initializer list and that the two ctors' initializers will be different, I like having a way to make them the same.

...

...

but you can avoid duplication by putting the constructor's body in a private member function and forward from both constructors to it. At least that is what I have been doing.

If it were that easy you could do it with macros by inserting a surrounding brace pair and initializing a T const& there. Unfortunately, it doesn't handle the initializer list and I almost never have code in the ctor body.

Yes users would still have to provide the initializer lists.

Only one initializer list need be provided, using implicit_cast<> if neccessary.

Clearly delegating the construction is worth only if the constructor has a non-trivial body

Yep. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Andrea Torsello <torsello@dsi.unive.it> writes:

David Abrahams wrote:

...

My library is not relying on it being enough. It's just turning off the mechanisms needed for non-RVO'ing compilers where appropriate.

This is clearly the best thing, which leaves us where we do not provide any X(X const &) constructor (explicit or implicit) unless the compiler requires it.

Uh, no: when turning off the mechanisms needed for non-RVO'ing compilers we supply a non-explicit X(X const&) ctor, and no X(X&) ctor.

...

...

...

...

You still need to convert to temporary in initializer lists, since you cannot use the assigment constructor there, but most likely you would have to do it anyway. [...]

That's what the explicit "move(x)" function is for:

exactly, my point was that in initialization lists you probably need to convert to temporary with the move(x) function. By "convert to temporary" I meant the X::temporary type I had in my examples. The move_from<X> type in your code.

Right... but that "conversion to temporary" is not the same as *actually* converting to a C++ temporary, which involves making a copy.

...

Considering that most/all of my ctors' logic is in the initializer list and that the two ctors' initializers will be different, I like having a way to make them the same.

[...]

...

...

Yes users would still have to provide the initializer lists.

Only one initializer list need be provided, using implicit_cast<> if neccessary.

With complicated initializer lists, I agree that one doesn't want to duplicate the list. I guess that this is a personal coding-style issue.

Yes. And you can take either approach with both technique 1 and technique 2. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Andrea Torsello <torsello@dsi.unive.it> writes:

Sorry for the late reply: I have been away on Sunday. By the way, you guys are amazing:

I'm doubly flattered to be "pluralized" in that way! BTW, the message you're replying to is a pretty old message in the scheme of things; I've moved past much of what I said there.

I go away for one day and you have already inserted my alternative approach in the sandbox and thoroughly tested it. WOW!

It's more a product of an unhealthy obsession than anything you should be impressed by. Also, you may not believe that I got the alternative approach right, since I don't use explicit copy ctors. See my other message for an explanation, though.

David Abrahams wrote:

...

None of what you wrote below is "showing an example". I meant, please write down some code demonstrating that your technique "has less interactions with other language features". I'm not sure what "other" means in this case.

I meant that the user has to know less about the internals of the approach when dealing with templated constructors and/or functions that distinguish between temps and non-temps.

I agree.

Anyway, I am downloading the sandbox right now. I will creating a few tests and use cases to highlight the differences in usage patterns for the two approaches. If anything, this will be useful for documentation purpose. Will be back with as soon as my rent-paying job allows me to.

Looking forward to it.

...

...

1) You need to treat the templated conversion-copy-constructor I was talking about differently: add a second default parameter with enable_if_different or something like that.

OK, so I'll supply disable_if_same. I'm not sure where that leaves us, but I think my approach still handles more cases (including direct initialization on most compilers) with a similar amount of syntactic overhead.

Your approach handles direct initialization in more cases, since my approach pretty much gives up moving direct initialization and hopes the compiler can perform return value optimization. In defense of my method I must say that this case is arguably the simples RVO possible, which, unfortunately, doesn't mean that it actually is simple nor that all compiler get it right, as your tests clearly show.

I've been trying to convince the EDG guys that it's "obvious" that *if* you absolutely must choose between direct and copy initialization for the RVO, the intuitive choice is to go for direct initialization, because, well, it's "direct". That's the user perspective. The point is that from the point-of-view of a standards wonk and compiler implementor, there's no particular reason to think that RVO is easier in the direct case than in the other one.

As for the syntactic overhead, I believe that my approach has an edge here, since it minimizes the amount of adjustments the user has to go through in order to use it, but of course this is a matter of personal opinion and I am certainly biased ;)

I think you have a good point, actually. The two approaches each have their own strengths apparently, and I'm less convinced of the advantages of technique 1 than I used to be. OTOH, I'm not convinced there's any advantage at all to using an explicit copy ctor.

...

...

2) When overloading a function between temporaries and non-temporaries you have to use the enable_if_equal trick on return values while the changes with my approach are restricted to the passed types

What's the significance of that? It doesn't change the function's returned type.

No, but it is one more thing the user has to do differently when using the move library.

Yep.

...

...

3) Non temporaries overloads must be templated.

I don't see that as a liability when the alternative is code duplication or forwarding.

Template code can be arbitrarily long and impacts compile-time on each compilation unit using it, forwarding is one-line long, almost certainly inlined by optimizing compilers, and has negligible impact on compile-time.

You're right; that's a real issue. Also templates will usually generate duplicate code for const and non-const lvalues. If I were wedded to technique 1, I would consider doing: // f.hpp template <class T> typename enable_if_same<T const, X const,void>::type f(T& x); template <> inline void f(X const& x); template <> inline void f(X& x) { return f<X const>(x); } // f.cpp template <> void f(X const& x) { // whatever } Which I think should have an equivalent effect. Maybe a nicer enabler should be provided, that allows us to write: typename enable_lvalue<T, X, void>::type // no "const" The enablers get more interesting if you want to write a generic function over a templated type Y<U>: template <class T> typename enable_if<is_Y<T>,void>::type f(T& x); fortunately I have macros to generate is_Y (see boost/python/detail/is_xxx.hpp). On the other hand, with technique 2 I think you can't write the generic function at all, because template arguments have to match exactly: template <class T> void f(const_lvalue<Y<T> > x); // can never match a Y<T> argument

...

...

If you want to provide explicit overload on another compilation unit you must explicitly overload for both X and X const. ^^^^^^^^

No, that won't work. The rvalue will bind to the X const& parameter. It's as I said: you just omit the bodies and use explicit instantiation.

I was under the impression that template typee would always resolve to the same type that the templated code would instantiate, i.e. non-const lvalue would instantiate a X& version of the template function and hence it would bind only to a X& explicit instantiation.

I think you're possibly suggesting what I did in the example above? I must've misinterpreted you because technically there's no overload there.

...

...

While all these are doable, they expose a lot of details of the internals of the library to the user. With my approach The only requirement for #1 is that the conversion-copy-constructor be explicit (something that you better do anyway).

Which one is that, and why had you better do it anyway?

template <class U> X(X<U> const& other);

As for the fact that I believe that in most cases you should make it explicit: would you want a vector<int> be _IMPLICITLY_ convertible to a vector<double>?

Let's see, would you want a shared_ptr<Derived> to be _IMPLICITLY_ convertible to a shared_ptr<Base>? ;-) Seriously, most such converting ctors I've seen in the wild are of that nature. I don't think there's any way to generalize about whether explicit is usually needed.

You can always write the constructor as

template<class U> X(X<U> const &, typename enable_if_different<T,U>::type=0)

which would make it impossible for the constructor to resolve to X(X<T> const &)

...

You're right. That's why we need &&.

Couldn't possibly agree more or more emphatically!

And, let me point out, we need T&& with "perfect forwarding" argument deduction: T is U cv& when the argument is an lvalue, and plain U when the argument is an rvalue. The reason, of course, is that having an rvalue reference without it just compounds the forwarding problem because now we have 3 kinds of reference to consider: T&, T const& and T&&. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Daniel Wallin <dalwan01@student.umu.se> writes:

David Abrahams wrote: [snip]

...

On the other hand, with technique 2 I think you can't write the generic function at all, because template arguments have to match exactly: template <class T> void f(const_lvalue<Y<T> > x); // can never match a Y<T> argument

Isn't this true for technique 1 as well (and everything that isn't &&)?

template<class T> void f(move_from<T> x);

Granted, you can detect lvalues in a generic function, but that isn't of much use if you can't detect temporaries, no?

Are you ready for this? template <class T> void tlsink(T volatile& y) { T const& x = const_cast<T const&>(y); } template <class T> void tlsink(T x) { SAY("in templated move sink"); } This works with EDG. GCC isn't so happy with it. I'm not sure which one is right, but I'm voting for EDG. I wonder if we can get rid of the const_cast... -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

David Abrahams wrote:

Andrea Torsello <torsello@dsi.unive.it> writes:

...

Sorry for the late reply: I have been away on Sunday. By the way, you guys are amazing:

I'm doubly flattered to be "pluralized" in that way! BTW, the message you're replying to is a pretty old message in the scheme of things; I've moved past much of what I said there.

Well, I couldn't be sure it was just you, so I erred on the side of caution...

I've been trying to convince the EDG guys that it's "obvious" that *if* you absolutely must choose between direct and copy initialization for the RVO, the intuitive choice is to go for direct initialization, because, well, it's "direct". That's the user perspective. The point is that from the point-of-view of a standards wonk and compiler implementor, there's no particular reason to think that RVO is easier in the direct case than in the other one.

I ment that the recieiving side is easyer that the returning side: it is harder to make sure that a function that returns a temporary does not make a copy than it is to make sure that a direct or assignment constructor is elided when passed a temporary. From a user perspective, the choice of copy-initialization (direct or assignmetn) should make no difference to RVO, no matter what language lawyers say about it.

...

As for the syntactic overhead, I believe that my approach has an edge here, since it minimizes the amount of adjustments the user has to go through in order to use it, but of course this is a matter of personal opinion and I am certainly biased ;)

I think you have a good point, actually. The two approaches each have their own strengths apparently, and I'm less convinced of the advantages of technique 1 than I used to be. OTOH, I'm not convinced there's any advantage at all to using an explicit copy ctor.

As your tests point out, technique #1 might force less moves with compilers that are not too aggressive when optimizing. I wonder if the two approaches should coexist, at least for a while and let the users choose between syntactic semplicity and (possibly) higher efficiency.

Maybe a nicer enabler should be provided, that allows us to write:

typename enable_lvalue<T, X, void>::type // no "const"

The enablers get more interesting if you want to write a generic function over a templated type Y<U>:

template <class T> typename enable_if<is_Y<T>,void>::type f(T& x);

fortunately I have macros to generate is_Y (see boost/python/detail/is_xxx.hpp). On the other hand, with technique 2 I think you can't write the generic function at all, because template arguments have to match exactly:

template <class T> void f(const_lvalue<Y<T> > x); // can never match a Y<T> argument

Right. That is why in my test code I had to explicitly qualify disctiminate<int> I wonder whether we can do something similar with technique #2... Something else I should look into.

I think you're possibly suggesting what I did in the example above? I must've misinterpreted you because technically there's no overload there.

Exactly.

Let's see, would you want a shared_ptr<Derived> to be _IMPLICITLY_ convertible to a shared_ptr<Base>? ;-) Seriously, most such converting ctors I've seen in the wild are of that nature. I don't think there's any way to generalize about whether explicit is usually needed.

Point taken. Different backgrounds, different use-cases in mind.