[range] adaptors vs. rvalue to lvalue& binding
Hello, ideally I would like to use range-adapted lvalue ranges like any other lvalue ranges. Writing it down naively, as below, does not compile because LoadRefRange takes (and should take?) the range it modifies by lvalue&, while adaptor expressions are rvalues. template<typename Range> void modifies_range( Range& rng ); A a[5]; main() { // modifies all elements of a modifies_range ( a ); // modifies all elements (except for one) of a // adaptor is rvalue, does not compile: modifies_range ( boost::range::join( boost::adaptors::slice( a, 0, 1 ), boost::adaptors::slice( a, 2, countof(a) ) ) ); // compiles, but ugly: auto rng=boost::range::join( boost::adaptors::slice( a, 0, 1 ), boost::adaptors::slice( a, 2, countof(a) ) ); modifies_range(rng); } Any thoughts? Should adaptors receiving lvalues offer an operator adaptor&() &&; // overload on rvalue-ness so they are passing through the lvalue or their argument? A suitable baseclass a la boost::additive could do that. Regards, Arno -- Dr. Arno Schödl | aschoedl@think-cell.com Technical Director think-cell Software GmbH | Chausseestr. 8/E | 10115 Berlin | Germany http://www.think-cell.com | phone +49 30 666473-10 | US phone +1 800 891 8091 Amtsgericht Berlin-Charlottenburg, HRB 85229 | European Union VAT Id DE813474306 Directors: Dr. Markus Hannebauer, Dr. Arno Schoedl
On 22/03/2012 06:40, Arno Schödl wrote:
Hello,
ideally I would like to use range-adapted lvalue ranges like any other lvalue ranges. Writing it down naively, as below, does not compile because LoadRefRange takes (and should take?) the range it modifies by lvalue&, while adaptor expressions are rvalues.
template<typename Range> void modifies_range( Range& rng );
A a[5];
main() { // modifies all elements of a modifies_range ( a );
// modifies all elements (except for one) of a // adaptor is rvalue, does not compile: modifies_range ( boost::range::join( boost::adaptors::slice( a, 0, 1 ), boost::adaptors::slice( a, 2, countof(a) ) ) );
// compiles, but ugly: auto rng=boost::range::join( boost::adaptors::slice( a, 0, 1 ), boost::adaptors::slice( a, 2, countof(a) ) ); modifies_range(rng); }
Any thoughts? Should adaptors receiving lvalues offer an
operator adaptor&()&&; // overload on rvalue-ness
Hello you don't want to add such operator to any class, as it would trigger automatic conversion of rvalue of given type to lvalue. Sometimes this is not right thing to do, but with added operator you will have no means of disabling this behaviour. Apart from that, ref-qualifiers unfortunately are not yet widely implemented and such code won't be very portable anyway (I only know of 2 compilers implementing this feature). What you want to do instead is to change the signature of your function to accept both lvalue and rvalue reference: template<typename Range> void modifies_range(Range && rng); I know it seems counter-intuitive since only && is used above, but it will work with both lvalues and rvalues thanks to reference collapsing rules. When Range is a reference type, && following it will do nothing (being collapsed) and lvalue binding will apply; when Range is an rvalue you will have proper rvalue binding; in both cases parameter "rng" is always an lvalue when used inside the function, since it's named; also in both cases rvalue-sness or lvalue-sness of actual function parameter is preserved in type Range, so it can be "recovered" e.g. with std::forward<Range>(rng). You can find more detailed explanation of how this works on Thomas Becker's explanation of rvalue-references: http://thbecker.net/articles/rvalue_references/section_08.html B.
Hello,
ideally I would like to use range-adapted lvalue ranges like any other lvalue ranges. Writing it down naively, as below, does not compile because LoadRefRange takes (and should take?) the range it modifies by lvalue&, while adaptor expressions are rvalues. ... Any thoughts? Should adaptors receiving lvalues offer an operator adaptor&()&&; // overload on rvalue-ness
you don't want to add such operator to any class, as it would trigger automatic conversion of rvalue of given type to lvalue. Sometimes this is not right thing to do, but with added operator you will have no means of disabling this behaviour.
Do you have an example for range adaptors where this is bad?
Apart from that, ref-qualifiers unfortunately are not yet widely implemented and such code won't be very portable anyway (I only know of 2 compilers implementing this feature).
That will hopefully change soon. I am looking for the direction we (or at least I with my project) want to go with this problem.
What you want to do instead is to change the signature of your function to accept both lvalue and rvalue reference:
template<typename Range> void modifies_range(Range && rng);
So do we want to change boost::sort( Rng& ) to boost::sort( Rng&& ) ? That would make boost::sort( std::vector() ) ok. C++ decided at some point to disallow binding rvalues to lvalue&. I think the reason is that rvalues are inaccessible for the caller, so modifying them is likely a bug. I am a bit hesitant to throw that out the window for ranges. The requirement to treat certain types special w.r.t. rvalue-ness is actually not new. C++11 does the same with iostreams, see 27.7.3.9. Regards, Arno -- Dr. Arno Schödl | aschoedl@think-cell.com Technical Director think-cell Software GmbH | Chausseestr. 8/E | 10115 Berlin | Germany http://www.think-cell.com | phone +49 30 666473-10 | US phone +1 800 891 8091 Amtsgericht Berlin-Charlottenburg, HRB 85229 | European Union VAT Id DE813474306 Directors: Dr. Markus Hannebauer, Dr. Arno Schoedl
On 3/22/2012 2:54 PM, Arno Schödl wrote:
Hello,
What you want to do instead is to change the signature of your function to accept both lvalue and rvalue reference:
template<typename Range> void modifies_range(Range&& rng);
So do we want to change boost::sort( Rng& ) to boost::sort( Rng&& ) ? That would make boost::sort( std::vector() ) ok. C++ decided at some point to disallow binding rvalues to lvalue&. I think the reason is that rvalues are inaccessible for the caller, so modifying them is likely a bug. I am a bit hesitant to throw that out the window for ranges.
Well, its quite often /not/ a bug at all: boost::push_back( a_vector, boost::sort( get_values() ) ); is entirely sensible. And once we get r-value references thoughout the range lib, we can move the elements all the way into the vector (so its optimally efficient and elegant to boot). -Thorsten
On 22/03/2012 13:54, Arno Schödl wrote:
Any thoughts? Should adaptors receiving lvalues offer an operator adaptor&()&&; // overload on rvalue-ness
you don't want to add such operator to any class, as it would trigger automatic conversion of rvalue of given type to lvalue. Sometimes this is not right thing to do, but with added operator you will have no means of disabling this behaviour.
Do you have an example for range adaptors where this is bad?
That would be anywhere when you are passing a temporary to function taking lvalue-reference. If the actual parameter is indeed a temporary, you may wish to apply rvalue-reference optimization, but with such an interface you cannot, since you cannot tell whether actual parameter is temporary or not. The only way to apply this optimization is to define rvalue-reference overload, thus preventing the use of operator T& and rendering it pointless. Also, if such an operator is added to class interface, it would be difficult to remove, since such change would break user code depending on this particular implicit conversion.
So do we want to change boost::sort( Rng& ) to boost::sort( Rng&& ) ? That would make boost::sort( std::vector() ) ok.
I fail to see what's wrong with this. Pointless - yes, but inducing runtime errors - no. Although of course I could have missed something. Back to your function - perhaps taking unqualified template parameter is not the best idea, and if you want to take a range you might want to be bit more explicit about it? E.g. like this: template<typename T1, typename T2> void modifies_range(boost::joined_range<T1, T2> && rng); I hope users of such "modifies_range" function would find nothing to complain about such an interface.
C++ decided at some point to disallow binding rvalues to lvalue&. I think the reason is that rvalues are inaccessible for the caller, so modifying them is likely a bug.
no, the reason was different. The function which takes rvalue-reference does it in order to apply optimization (move semantics) which is often/easily implemented as "stealing" data from its actual parameter. If binding lvalues to rvalue-reference parameters was allowed, such "stealing" could happen implicitly, on an lvalue parameter, and would likely result in runtime errors. As things stand now, you have to be explicit about moving data away from lvalues, e.g. by using std::move() Also, there is nothing wrong with modyfying rvalues and in fact, move semantics does exactly that. In fact, any member function can do that as well, e.g. as "swap" does here, applied in a popular idiom: std::vector<Data> data; // ... do some work and clean up afterwards std::vector<Data>().swap(data); B.
Actually I can see now that my last message contains few errors. On 22/03/2012 15:44, Bronek Kozicki wrote:
So do we want to change boost::sort( Rng& ) to boost::sort( Rng&& ) ? That would make boost::sort( std::vector() ) ok.
I fail to see what's wrong with this. Pointless - yes, but inducing runtime errors - no. Although of course I could have missed something.
Back to your function - perhaps taking unqualified template parameter is not the best idea, and if you want to take a range you might want to
... or perhaps not. Taking unqualified template parameter will allow you to write single function on anything with range-like interface, including boost::iterator_range itself. So, is there anything wrong with: template <typename Range> modifies_range(Range && rng); ?
C++ decided at some point to disallow binding rvalues to lvalue&. I think the reason is that rvalues are inaccessible for the caller, so modifying them is likely a bug.
on second reading I can see that you asked about exactly the opposite binding, taking the history back to last century :) Hope someone will find the explanation below helpful, but the context in which I have put it is obviously wrong - since it only applies to C++11 :)
no, the reason was different. The function which takes rvalue-reference does it in order to apply optimization (move semantics) which is often/easily implemented as "stealing" data from its actual parameter. If binding lvalues to rvalue-reference parameters was allowed, such "stealing" could happen implicitly, on an lvalue parameter, and would likely result in runtime errors. As things stand now, you have to be explicit about moving data away from lvalues, e.g. by using std::move()
B.
Back to your function - perhaps taking unqualified template parameter is not the best idea, and if you want to take a range you might want to be bit more explicit about it? E.g. like this:
template<typename T1, typename T2> void modifies_range(boost::joined_range<T1, T2> && rng);
I hope users of such "modifies_range" function would find nothing to complain about such an interface.
Unfortunately, the reference collapsing no longer works in this case, so this function will not accept an lvalue joined_range. Regards, Nate
On 23/03/2012 02:44, Nathan Ridge wrote:
Back to your function - perhaps taking unqualified template parameter is not the best idea, and if you want to take a range you might want to be bit more explicit about it? E.g. like this:
template<typename T1, typename T2> void modifies_range(boost::joined_range<T1, T2> && rng);
I hope users of such "modifies_range" function would find nothing to complain about such an interface.
Unfortunately, the reference collapsing no longer works in this case, so this function will not accept an lvalue joined_range.
Of course you are correct. Back to: template <typename Range> void modifies_range(Range && rng); B.
Arno Schödl wrote:
ideally I would like to use range-adapted lvalue ranges like any other lvalue ranges. Writing it down naively, as below, does not compile because LoadRefRange takes (and should take?) the range it modifies by lvalue&, while adaptor expressions are rvalues.
template<typename Range> void modifies_range( Range& rng );
<snip>
Any thoughts?
You can use something like this: template <typename Range> typename boost::enable_if< // Check whether // `typename range_reference<typename remove_reference<Range>::type>::type` // is non-const reference
::type modify_range(Range&& rng) { /* … */ }
Regards, Michel
on Thu Mar 22 2012, Arno Schödl <aschoedl-AT-think-cell.com> wrote:
Hello,
ideally I would like to use range-adapted lvalue ranges like any other lvalue ranges. Writing it down naively, as below, does not compile because LoadRefRange takes (and should take?) the range it modifies by lvalue&, while adaptor expressions are rvalues.
One possibility: - make adaptor expressions return const rvalues. The const rvalues will bind to T& arguments (where T is a template parameter) by deducing T to be const. - make even the const_iterators for such adaptors mutable (writable) iterators This accurately reflects the fact that such adaptors don't own their values. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
on Thu Mar 22 2012, Dave Abrahams <dave-AT-boostpro.com> wrote:
on Thu Mar 22 2012, Arno Schödl <aschoedl-AT-think-cell.com> wrote:
Hello,
ideally I would like to use range-adapted lvalue ranges like any other lvalue ranges. Writing it down naively, as below, does not compile because LoadRefRange takes (and should take?) the range it modifies by lvalue&, while adaptor expressions are rvalues.
One possibility:
- make adaptor expressions return const rvalues. The const rvalues will bind to T& arguments (where T is a template parameter) by deducing T to be const.
- make even the const_iterators for such adaptors mutable (writable) iterators
This accurately reflects the fact that such adaptors don't own their values.
Bump. Did nobody understand this suggestion? Bad idea for some reason? Or...? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 3/23/2012 4:20 PM, Dave Abrahams wrote:
One possibility:
- make adaptor expressions return const rvalues. The const rvalues will bind to T& arguments (where T is a template parameter) by deducing T to be const.
I don't like this one.
- make even the const_iterators for such adaptors mutable (writable) iterators
This accurately reflects the fact that such adaptors don't own their values.
Seems reasonable. -Thorsten
On Fri, Mar 23, 2012 at 8:33 AM, Thorsten Ottosen < thorsten.ottosen@dezide.com> wrote:
On 3/23/2012 4:20 PM, Dave Abrahams wrote:
One possibility:
- make adaptor expressions return const rvalues. The const rvalues will bind to T& arguments (where T is a template parameter) by deducing T to be const.
I don't like this one.
- make even the const_iterators for such adaptors mutable (writable)
iterators
This accurately reflects the fact that such adaptors don't own their values.
Seems reasonable.
With Thorsten and Dave on this latter one. - Jeff
on Fri Mar 23 2012, Thorsten Ottosen <thorsten.ottosen-AT-dezide.com> wrote:
On 3/23/2012 4:20 PM, Dave Abrahams wrote:
One possibility:
- make adaptor expressions return const rvalues. The const rvalues will bind to T& arguments (where T is a template parameter) by deducing T to be const.
I don't like this one.
and your specific objection is...? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 3/24/2012 4:14 AM, Dave Abrahams wrote:
on Fri Mar 23 2012, Thorsten Ottosen<thorsten.ottosen-AT-dezide.com> wrote:
On 3/23/2012 4:20 PM, Dave Abrahams wrote:
One possibility:
- make adaptor expressions return const rvalues. The const rvalues will bind to T& arguments (where T is a template parameter) by deducing T to be const.
I don't like this one.
and your specific objection is...?
well, it would require the other solution as well before it works. So the other solution is simpler. -Thorsten
on Mon Mar 26 2012, Thorsten Ottosen <thorsten.ottosen-AT-dezide.com> wrote:
On 3/24/2012 4:14 AM, Dave Abrahams wrote:
on Fri Mar 23 2012, Thorsten Ottosen<thorsten.ottosen-AT-dezide.com> wrote:
On 3/23/2012 4:20 PM, Dave Abrahams wrote:
One possibility:
- make adaptor expressions return const rvalues. The const rvalues will bind to T& arguments (where T is a template parameter) by deducing T to be const.
I don't like this one.
and your specific objection is...?
well, it would require the other solution as well before it works. So the other solution is simpler.
Sorry, what other solution would it require? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 02-04-2012 16:49, Dave Abrahams wrote:
on Mon Mar 26 2012, Thorsten Ottosen<thorsten.ottosen-AT-dezide.com> wrote:
On 3/24/2012 4:14 AM, Dave Abrahams wrote:
on Fri Mar 23 2012, Thorsten Ottosen<thorsten.ottosen-AT-dezide.com> wrote:
On 3/23/2012 4:20 PM, Dave Abrahams wrote:
One possibility:
- make adaptor expressions return const rvalues. The const rvalues will bind to T& arguments (where T is a template parameter) by deducing T to be const.
I don't like this one.
and your specific objection is...?
well, it would require the other solution as well before it works. So the other solution is simpler.
Sorry, what other solution would it require?
The other solution you mentioned: make const_iterator equal to iterator for the returned range types. -Thorsten
on Wed Apr 11 2012, Thorsten Ottosen <thorsten.ottosen-AT-dezide.com> wrote:
On 02-04-2012 16:49, Dave Abrahams wrote:
on Mon Mar 26 2012, Thorsten Ottosen<thorsten.ottosen-AT-dezide.com> wrote:
On 3/24/2012 4:14 AM, Dave Abrahams wrote:
on Fri Mar 23 2012, Thorsten Ottosen<thorsten.ottosen-AT-dezide.com> wrote:
On 3/23/2012 4:20 PM, Dave Abrahams wrote:
> > One possibility: > > - make adaptor expressions return const rvalues. The const rvalues will > bind to T& arguments (where T is a template parameter) by deducing T > to be const.
I don't like this one.
and your specific objection is...?
well, it would require the other solution as well before it works. So the other solution is simpler.
Sorry, what other solution would it require?
The other solution you mentioned: make const_iterator equal to iterator for the returned range types.
That's not "another solution;" it's part of the same solution. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On 3/23/2012 8:33 AM, Thorsten Ottosen wrote:
On 3/23/2012 4:20 PM, Dave Abrahams wrote:
One possibility:
- make adaptor expressions return const rvalues. The const rvalues will bind to T& arguments (where T is a template parameter) by deducing T to be const.
I don't like this one.
For the record, Boost.Proto uses the const rvalue trick extensively. It works well in practice, in my experience.
- make even the const_iterators for such adaptors mutable (writable) iterators
This accurately reflects the fact that such adaptors don't own their values.
Seems reasonable.
Agreed. -- Eric Niebler BoostPro Computing http://www.boostpro.com
Hello,
- make adaptor expressions return const rvalues. The const rvalues will bind to T& arguments (where T is a template parameter) by deducing T to be const.
- make even the const_iterators for such adaptors mutable (writable) iterators
This accurately reflects the fact that such adaptors don't own their values.
If boost::sort is supposed to be chainable (which seems a good idea), then let's not start with adaptors. We already need a solution for boost::sort( std::vector() ); If the solution is template<typename Range> boost::sort( Range && ); then in boost::sort( boost::adaptors::reverse( std::vector() ) ); reverse should probably derive mutable iterators from mutable rvalues, so would have signature: reverse( Range && ); So the policy is: All range functions and adaptors take all ranges by &&, and range::iterator<rvalue>::type is mutable, in particular for adaptors. To chain from a function, return via std::forward<Range>. Problem solved? Arno -- Dr. Arno Schödl | aschoedl@think-cell.com Technical Director think-cell Software GmbH | Chausseestr. 8/E | 10115 Berlin | Germany http://www.think-cell.com | phone +49 30 666473-10 | US phone +1 800 891 8091 Amtsgericht Berlin-Charlottenburg, HRB 85229 | European Union VAT Id DE813474306 Directors: Dr. Markus Hannebauer, Dr. Arno Schoedl
on Fri Mar 23 2012, Arno Schödl <aschoedl-AT-think-cell.com> wrote:
Hello,
- make adaptor expressions return const rvalues. The const rvalues will bind to T& arguments (where T is a template parameter) by deducing T to be const.
- make even the const_iterators for such adaptors mutable (writable) iterators
This accurately reflects the fact that such adaptors don't own their values.
If boost::sort is supposed to be chainable (which seems a good idea), then let's not start with adaptors. We already need a solution for
boost::sort( std::vector() );
Do we? You can't pass a std::vector() rvalue to any other mutating function. std::swap(std::vector<int>(), std::vector<int>()); // ERROR
If the solution is
template<typename Range> boost::sort( Range && );
I think the interesting question is really: what should boost::sort return? For example, it *could* return an adaptor that holds a permutation vector and sorts it lazily. Then you could slice the first ten elements off the sorted result and it would only have to do a partial_sort in the implementation. Or it could return an adaptor that holds a permutation vector and sorts it eagerly. Or, it could return a brand new container. Or, it could sort in-place and return and return a const pair of mutable iterators.
then in
boost::sort( boost::adaptors::reverse( std::vector() ) );
reverse should probably derive mutable iterators from mutable rvalues,
I don't know what you mean by "derive mutable iterators from mutable rvalues." If you mean that you ought to be able to get a mutable iterator from a mutable rvalue... IMO that sort of goes against the grain of the type system. std::vector<int>& v = std::vector<int>(); // Error std::vector<int> const& v = std::vector<int>(); // OK v.begin() -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Hello,
If the solution is>
template<typename Range> boost::sort( Range && );
I think the interesting question is really: what should boost::sort return?
Let's try to be more general. - There are read-only functions like boost::accumulate. - There are range-transforming functions, taking a range and returning a range. I think it is a discussion for a separate thread whether they are always lazy (like adaptors) or sometimes eager (boost::sort?). - And there are range-mutating functions. boost::for_each is currently an example, the range may be modified but is not returned. Many functions in application code also fall into this category. A ) We probably agree that we want to be able to stack transforming functions and use them with rvalues: adaptors::transform(adaptors::reverse( vector<int>() ), func); B) We probably also agree that a mutating function can take a transformed lvalue: vector<int> vec; for_each( adaptors::transform(vec, func), mutating_func() ); // must compile C) And we would probably prefer that a mutating function cannot take an rvalue: for_each( vector<int>(), mutating_func() ) // must not compile D) even if this rvalue is transformed: for_each( adaptors::reverse(vector<int>()), mutating_func() ) // must not compile E) Read-only functions should not be able to modify their argument, even if it is transformed: vector<int> vec; accumulate(adaptors::reverse(vec), 0, mutating_func() ) // must not compile If, as Bronek et. al. propose, - read-only functions take ranges by const&, - transforming functions take ranges by && and - mutating functions take ranges by &&, A and B and E are fulfilled. C and D are not. If, as I understand your proposal, - read-only functions take ranges by const& - transforming functions take ranges by &&, and - mutating function take by ranges &, and - t adaptors return const rvalues with range_iterator<const Range>::type mutable, A, B and C are fulfilled. D and E are not. If we change it to - read-only functions take ranges by const&, - transforming functions take ranges by &&, and - mutating function take by ranges &, and - adaptors return const rvalues with range_iterator<const Range>::type mutable _only if their input was an lvalue&_ A, B, C, and D are fulfilled, but still not E. My proposal, - read-only functions take ranges by const&, - transforming functions take ranges by &&, and - mutating function take by ranges &, and - adaptors are convertible to lvalue& if their input was an lvalue& fulfills A, B, C, D and E, but unfortunately, does not work, because it looks like when matching a template, conversion operators are not considered: struct A { operator A&() const { // should be && but unsupported return const_cast<A&>(*this); } }; void foo( A& a ){} template< class T > void bar( T& t ){} int main() { foo(A()); // ok bar(A()); // does not compile return 0; } My personal opinion is that respecting constness, E, are more important than respecting rvalueness, C and D. What do you think? Are the requirements reasonable? Any more proposals? Regards, Arno -- Dr. Arno Schödl | aschoedl@think-cell.com Technical Director think-cell Software GmbH | Chausseestr. 8/E | 10115 Berlin | Germany http://www.think-cell.com | phone +49 30 666473-10 | US phone +1 800 891 8091 Amtsgericht Berlin-Charlottenburg, HRB 85229 | European Union VAT Id DE813474306 Directors: Dr. Markus Hannebauer, Dr. Arno Schoedl
participants (8)
-
Arno Schödl -
Bronek Kozicki -
Dave Abrahams -
Eric Niebler -
Jeffrey Lee Hellrung, Jr. -
Michel Morin -
Nathan Ridge -
Thorsten Ottosen