I'm writing some piece of software working that does a lot of data structures traversal and that really needed filtering of traversal etc. I thought I'd get that from Boost.Range, but I was surprised to see Boost.Range provides very little: all the good stuff (the range-based algorithms and adaptors) are in another library, range_ex, which isn't in the main distribution yet. Will this library be included in Boost soon? It's really great. As for the open issues, here are my opinions: - Output ranges: don't adaptors already output ranges? - Two input ranges, or a range and an iterator?: two ranges can do more - For any algorithm foo(container, args...), if foo(args...) is valid then it should be called - Null-termination of C-style strings: Isn't that up to Boost.Range to decide whether it's a range or not?
On Tue, 2008-06-17 at 17:47 +0200, Mathias Gaunard wrote:
I'm writing some piece of software working that does a lot of data structures traversal and that really needed filtering of traversal etc.
I thought I'd get that from Boost.Range, but I was surprised to see Boost.Range provides very little: all the good stuff (the range-based algorithms and adaptors) are in another library, range_ex, which isn't in the main distribution yet.
Will this library be included in Boost soon? It's really great.
I am the current maintainer of the Boost.RangeEx proposal. I am therefore responsible for making Boost.RangeEx do what you desire and for the introduction of this library into the main distribution being delayed. I have fixed several issues with the code and continued the development. I have yet to produce documentation that is acceptable for review. I aim to have this done by the end of August 2008. I certainly hope the library will be included in the main distribution soon. I believe the code to be in a good state. I will attempt to resolve any issues that anyone raises in a timely manner. It has been used on Windows, Linux and Mac under Visual C++ 7.1, 8.0 and Gcc 3.x and 4.x. There are no known defects.
As for the open issues, here are my opinions: - Output ranges: don't adaptors already output ranges? The output is an output iterator. Half-open ranges are modified in-place by several algorithms. This works, but may be improved upon later. I believe that the current code is very useful already.
- Two input ranges, or a range and an iterator?: two ranges can do more - For any algorithm foo(container, args...), if foo(args...) is valid then it should be called
- Null-termination of C-style strings: Isn't that up to Boost.Range to decide whether it's a range or not?
It's not quite that simple since supporting C-style strings requires more overloads for many algorithms. It often effects range algorithm implementation too. I believe that leaving the C-style strings behind has been agreed. Please let me know if I may be of further assistance. Regards, Neil Groves
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Am Dienstag 17 Juni 2008 21:52:09 schrieb Neil Groves:
On Tue, 2008-06-17 at 17:47 +0200, Mathias Gaunard wrote:
I'm writing some piece of software working that does a lot of data structures traversal and that really needed filtering of traversal etc.
I thought I'd get that from Boost.Range, but I was surprised to see Boost.Range provides very little: all the good stuff (the range-based algorithms and adaptors) are in another library, range_ex, which isn't in the main distribution yet.
Will this library be included in Boost soon? It's really great.
I am the current maintainer of the Boost.RangeEx proposal.
I wonder about the directory structure of the range_ex.zip in vault: +range_ex +doc +range + test *index.html rather than the usual: +boost +range_ex(or range) +libs +doc +test *index.html Is this by intention? Best, -- Maik
On Wed, 2008-06-18 at 09:23 +0200, Maik Beckmann wrote:
Am Dienstag 17 Juni 2008 21:52:09 schrieb Neil Groves:
On Tue, 2008-06-17 at 17:47 +0200, Mathias Gaunard wrote:
I'm writing some piece of software working that does a lot of data structures traversal and that really needed filtering of traversal etc.
I thought I'd get that from Boost.Range, but I was surprised to see Boost.Range provides very little: all the good stuff (the range-based algorithms and adaptors) are in another library, range_ex, which isn't in the main distribution yet.
Will this library be included in Boost soon? It's really great.
I am the current maintainer of the Boost.RangeEx proposal.
I wonder about the directory structure of the range_ex.zip in vault: +range_ex +doc +range + test *index.html rather than the usual: +boost +range_ex(or range) +libs +doc +test *index.html Is this by intention?
The intent is to fit in with whatever else boost does. I do intend to replace the range folder rather than add a range_ex and have confusion about which features belong in which library part. The contents other than the source code do require some work. I will layout the folders more appropriately by the end of the week. Thank you for highlighting this error.
Best, -- Maik
Neil Groves
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Am Mittwoch 18 Juni 2008 10:02:18 schrieb Neil Groves:
On Wed, 2008-06-18 at 09:23 +0200, Maik Beckmann wrote:
rather than the usual: +boost +range_ex(or range) +libs +doc +test *index.html Is this by intention?
argh.. this should have been: +boost +range_ex(or range) +libs +range_ex(or range) +doc +test *index.html
The intent is to fit in with whatever else boost does. I do intend to replace the range folder rather than add a range_ex and have confusion about which features belong in which library part.
This was my guess.
The contents other than the source code do require some work. I will layout the folders more appropriately by the end of the week.
Nice!
Thank you for highlighting this error.
np Best, -- Maik
range_ex.zip has been updated in the root of the Vault to correct the folder structure. HTH, Neil Groves On Wed, Jun 18, 2008 at 10:06 AM, Maik Beckmann < beckmann.maik@googlemail.com> wrote:
argh.. this should have been: +boost +range_ex(or range) +libs +range_ex(or range) +doc +test *index.html
Best, -- Maik
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Neil Groves wrote:
range_ex.zip has been updated in the root of the Vault to correct the folder structure.
I just downloaded your library and I was surprised to see the make_filtered_range-like syntax was dropped. While the pipe-like syntax is interesting, I think both interfaces should be there. Also, I think you should add to the documentation that the functors need to have result_type (or be real functions). It might be interesting to make it so decltype/typeof/emulation can be used instead, by the way. It shouldn't be too difficult to wrap the function objects so that result_type can be automatically deduced, since in all cases the types of all arguments are fixed.
Mathias Gaunard wrote:
Neil Groves wrote:
range_ex.zip has been updated in the root of the Vault to correct the folder structure.
I just downloaded your library and I was surprised to see the make_filtered_range-like syntax was dropped. While the pipe-like syntax is interesting, I think both interfaces should be there.
Thank you for this feedback. I hope to get a few more responses and then respond appropriately. I do not have a strong preference one way or another. If no one objects within this week then I shall reintroducing make_filtered_range et al. I expect to be able to upload by 6 June 2008.
Also, I think you should add to the documentation that the functors need to have result_type (or be real functions). It might be interesting to make it so decltype/typeof/emulation can be used instead, by the way. It shouldn't be too difficult to wrap the function objects so that result_type can be automatically deduced, since in all cases the types of all arguments are fixed.
I will look into this suggestion. To be perfectly honest I am not sure how to make it work for all function types. I suspect that a good look around the other Boost libraries or search engines will reveal a solution. This may take me longer due to my lack of experience with this approach.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Most of all, thank you enormously for taking the time to provide useful feedback. Best Wishes, Neil Groves
AMDG Neil Groves wrote:
Also, I think you should add to the documentation that the functors need to have result_type (or be real functions). It might be interesting to make it so decltype/typeof/emulation can be used instead, by the way. It shouldn't be too difficult to wrap the function objects so that result_type can be automatically deduced, since in all cases the types of all arguments are fixed.
I will look into this suggestion. To be perfectly honest I am not sure how to make it work for all function types. I suspect that a good look around the other Boost libraries or search engines will reveal a solution. This may take me longer due to my lack of experience with this approach.
Can't you use result_of? In Christ, Steven Watanabe
Steven Watanabe wrote:
Can't you use result_of?
As far as I can see, the current boost::result_of implementation doesn't use decltype or typeof. I suppose implementing decltype on top of typeof is not a big problem (you only need to check whether the expression is a reference or not) however both have problems with mangling when used in a return type, at least in GCC. That basically means you can't always rely on them working. I thought decltype didn't have those mangling problems, looks like it is a regression. As for the correct way to do that, I suppose it should be possible to tell boost::result_type with a macro definition to use Boost.Typeof, which in turn can use either native or emulation. However range_ex only forwards algorithms and adaptors to Boost.Iterator and maybe the standard library. Maybe it's these that ought to use boost::result_of in the first place?
On Tue, Jun 24, 2008 at 12:12 PM, Mathias Gaunard <mathias.gaunard@ens-lyon.org> wrote:
Steven Watanabe wrote:
Can't you use result_of?
As far as I can see, the current boost::result_of implementation doesn't use decltype or typeof.
The boost::result_of on svn trunk uses decltype when BOOST_HAS_DECLTYPE is defined. This happens automatically for gcc versions after 4.2.
I suppose implementing decltype on top of typeof is not a big problem (you only need to check whether the expression is a reference or not) however both have problems with mangling when used in a return type, at least in GCC. That basically means you can't always rely on them working. I thought decltype didn't have those mangling problems, looks like it is a regression.
As for the correct way to do that, I suppose it should be possible to tell boost::result_type with a macro definition to use Boost.Typeof, which in turn can use either native or emulation.
This might not be a bad idea for compilers that lack decltype - perhaps it could be offered as an alternative to the TR1 heuristic. Of course, this would break compatibility with TR1 and C++0x in some cases, which might cause more problems than it solves. But if the various alternative result_of implementations are documented properly (and perhaps placed in separate namespaces) then it shouldn't be so bad.
However range_ex only forwards algorithms and adaptors to Boost.Iterator and maybe the standard library. Maybe it's these that ought to use boost::result_of in the first place?
This is an excellent point. The standard library essentially relies on concepts like SGI's AdaptableUnaryFunction, which requires expressions of the form funtor::result_type, funtor::argument_type, etc. I think it would be useful to have <functional> and <algorithm> implementations that support an alternative concept framework requiring expressions of the form result_of<functor(T)>::type and argument_of<functor(T)>::type. Daniel Walker
On Tue, Jun 24, 2008 at 12:51 PM, Daniel Walker <daniel.j.walker@gmail.com> wrote:
On Tue, Jun 24, 2008 at 12:12 PM, Mathias Gaunard <mathias.gaunard@ens-lyon.org> wrote:
Steven Watanabe wrote:
Can't you use result_of?
As far as I can see, the current boost::result_of implementation doesn't use decltype or typeof.
The boost::result_of on svn trunk uses decltype when BOOST_HAS_DECLTYPE is defined. This happens automatically for gcc versions after 4.2.
Correction - for gcc > 4.2, the boost::result_of on svn trunk uses decltype only when you compile with -std=c++0x. It uses the old TR1 heuristic otherwise. Daniel Walker
On Tue, Jun 24, 2008 at 7:07 PM, Daniel Walker <daniel.j.walker@gmail.com> wrote:
On Tue, Jun 24, 2008 at 12:51 PM, Daniel Walker <daniel.j.walker@gmail.com> wrote:
On Tue, Jun 24, 2008 at 12:12 PM, Mathias Gaunard <mathias.gaunard@ens-lyon.org> wrote:
Steven Watanabe wrote:
Can't you use result_of?
As far as I can see, the current boost::result_of implementation doesn't use decltype or typeof.
The boost::result_of on svn trunk uses decltype when BOOST_HAS_DECLTYPE is defined. This happens automatically for gcc versions after 4.2.
Correction - for gcc > 4.2, the boost::result_of on svn trunk uses decltype only when you compile with -std=c++0x. It uses the old TR1 heuristic otherwise.
"__decltype()" works even outside of C++0x mode. Shouldn't it be used instead? -- gpd
On Tue, Jun 24, 2008 at 1:19 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Tue, Jun 24, 2008 at 7:07 PM, Daniel Walker <daniel.j.walker@gmail.com> wrote:
On Tue, Jun 24, 2008 at 12:51 PM, Daniel Walker <daniel.j.walker@gmail.com> wrote:
On Tue, Jun 24, 2008 at 12:12 PM, Mathias Gaunard <mathias.gaunard@ens-lyon.org> wrote:
Steven Watanabe wrote:
Can't you use result_of?
As far as I can see, the current boost::result_of implementation doesn't use decltype or typeof.
The boost::result_of on svn trunk uses decltype when BOOST_HAS_DECLTYPE is defined. This happens automatically for gcc versions after 4.2.
Correction - for gcc > 4.2, the boost::result_of on svn trunk uses decltype only when you compile with -std=c++0x. It uses the old TR1 heuristic otherwise.
"__decltype()" works even outside of C++0x mode. Shouldn't it be used instead?
C++0x decltype and the TR1 heuristic are not completely compatible (decltype always succeeds at return type deduction, the heuristic does not), so I think it's good to only get the new and improved boost::result_of when using the new and improved standard. However, there is a need for improving result_of for compilers that don't explicitly support C++0x decltype, so long as the user knows when/how it's being improved. Boost.Typeof already aims to provide aggressive type deduction for an array of compilers with varying capabilities. As has been suggested, perhaps result_of could use Boost.Typeof on C++03 compilers, which in turn could use various compiler extension/hooks when available. So, for example, here's one way of concisely providing several different strategies for return type deduction based on the standard and/or compiler's capabilities. - boost::tr1::result_of uses the legacy heuristic - boost::cpp0x::result_of uses the standard decltype - boost::cpp03::result_of uses Boost.Typeof (see the Boost.Typeof documentation for whether typeof, __decltype, or the type registry is used) For C++0x, boost::result_of would be an alias for boost::cpp0x::result_of. For C++03, boost::result_of would be an alias for boost::tr1::result_of, so as not to break any existing code. If users want a better result_of for C++03 compilers, they can explicitly upgrade from boost::result_of to boost::cpp03::result_of. And of course, in practice, as compiler vendors supply C++0x standard libraries, folks will simply use std::result_of. That's probably not too much to keep track of as we migrate from one standard to the next. Daniel Walker
On Tue, Jun 24, 2008 at 6:12 PM, Mathias Gaunard <mathias.gaunard@ens-lyon.org> wrote:
Steven Watanabe wrote:
Can't you use result_of?
As far as I can see, the current boost::result_of implementation doesn't use decltype or typeof. I suppose implementing decltype on top of typeof is not a big problem (you only need to check whether the expression is a reference or not) however both have problems with mangling when used in a return type, at least in GCC. That basically means you can't always rely on them working. I thought decltype didn't have those mangling problems, looks like it is a regression.
To workaround the gcc bug you can hoist the type computation to an helper metafunction: template<class A, class B> decltype(A() + B()) sum(A, B); //fails with "sorry not implemented" template<class A, class B> struct sum_result { typedef dectlyp(A(), B()) type; } template<class A, class B> typename sum_result<A,B>::type sum(A,B); // ok This way the "A() + B()" expression need not to be mangled, only the metafunction invocation which gcc handles just fine. It is inconvenient, but using decltype this way beats doing type deduction by hand. I'm not sure if this works in all cases though.
As for the correct way to do that, I suppose it should be possible to tell boost::result_type with a macro definition to use Boost.Typeof, which in turn can use either native or emulation.
AFAIK Boost.Typeof would deduce the wrong result type when a function returns a reference (wheter using emulation of native). I do not think it can be made to deduce references correctly. IMHO Range_ex should just use result_of. It is to the provider of the functor (i.e. the range_ex user) the decision of implementing the TR1 result_of protocol or use a compiler which support decltype (I think that someone proposed a patch to have boost::result_of use decltype where available).
However range_ex only forwards algorithms and adaptors to Boost.Iterator and maybe the standard library. Maybe it's these that ought to use boost::result_of in the first place?
I theory yes, but as boost.iterator adaptors allows the user to easily specify the value/reference type, strictly speaking there is no need of result_of support there. -- gpd
Giovanni Piero Deretta wrote:
To workaround the gcc bug you can hoist the type computation to an helper metafunction:
template<class A, class B> decltype(A() + B()) sum(A, B); //fails with "sorry not implemented"
template<class A, class B> struct sum_result { typedef dectlyp(A(), B()) type; }
template<class A, class B> typename sum_result<A,B>::type sum(A,B); // ok
This way the "A() + B()" expression need not to be mangled, only the metafunction invocation which gcc handles just fine. It is inconvenient, but using decltype this way beats doing type deduction by hand.
Good to know. I suppose Boost.Typeof could simply add a special macro for usage in return types that does just that.
AFAIK Boost.Typeof would deduce the wrong result type when a function returns a reference (wheter using emulation of native). I do not think it can be made to deduce references correctly.
Here is why I think implementing decltype on top of typeof is easy. The expression exp could simply be wrapped in a foo(exp) with two overloads of foo, one for T and one for T&. foo would then tag the expression somehow to tell whether it was a reference or not. Then after applying __typeof__, if the tag says the expression was of a reference type, simply readd the reference qualifier. As for the emulation mode, I have no idea about how it's implemented but I assume it's manually removing reference qualifiers. Boost.Typeof really should add a BOOST_DECLTYPE and not simply BOOST_AUTO and BOOST_TYPEOF.
I theory yes, but as boost.iterator adaptors allows the user to easily specify the value/reference type, strictly speaking there is no need of result_of support there.
transform at least needs to know what the result type of the function object is.
On Wed, Jun 25, 2008 at 2:46 AM, Mathias Gaunard <mathias.gaunard@ens-lyon.org> wrote:
Giovanni Piero Deretta wrote:
AFAIK Boost.Typeof would deduce the wrong result type when a function returns a reference (wheter using emulation of native). I do not think it can be made to deduce references correctly.
Here is why I think implementing decltype on top of typeof is easy.
The expression exp could simply be wrapped in a foo(exp) with two overloads of foo, one for T and one for T&. foo would then tag the expression somehow to tell whether it was a reference or not. Then after applying __typeof__, if the tag says the expression was of a reference type, simply readd the reference qualifier.
As for the emulation mode, I have no idea about how it's implemented but I assume it's manually removing reference qualifiers.
Boost.Typeof really should add a BOOST_DECLTYPE and not simply BOOST_AUTO and BOOST_TYPEOF.
I do not think there is any way, at compile time, to portably distinguish between a const qualified reference and an rvalue return type: i.e, give struct bar{}; no kind of overloading/traits trick will let you distinguish between these two 'foo': bar const& baz(); bar baz(); you would need to overload your 'foo' (or better, is_reference) detection function like this: template<typename T> true_type is_reference(T); template<typename T> true_type is_reference(T const&); which of course is ambiguous (even dropping the const from the second and letting the compiler deduce T as const bar). Note that the above problem is present for both emulated and native typeof. It is possible to do the detection at runtime using the ?: trick (see BOOST_FOREACH), but AFAIK it cannot be done portably at compiletime. Well, at leat until someone discover some dirty trick of course :). In the meantime we will have to use the result<> protocol. -- gpd
Mathias Gaunard wrote:
Steven Watanabe wrote:
Can't you use result_of?
As far as I can see, the current boost::result_of implementation doesn't use decltype or typeof. I suppose implementing decltype on top of typeof is not a big problem (you only need to check whether the expression is a reference or not) however both have problems with mangling when used in a return type, at least in GCC. That basically means you can't always rely on them working. I thought decltype didn't have those mangling problems, looks like it is a regression.
As for the correct way to do that, I suppose it should be possible to tell boost::result_type with a macro definition to use Boost.Typeof, which in turn can use either native or emulation.
However range_ex only forwards algorithms and adaptors to Boost.Iterator and maybe the standard library. Maybe it's these that ought to use boost::result_of in the first place?
I've been thinking for a long time that we need to develop a coordinated approach to developing some of these more "basic" facilities. We have too many great bits and pieces that don't quite interoperate as well as they could. This was driven home for me recently when I taught a course on Boost with hands-on workshops and the number of pitfalls that people (including me!) ran into was just too high. In order to really make the case for the power of library use, it needs to be easier to write expressive C++. I'm not sure exactly how to proceed from here. Maybe the first thing is to develop a list of things in (or that should be in) Boost that seem to fall into this category, because I don't really understand the rules my brain has used to decide the category boundaries. Maybe after that we need a "library domain maintainer" or something who can see to the coordination of these things... but let's not get ahead of ourselves. Here are the parts and topics that I see as being in the category: result_of BOOST_TYPEOF range_ex BOOST_FOREACH our own implementations of std::containers (in interprocess, IIRC) lambda/phoenix/bind move semantics Boost.Iterator Boost.Algorithm segmented iterator support (http://lafstern.org/matt/segmented.pdf) property maps Thoughts? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Thu, Jun 26, 2008 at 9:08 PM, David Abrahams <dave@boostpro.com> wrote:
I've been thinking for a long time that we need to develop a coordinated approach to developing some of these more "basic" facilities. We have too many great bits and pieces that don't quite interoperate as well as they could. This was driven home for me recently when I taught a course on Boost with hands-on workshops and the number of pitfalls that people (including me!) ran into was just too high. In order to really make the case for the power of library use, it needs to be easier to write expressive C++.
What would be helpful I think would be a statement of exactly what the problems are with the great bits and pieces and in what particular context they are considered problem(s). I think it would be easy to agree that we'd love to have everything in Boost work like it new every other library in it, but that makes Boost more a Framework than a library collection. I don't necessarily think that would be a Bad Thing, but rather too close a coupling for my taste.
I'm not sure exactly how to proceed from here. Maybe the first thing is to develop a list of things in (or that should be in) Boost that seem to fall into this category, because I don't really understand the rules my brain has used to decide the category boundaries. Maybe after that we need a "library domain maintainer" or something who can see to the coordination of these things... but let's not get ahead of ourselves.
Here are the parts and topics that I see as being in the category:
result_of BOOST_TYPEOF range_ex BOOST_FOREACH our own implementations of std::containers (in interprocess, IIRC) lambda/phoenix/bind move semantics Boost.Iterator Boost.Algorithm segmented iterator support (http://lafstern.org/matt/segmented.pdf) property maps
Thoughts?
How about we first collect the problems you encounter and what other people already encounter with the specific libraries you've listed above -- and what precisely or roughly you'd like to achieve with coodinating the effort(s) of the above. Would it be safe to say you'd like to coordinate these to achieve true "expressive C++" with libraries? -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
Dean Michael Berris wrote:
On Thu, Jun 26, 2008 at 9:08 PM, David Abrahams <dave@boostpro.com> wrote:
I've been thinking for a long time that we need to develop a coordinated approach to developing some of these more "basic" facilities. We have too many great bits and pieces that don't quite interoperate as well as they could. This was driven home for me recently when I taught a course on Boost with hands-on workshops and the number of pitfalls that people (including me!) ran into was just too high. In order to really make the case for the power of library use, it needs to be easier to write expressive C++.
What would be helpful I think would be a statement of exactly what the problems are with the great bits and pieces and in what particular context they are considered problem(s).
OK, but it's not just about problems.
I think it would be easy to agree that we'd love to have everything in Boost work like it new every other library in it, but that makes Boost more a Framework than a library collection. I don't necessarily think that would be a Bad Thing, but rather too close a coupling for my taste.
I don't think there's anything "coupled" about coordinating foundational parts so they interoperate properly.
Here are the parts and topics that I see as being in the category:
result_of
* Too hard to define result<> templates for polymorphic function objects. * A whole spate of questions that came up in http://article.gmane.org/gmane.comp.lib.boost.devel/173370 were not answered with a page of documentation, utility library, or other facility. I still don't know what the upshot was
BOOST_TYPEOF
* Doesn't coordinate with result_of
range_ex
* Not in Boost. Should have pipe syntax. Maybe should use expression templates and be lazier.
BOOST_FOREACH
* We don't have range_ex ;-)
our own implementations of std::containers (in interprocess, IIRC)
* Not well-known. * Not in a general place (bound to interprocess?) * Ought to support move semantics
lambda/phoenix/bind
* Incompatible placeholders * Been waiting for the lambda/phoenix merger forever * Lambda has lots of limitations and gotchas that we can improve on now * Phoenix 2.0 not in Boost
move semantics
* Not supported by many foundational Boost types that could benefit (e.g. shared_ptr) * No general move semantics library in Boost
Boost.Iterator
* Needs an update (e.g. "new iterator concepts" have to go -- C++0x resolves that differently by simply allowing proxies everywhere) * Several important fixes and features needed
Boost.Algorithm
* Should support ranges, move semantics, segmented iterators, property maps
segmented iterator support (http://lafstern.org/matt/segmented.pdf) property maps
The above are just related to the rest of them.
How about we first collect the problems you encounter and what other people already encounter with the specific libraries you've listed above -- and what precisely or roughly you'd like to achieve with coodinating the effort(s) of the above.
* More "it just works out of the box" experience * More efficiency!
Would it be safe to say you'd like to coordinate these to achieve true "expressive C++" with libraries?
Something like that. I guess efficiency is the other big concern. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Mon, Jun 30, 2008 at 3:37 PM, David Abrahams <dave@boostpro.com> wrote:
On Thu, Jun 26, 2008 at 9:08 PM, David Abrahams <dave@boostpro.com> wrote:
Here are the parts and topics that I see as being in the category:
result_of
* Too hard to define result<> templates for polymorphic function objects.
I use a simple adaptor that makes it possible to use an mpl expression to compute result types.
* A whole spate of questions that came up in http://article.gmane.org/gmane.comp.lib.boost.devel/173370 were not answered with a page of documentation, utility library, or other facility. I still don't know what the upshot was
The solution used by Eric is basically the same used by Egg::poly: http://tinyurl.com/64k4k6 I haven't really tried it so far, but it seems pretty good and simple.
BOOST_TYPEOF
* Doesn't coordinate with result_of
Should it? How?
range_ex
* Not in Boost. Should have pipe syntax. Maybe should use expression templates and be lazier.
Does it really need expression templates? That is, given the range builders make_*_view [1], it should return a *_view (for example a filtered_view). Combining multiple views, you get for example: filtered_view<mapped_view<taken_view<BaseRange> , Mapper>, Filter> > I.e. a series of range views are already an expression templates. Now, if you make some (reasonable, and hopefully documented) assumptions about Filter and Mapper, you should be always able to convert sequences of nested views that contains only views known to range_ex in a normal form: for example: taken_view<mapped_view<filtered_view<BaseRange, compose<Mapper, Filter> >, Mapper> > You can of course collapse multiple mapped, filtered and taken range in a single instance of each range. [2] A top level boost::for_each could be able to unwind the normalized expression and apply a simple iteration over the Base range instead of relying on the compiler of eliminating the abstraction overhead of four different iterators. I think that the basic ranges that should (at least) be supported are: - mapped_range - filtered_range - taken_range (i.e. just the first n elements of a range) - taken_while_range (i.e. the first elements such as a predicate is true) - zipped_range - unfold_range (with which you can pretty much express any other range, see http://tinyurl.com/5mus25) An eventual 'drop' (take all the elements after the first n) and 'drop_while' (drop the first elements such as a predicate is true) probably need strict evaluation anyway and aren't worth supporting explicitly. Finally, what do you mean that range_ex views should be lazier? Lazier than what? [1] Btw, I greatly dislike the verbose make_filtered_view, make_transformed_view names. I use these builders a lot, and simply call them 'map' and 'filter'. [2] I think that this is quite similar to the deforestation technique used by functional languages compilers. (see the wikipedia page and the linked articles). In FP it is used mostly to eliminate intermediate (lazily evaluated) lists, but the same technique can be applied in c++. In fact the rewrite rules used map perfectly to c++ templates.
our own implementations of std::containers (in interprocess, IIRC)
* Not well-known. * Not in a general place (bound to interprocess?) * Ought to support move semantics
Yes please. And segmented iterators support for deque, and the guarantee that every deque subrange is contiguous in memory (it is great for network programming). And inplace allocation, and... <snip>
Boost.Iterator
* Needs an update (e.g. "new iterator concepts" have to go -- C++0x resolves that differently by simply allowing proxies everywhere)
What about older compilers? Also it would be interesting to have a mapping between the C++0x solution and the "new iterator concepts". <snip> -- gpd
On Mon, Jun 30, 2008 at 10:51 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Mon, Jun 30, 2008 at 3:37 PM, David Abrahams <dave@boostpro.com> wrote:
range_ex
* Not in Boost. Should have pipe syntax. Maybe should use expression templates and be lazier.
Does it really need expression templates? That is, given the range builders make_*_view [1], it should return a *_view (for example a filtered_view). Combining multiple views, you get for example:
filtered_view<mapped_view<taken_view<BaseRange> , Mapper>, Filter> >
I.e. a series of range views are already an expression templates. Now, if you make some (reasonable, and hopefully documented) assumptions about Filter and Mapper, you should be always able to convert sequences of nested views that contains only views known to range_ex in a normal form: for example:
taken_view<mapped_view<filtered_view<BaseRange, compose<Mapper, Filter> >, Mapper> >
You can of course collapse multiple mapped, filtered and taken range in a single instance of each range. [2] A top level boost::for_each could be able to unwind the normalized expression and apply a simple iteration over the Base range instead of relying on the compiler of eliminating the abstraction overhead of four different iterators.
I think the point of using expression templates is to be able to do something like: copy( range(source) | taken(10) | map(mapper()), make_function_output_iterator(reducer()) // should have a better name? ); Without having to go through the explicit 'make_*_iterator' or explicit type of the iterator you want to build.
I think that the basic ranges that should (at least) be supported are:
- mapped_range - filtered_range - taken_range (i.e. just the first n elements of a range) - taken_while_range (i.e. the first elements such as a predicate is true) - zipped_range - unfold_range (with which you can pretty much express any other range, see http://tinyurl.com/5mus25)
An eventual 'drop' (take all the elements after the first n) and 'drop_while' (drop the first elements such as a predicate is true) probably need strict evaluation anyway and aren't worth supporting explicitly.
I also think there should be a zipped_function which: - takes N function objects and an N-tuple - applies each function to the appropriate tuple - returns a tuple of results This looks like (when used): transform( zipped_range(source1, source2, source3), make_function_output_iterator(make_zipped_function(f1, f2, f3)), make_zipped_function(g1, g2, g3) );
Finally, what do you mean that range_ex views should be lazier? Lazier than what?
[1] Btw, I greatly dislike the verbose make_filtered_view, make_transformed_view names. I use these builders a lot, and simply call them 'map' and 'filter'.
I agree.
[2] I think that this is quite similar to the deforestation technique used by functional languages compilers. (see the wikipedia page and the linked articles). In FP it is used mostly to eliminate intermediate (lazily evaluated) lists, but the same technique can be applied in c++. In fact the rewrite rules used map perfectly to c++ templates.
Here I think you need the zipped_function idiom where you can create an 'in-step' function that can deal with tuples of inputs. zipped_function(zipped_input) -> zipped_output Another thing that's missing is a way to transform tuples of one form into another -- possibly ignoring tuple elements. I've posted a possible implementation based on Fusion's at_c, which allows: map<int, string> source; // populate source.. for_each(source.begin(), source.end(), select<1> | (cout << arg1 << endl) //using fusion ); // select will choose the second element of the pair -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
On Tue, Jul 1, 2008 at 7:12 AM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
On Mon, Jun 30, 2008 at 10:51 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Mon, Jun 30, 2008 at 3:37 PM, David Abrahams <dave@boostpro.com> wrote:
range_ex
* Not in Boost. Should have pipe syntax. Maybe should use expression templates and be lazier.
Does it really need expression templates? That is, given the range builders make_*_view [1], it should return a *_view (for example a filtered_view). Combining multiple views, you get for example:
filtered_view<mapped_view<taken_view<BaseRange> , Mapper>, Filter> >
I.e. a series of range views are already an expression templates. Now, if you make some (reasonable, and hopefully documented) assumptions about Filter and Mapper, you should be always able to convert sequences of nested views that contains only views known to range_ex in a normal form: for example:
taken_view<mapped_view<filtered_view<BaseRange, compose<Mapper, Filter> >, Mapper> >
You can of course collapse multiple mapped, filtered and taken range in a single instance of each range. [2] A top level boost::for_each could be able to unwind the normalized expression and apply a simple iteration over the Base range instead of relying on the compiler of eliminating the abstraction overhead of four different iterators.
I think the point of using expression templates is to be able to do something like:
copy( range(source) | taken(10) | map(mapper()), make_function_output_iterator(reducer()) // should have a better name? );
You do not need anything more than what is in Lambda/Phoenix here. Instead of having a different function for piping an generating views, you use the same function and curry it: auto reduced = source | take(_, 10) | map(_, mapper()) | fold(_, reducer()) ; the _ is like _1 in bind/lambda/phoenix, but it doesn't make the whole expression a lambda expression (it stops at one level). The pipe in practice works like a compose operator. If you put the range at the end, you wouldn't even need the '_' (just provide one less argument, hey it is haskell :) ), but you give up variadic (or overloaded) functions.
I think that the basic ranges that should (at least) be supported are:
- mapped_range - filtered_range - taken_range (i.e. just the first n elements of a range) - taken_while_range (i.e. the first elements such as a predicate is true) - zipped_range - unfold_range (with which you can pretty much express any other range, see http://tinyurl.com/5mus25)
An eventual 'drop' (take all the elements after the first n) and 'drop_while' (drop the first elements such as a predicate is true) probably need strict evaluation anyway and aren't worth supporting explicitly.
I also think there should be a zipped_function which:
- takes N function objects and an N-tuple - applies each function to the appropriate tuple - returns a tuple of results
D'oh, of course, forgot this. I think that zipped_range on top of zipped_iterator is enough, you get the other functionality by composing with transformed_iterator. <snip>
[2] I think that this is quite similar to the deforestation technique used by functional languages compilers. (see the wikipedia page and the linked articles). In FP it is used mostly to eliminate intermediate (lazily evaluated) lists, but the same technique can be applied in c++. In fact the rewrite rules used map perfectly to c++ templates.
Here I think you need the zipped_function idiom where you can create an 'in-step' function that can deal with tuples of inputs.
zipped_function(zipped_input) -> zipped_output
Another thing that's missing is a way to transform tuples of one form into another -- possibly ignoring tuple elements. I've posted a possible implementation based on Fusion's at_c, which allows:
map<int, string> source; // populate source.. for_each(source.begin(), source.end(), select<1> | (cout << arg1 << endl) //using fusion ); // select will choose the second element of the pair
Ok, but this is beyond boost range. I think that fusion svn already provides function objects for every algorithm/accessors, so this would becomes for_each(source | map(_, select<1>() ) , coust << arg1 << endl ) BTW I call select<1>() simply first (and of course also have second, third, etc...). -- gpd
On Tue, Jul 1, 2008 at 6:22 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Tue, Jul 1, 2008 at 7:12 AM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
I think the point of using expression templates is to be able to do something like:
copy( range(source) | taken(10) | map(mapper()), make_function_output_iterator(reducer()) // should have a better name? );
You do not need anything more than what is in Lambda/Phoenix here. Instead of having a different function for piping an generating views, you use the same function and curry it:
auto reduced = source | take(_, 10) | map(_, mapper()) | fold(_, reducer()) ;
the _ is like _1 in bind/lambda/phoenix, but it doesn't make the whole expression a lambda expression (it stops at one level). The pipe in practice works like a compose operator. If you put the range at the end, you wouldn't even need the '_' (just provide one less argument, hey it is haskell :) ), but you give up variadic (or overloaded) functions.
Isn't the whole point of creating an expression template based-approach to create these (maybe using proto I guess) and allow: 1. The same placeholders used in Phoenix+Proto to be used in the pipe syntax in range_ex (or an imaginary new RangeEx implementation). 2. Allow for efficient (i.e., compiler optimized/assisted) implementations of the composition of these new views/ranges/iterators. ? Although I do like how this looks granted that you are using C++0x's auto -- in today's C++, the reason I used the STL was to avoid having to compute the actual type produced by the pipe syntax. Maybe this is why Dave is asking for a better BOOST_TYPEOF. ;-)
I think that the basic ranges that should (at least) be supported are:
- mapped_range - filtered_range - taken_range (i.e. just the first n elements of a range) - taken_while_range (i.e. the first elements such as a predicate is true) - zipped_range - unfold_range (with which you can pretty much express any other range, see http://tinyurl.com/5mus25)
An eventual 'drop' (take all the elements after the first n) and 'drop_while' (drop the first elements such as a predicate is true) probably need strict evaluation anyway and aren't worth supporting explicitly.
I also think there should be a zipped_function which:
- takes N function objects and an N-tuple - applies each function to the appropriate tuple - returns a tuple of results
D'oh, of course, forgot this. I think that zipped_range on top of zipped_iterator is enough, you get the other functionality by composing with transformed_iterator.
Okay, but shouldn't this composition with transform_iterator be done with something that 'comes out of the box'?
<snip>
[2] I think that this is quite similar to the deforestation technique used by functional languages compilers. (see the wikipedia page and the linked articles). In FP it is used mostly to eliminate intermediate (lazily evaluated) lists, but the same technique can be applied in c++. In fact the rewrite rules used map perfectly to c++ templates.
Here I think you need the zipped_function idiom where you can create an 'in-step' function that can deal with tuples of inputs.
zipped_function(zipped_input) -> zipped_output
Another thing that's missing is a way to transform tuples of one form into another -- possibly ignoring tuple elements. I've posted a possible implementation based on Fusion's at_c, which allows:
map<int, string> source; // populate source.. for_each(source.begin(), source.end(), select<1> | (cout << arg1 << endl) //using fusion ); // select will choose the second element of the pair
Ok, but this is beyond boost range. I think that fusion svn already provides function objects for every algorithm/accessors, so this would becomes
for_each(source | map(_, select<1>() ) , coust << arg1 << endl )
BTW I call select<1>() simply first (and of course also have second, third, etc...).
Ok. I'd love to see Phoenix 2.0 come out and 'just work' with Boost.Range/RangeEx/Iterator/Algorithms -- I think now I "get" what Dave is looking for. :D -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
On Tue, Jul 1, 2008 at 12:50 PM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
On Tue, Jul 1, 2008 at 6:22 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Tue, Jul 1, 2008 at 7:12 AM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
I think the point of using expression templates is to be able to do something like:
copy( range(source) | taken(10) | map(mapper()), make_function_output_iterator(reducer()) // should have a better name? );
You do not need anything more than what is in Lambda/Phoenix here. Instead of having a different function for piping an generating views, you use the same function and curry it:
auto reduced = source | take(_, 10) | map(_, mapper()) | fold(_, reducer()) ;
the _ is like _1 in bind/lambda/phoenix, but it doesn't make the whole expression a lambda expression (it stops at one level). The pipe in practice works like a compose operator. If you put the range at the end, you wouldn't even need the '_' (just provide one less argument, hey it is haskell :) ), but you give up variadic (or overloaded) functions.
Isn't the whole point of creating an expression template based-approach to create these (maybe using proto I guess) and allow:
1. The same placeholders used in Phoenix+Proto to be used in the pipe syntax in range_ex (or an imaginary new RangeEx implementation). 2. Allow for efficient (i.e., compiler optimized/assisted) implementations of the composition of these new views/ranges/iterators.
Of course. My point is that you need very few beyond what is available in Phoenix. Just the ability to have a placeholder which doesn't make the whole expression lazy ( i.e. foo(_) should be the same as unlambda(bind(foo, _1)) ).
?
Although I do like how this looks granted that you are using C++0x's auto -- in today's C++, the reason I used the STL was to avoid having to compute the actual type produced by the pipe syntax.
You need to know the result type of the reducer even in your example, so it is not specific to the 'sugared' syntax :)
Maybe this is why Dave is asking for a better BOOST_TYPEOF. ;-)
Well, as long as the result type of the final fold has been registered, it should work out of the box.
I think that the basic ranges that should (at least) be supported are:
- mapped_range - filtered_range - taken_range (i.e. just the first n elements of a range) - taken_while_range (i.e. the first elements such as a predicate is true) - zipped_range - unfold_range (with which you can pretty much express any other range, see http://tinyurl.com/5mus25)
An eventual 'drop' (take all the elements after the first n) and 'drop_while' (drop the first elements such as a predicate is true) probably need strict evaluation anyway and aren't worth supporting explicitly.
I also think there should be a zipped_function which:
- takes N function objects and an N-tuple - applies each function to the appropriate tuple - returns a tuple of results
D'oh, of course, forgot this. I think that zipped_range on top of zipped_iterator is enough, you get the other functionality by composing with transformed_iterator.
Okay, but shouldn't this composition with transform_iterator be done with something that 'comes out of the box'?
It is ok to provide a zip+transform in a single function, but it is not a primitive. Let's call it zip_with, its result type should be transformed_range<zipped_range<RangeTuple> , Mapper>, we do not need an explicit zippedWith_range (and in fact it would make deforestation harder). -- gpd
On Tue, Jul 1, 2008 at 8:28 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Tue, Jul 1, 2008 at 12:50 PM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
Isn't the whole point of creating an expression template based-approach to create these (maybe using proto I guess) and allow:
1. The same placeholders used in Phoenix+Proto to be used in the pipe syntax in range_ex (or an imaginary new RangeEx implementation). 2. Allow for efficient (i.e., compiler optimized/assisted) implementations of the composition of these new views/ranges/iterators.
Of course. My point is that you need very few beyond what is available in Phoenix. Just the ability to have a placeholder which doesn't make the whole expression lazy ( i.e. foo(_) should be the same as unlambda(bind(foo, _1)) ).
When you say the whole expression lazy, do you mean even the for_each? Why don't you want the range generation functions (or the pipe syntax) return a lazy range? Perhaps something that only gets activated when begin(range) and end(range) are invoked?
?
Although I do like how this looks granted that you are using C++0x's auto -- in today's C++, the reason I used the STL was to avoid having to compute the actual type produced by the pipe syntax.
You need to know the result type of the reducer even in your example, so it is not specific to the 'sugared' syntax :)
for_each ( range(source) | take(10) | map(mapper()), cout << arg1 << endl ); // with Phoenix Assuming that there's a boost::for_each(range<T>, F) -- I don't think I need to know the result type of the reducer in that example. Am I missing something?
Maybe this is why Dave is asking for a better BOOST_TYPEOF. ;-)
Well, as long as the result type of the final fold has been registered, it should work out of the box.
Of course, registered manually you mean? Or do you rely on compiler-specific 'typeof'?
Okay, but shouldn't this composition with transform_iterator be done with something that 'comes out of the box'?
It is ok to provide a zip+transform in a single function, but it is not a primitive. Let's call it zip_with, its result type should be transformed_range<zipped_range<RangeTuple> , Mapper>, we do not need an explicit zippedWith_range (and in fact it would make deforestation harder).
Shouldn't the deforestation already have happened in the zipping and application of the function to yield another tuple? -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
On Wed, Jul 2, 2008 at 4:45 AM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
On Tue, Jul 1, 2008 at 8:28 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Tue, Jul 1, 2008 at 12:50 PM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
Isn't the whole point of creating an expression template based-approach to create these (maybe using proto I guess) and allow:
1. The same placeholders used in Phoenix+Proto to be used in the pipe syntax in range_ex (or an imaginary new RangeEx implementation). 2. Allow for efficient (i.e., compiler optimized/assisted) implementations of the composition of these new views/ranges/iterators.
Of course. My point is that you need very few beyond what is available in Phoenix. Just the ability to have a placeholder which doesn't make the whole expression lazy ( i.e. foo(_) should be the same as unlambda(bind(foo, _1)) ).
When you say the whole expression lazy, do you mean even the for_each?
Why don't you want the range generation functions (or the pipe syntax) return a lazy range? Perhaps something that only gets activated when begin(range) and end(range) are invoked?
Wait, there is a bit of confusion. There are two kind of lazyness: the first kind are lazily evaluated ranges, like those that use transform_iterator, filter_iterator, etc... the other kind are lazily evaluated functions like those in Phoenix (i.e. higher order functions that return other functions). Let's take a map function. It can be lazy in two ways: map(range, mapper) [1] returns a lazy range (i.e. the moral equivalent of two transform_iterators). map(_1, mapper) [2] returns a lazy unary function that takes a range and returns another (lazy range). There is a third syntax that can be used: range | map(mapper) [3] Now, in the usual phoenix world, the 'map' in [1] and [2] would actually be two different functions (functions in phoenix are always lazy even if all arguments are provided). Map in [3] must be again a different function. This means that a lazy range algorithm should provide all three variants to satisfy all the use cases. Unless... Maps in [1] and [2] can actually be the same function by adding "lazyness on demand" (which is not hard to do). Also lazy functions let's us implement the third syntax easily: Let's assume that 'operator|(r, f)' is a global operator that returns 'f(r)'. This doesn't work quite right though: range | map(_1, mapper) Because map(_1, mapper) doesn't simply return an unary function, but it is actually an expression template, which will try to make operator| 'its own'. You need to stop the expression template machinery. In boost.lambda you use 'unlambda', in phoenix I think that the magic keyword is 'lambda[]': range | lambda[ map(_1, mapper) ] It should works, but the 'lambda' is a bit ugly. If we introduced a new placeholder, like _, we could make the syntax more lightweight: range | map(_, mapper) [4] I.e. _ instructs phoenix to automatically wrap the lazy function in a lambda block. Note that this syntax is quite similar to the syntax in [3]. if we followed FC++ lead and swapper the order of the range and mapper argument in 'map', we could add a rule that says that any missing argument is automatically substituted with a _ (in practice, currying, in fact FC++ itself introduced the '_' syntax for generalized currying): range | map(mapper, _) -> range | map(mapper) [5] Nice! Unfortunately, automatic currying is (very arguably) error prone, and most importantly, it makes it impossible to have variadic functions (for example ,in sort(range), how do you know if this is a curried expression or you are using the default comparator?). I think that [4] is desirable, but [5] is not worth it.
?
Although I do like how this looks granted that you are using C++0x's auto -- in today's C++, the reason I used the STL was to avoid having to compute the actual type produced by the pipe syntax.
You need to know the result type of the reducer even in your example, so it is not specific to the 'sugared' syntax :)
for_each ( range(source) | take(10) | map(mapper()), cout << arg1 << endl ); // with Phoenix
Assuming that there's a boost::for_each(range<T>, F) -- I don't think I need to know the result type of the reducer in that example. Am I missing something?
There is no reducer (i.e. a fold) in the above expression. If you want to reduce a range you need to know the result type of the reducer function if you want to actually save the final result (which is usually the case :) ).
Maybe this is why Dave is asking for a better BOOST_TYPEOF. ;-)
Well, as long as the result type of the final fold has been registered, it should work out of the box.
Of course, registered manually you mean? Or do you rely on compiler-specific 'typeof'?
manually if there is no native typeof available.
Okay, but shouldn't this composition with transform_iterator be done with something that 'comes out of the box'?
It is ok to provide a zip+transform in a single function, but it is not a primitive. Let's call it zip_with, its result type should be transformed_range<zipped_range<RangeTuple> , Mapper>, we do not need an explicit zippedWith_range (and in fact it would make deforestation harder).
Shouldn't the deforestation already have happened in the zipping and application of the function to yield another tuple?
let say you have the rewrite rule: map<map<Range,A>,B> -> map<Range, compose<A,B> > If zip_with returns a map<zip<R1, R2>, A>, you can simplify the expression map(zip_with(r1, r2, a), b) as: map<map<zip<R1, R2>, A>, B> -> map<zip<R1, R2>, compose<A,B> > without any other special rule. If zip_with were to return a special zip<R1, R2, A>, you need a specific rule to handle this case. I.e. zip_with shouldn't be a primitive, but should be constructed on top of map and zip. HTH, -- gpd
On Wed, Jul 2, 2008 at 6:25 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
Wait, there is a bit of confusion. There are two kind of lazyness: the first kind are lazily evaluated ranges, like those that use transform_iterator, filter_iterator, etc... the other kind are lazily evaluated functions like those in Phoenix (i.e. higher order functions that return other functions).
Let's take a map function. It can be lazy in two ways:
map(range, mapper) [1]
returns a lazy range (i.e. the moral equivalent of two transform_iterators).
map(_1, mapper) [2]
returns a lazy unary function that takes a range and returns another (lazy range). There is a third syntax that can be used:
range | map(mapper) [3]
Now, in the usual phoenix world, the 'map' in [1] and [2] would actually be two different functions (functions in phoenix are always lazy even if all arguments are provided). Map in [3] must be again a different function. This means that a lazy range algorithm should provide all three variants to satisfy all the use cases. Unless...
Maps in [1] and [2] can actually be the same function by adding "lazyness on demand" (which is not hard to do). Also lazy functions let's us implement the third syntax easily: Let's assume that 'operator|(r, f)' is a global operator that returns 'f(r)'. This doesn't work quite right though:
range | map(_1, mapper)
Because map(_1, mapper) doesn't simply return an unary function, but it is actually an expression template, which will try to make operator| 'its own'. You need to stop the expression template machinery. In boost.lambda you use 'unlambda', in phoenix I think that the magic keyword is 'lambda[]':
range | lambda[ map(_1, mapper) ]
It should works, but the 'lambda' is a bit ugly. If we introduced a new placeholder, like _, we could make the syntax more lightweight:
range | map(_, mapper) [4]
I.e. _ instructs phoenix to automatically wrap the lazy function in a lambda block. Note that this syntax is quite similar to the syntax in [3]. if we followed FC++ lead and swapper the order of the range and mapper argument in 'map', we could add a rule that says that any missing argument is automatically substituted with a _ (in practice, currying, in fact FC++ itself introduced the '_' syntax for generalized currying):
range | map(mapper, _) -> range | map(mapper) [5]
Nice! Unfortunately, automatic currying is (very arguably) error prone, and most importantly, it makes it impossible to have variadic functions (for example ,in sort(range), how do you know if this is a curried expression or you are using the default comparator?).
I think that [4] is desirable, but [5] is not worth it.
Thanks for the detailed explanation. I on the other hand am thinking of something a little bit different: Consider: operator|(r, f) -> lazy_range<typeof(r), apply<typeof(f)> > Now that allows you to chain operator|(r, f) infinitely, just making sure that along the way you create an instance of a specific type (lazy_range) which when applied a begin(...) or end(...) function will yield the appropriately constructed iterator. Note that in the examples we've been giving: source | take(10) | filter(filter_function()) | map(mapper()) We'd have a lazy_range<...> which has a type: lazy_range< lazy_range< lazy_range< range_type, taken_range_generator >, filer_function
, mapper
An instance of this is returned by the whole expression template (the argument to the construction of the instance would be the original source range), which we can call begin(...) or end(...) against, and get the correct iterator type. If we have the above typedefed as 'some_lazy_range', doing: begin(some_lazy_range(source)) -> the beginning of the lazy range end(some_lazy_range()) -> the 'end' iterator given the lazy range
for_each ( range(source) | take(10) | map(mapper()), cout << arg1 << endl ); // with Phoenix
Assuming that there's a boost::for_each(range<T>, F) -- I don't think I need to know the result type of the reducer in that example. Am I missing something?
There is no reducer (i.e. a fold) in the above expression. If you want to reduce a range you need to know the result type of the reducer function if you want to actually save the final result (which is usually the case :) ).
Right. Maybe that's a bad example. Considering that we can make a reducer an external function and adapted using 'make_function_output_iterator', we can use it in std::copy (of course something like boost::copy(range, function) ): reducer reducer_instance; copy ( source | take(10) | map(mapper()) , make_function_output_iterator(reducer_instance) ); In this case I really don't need to know the exact type of the range(s) created by the pipe syntax, because type deduction will do the trick for me.
Maybe this is why Dave is asking for a better BOOST_TYPEOF. ;-)
Well, as long as the result type of the final fold has been registered, it should work out of the box.
Of course, registered manually you mean? Or do you rely on compiler-specific 'typeof'?
manually if there is no native typeof available.
Okay... I think this is the part that sucks somehow (at least before C++0x).
Shouldn't the deforestation already have happened in the zipping and application of the function to yield another tuple?
let say you have the rewrite rule: map<map<Range,A>,B> -> map<Range, compose<A,B> >
If zip_with returns a map<zip<R1, R2>, A>, you can simplify the expression map(zip_with(r1, r2, a), b) as:
map<map<zip<R1, R2>, A>, B> -> map<zip<R1, R2>, compose<A,B> >
without any other special rule. If zip_with were to return a special zip<R1, R2, A>, you need a specific rule to handle this case. I.e. zip_with shouldn't be a primitive, but should be constructed on top of map and zip.
True, but zip_with can return a range: map<map<zip<transform<R1, F1>, transform<R2, F2> > >, A>, B> which can still be re-written. BTW, are the rewrite rules already there, or will this have to be something that has to still be written using Boost.MPL? -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
On Thu, Jul 3, 2008 at 2:32 PM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
On Wed, Jul 2, 2008 at 6:25 PM, Giovanni Piero Deretta <snip> Consider:
operator|(r, f) -> lazy_range<typeof(r), apply<typeof(f)> >
IMHO operator| shouldn't be restricted to ranges. Logically it should just apply its lhs to its rhs, regardless of the type of both lhs and rhs [1]. It is not the job of operator| to create lazy ranges, but of 'f'. [1] in fact, lacking concepts, rhs or lhs should provide a way to find operator| via adl, for example inheriting from a pipeable empty class. Putting operator| in the global namespace is not an option.
Now that allows you to chain operator|(r, f) infinitely, just making sure that along the way you create an instance of a specific type (lazy_range) which when applied a begin(...) or end(...) function will yield the appropriately constructed iterator.
what is the advantage with respect to having 'f' construct the lazy range instead? It just complicates the implementation of operator|.
Note that in the examples we've been giving:
source | take(10) | filter(filter_function()) | map(mapper())
We'd have a lazy_range<...> which has a type:
lazy_range< lazy_range< lazy_range< range_type, taken_range_generator
, filer_function , mapper
how does lazy_range distinguish from a filter range and a map range? lazy_range<A, B> has to somehow delegate the decision to the B parameter. At that point you might remove all the smartness from operator| and move it to B. Also, how is the above different from mapped_range<filtered_range<taken_range<range_type>, Filter>,Mapper> which is still lazy?
An instance of this is returned by the whole expression template (the argument to the construction of the instance would be the original source range), which we can call begin(...) or end(...) against, and get the correct iterator type. If we have the above typedefed as 'some_lazy_range', doing:
begin(some_lazy_range(source)) -> the beginning of the lazy range end(some_lazy_range()) -> the 'end' iterator given the lazy range
How this is an unique property of lazy_range? This would work with all the range_ex proposals I have seen so far.
for_each ( range(source) | take(10) | map(mapper()), cout << arg1 << endl ); // with Phoenix
Assuming that there's a boost::for_each(range<T>, F) -- I don't think I need to know the result type of the reducer in that example. Am I missing something?
There is no reducer (i.e. a fold) in the above expression. If you want to reduce a range you need to know the result type of the reducer function if you want to actually save the final result (which is usually the case :) ).
Right. Maybe that's a bad example. Considering that we can make a reducer an external function and adapted using 'make_function_output_iterator', we can use it in std::copy (of course something like boost::copy(range, function) ):
reducer reducer_instance; copy ( source | take(10) | map(mapper()) , make_function_output_iterator(reducer_instance) );
A Reduce is a binary function: it takes the accumulated value and the i'th element to generate the new accumulated value (think std::accumulate). So, the above cannot work: an output iterator is logically unary, not binary, unless the reducer stores the accumulator value 'inside' (then you should be catching the result of copy to retrieve the accumulated result). In any case, you need to know the type of the accumulated value, and you are no better than in my example.
In this case I really don't need to know the exact type of the range(s) created by the pipe syntax, because type deduction will do the trick for me.
You do not need to know the exact type of the range in my example either. Just the value of the type returned by the final reducer. A reducer doesn't usually (but not always) return a range.
Shouldn't the deforestation already have happened in the zipping and application of the function to yield another tuple?
let say you have the rewrite rule: map<map<Range,A>,B> -> map<Range, compose<A,B> >
If zip_with returns a map<zip<R1, R2>, A>, you can simplify the expression map(zip_with(r1, r2, a), b) as:
map<map<zip<R1, R2>, A>, B> -> map<zip<R1, R2>, compose<A,B> >
without any other special rule. If zip_with were to return a special zip<R1, R2, A>, you need a specific rule to handle this case. I.e. zip_with shouldn't be a primitive, but should be constructed on top of map and zip.
True, but zip_with can return a range: map<map<zip<transform<R1, F1>, transform<R2, F2> > >, A>, B> which can still be re-written.
eh? What is transform there? What are F1 and F2?
BTW, are the rewrite rules already there, or will this have to be something that has to still be written using Boost.MPL?
Of course not, for now this is just 'wouldn't it be great if...' :) -- gpd
Sorry this took a while. On Thu, Jul 3, 2008 at 9:20 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Thu, Jul 3, 2008 at 2:32 PM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
operator|(r, f) -> lazy_range<typeof(r), apply<typeof(f)> >
IMHO operator| shouldn't be restricted to ranges. Logically it should just apply its lhs to its rhs, regardless of the type of both lhs and rhs [1]. It is not the job of operator| to create lazy ranges, but of 'f'.
[1] in fact, lacking concepts, rhs or lhs should provide a way to find operator| via adl, for example inheriting from a pipeable empty class. Putting operator| in the global namespace is not an option.
Right.
Now that allows you to chain operator|(r, f) infinitely, just making sure that along the way you create an instance of a specific type (lazy_range) which when applied a begin(...) or end(...) function will yield the appropriately constructed iterator.
what is the advantage with respect to having 'f' construct the lazy range instead? It just complicates the implementation of operator|.
The advantage would be that if operator| constructed the type by itself compared to having 'f' construct the type at runtime, then the compiler can (and usually would) perform optimizations on the code it generates for the type created by operator|. This is the same reason why expression templates work very well for Blitz++ and the ublas library IIRC. The argument goes like: matrix A, B, C; C = A * B; This would be fine if you just had an operator* overload that worked for two matrices -- then your problem becomes the temporary object created by the following: matrix A, B, C, D; D = A * B * C; If you use expression templates in this situation, you would do away with the temporary object(s) created in the intermediate multiplication -- and have the compiler optimize the code it creates for you because at compile time you're dealing first with types rather than values deferred at runtime. The final assignment in the above line would perform the necessary traversal(s) and assignments with the benefit of the compiler having optimized already everything it would have had optimized. In the case of implementing operator| in this way, we are then able to compose/combine the chained lazy range implementation by building the type at compile time -- where at runtime, the code would have benefitted from compiler opimizations. More concretely, if 'f' created the lazy range, then you'd be passing the instance of the temporary range to the next 'f' in the chain, then the next, then the next... Then the temporaries you'd be creating are wasteful which you would avoid when using expression templates. Having a 'lazy_range' allows you to apply your own re-write rules (like what you mention below) when the nested 'type' class is accessed to yield a real instance of the range.
Note that in the examples we've been giving:
source | take(10) | filter(filter_function()) | map(mapper())
We'd have a lazy_range<...> which has a type:
lazy_range< lazy_range< lazy_range< range_type, taken_range_generator
, filer_function , mapper
how does lazy_range distinguish from a filter range and a map range? lazy_range<A, B> has to somehow delegate the decision to the B parameter. At that point you might remove all the smartness from operator| and move it to B.
You use tag dispatching at compile time.
Also, how is the above different from mapped_range<filtered_range<taken_range<range_type>, Filter>,Mapper> which is still lazy?
Not much except the actual type yielded by 'lazy_range<T>::type' can be what mapped_range<filtered_range<taken_range<range_type>, Filter>,Mapper>.
An instance of this is returned by the whole expression template (the argument to the construction of the instance would be the original source range), which we can call begin(...) or end(...) against, and get the correct iterator type. If we have the above typedefed as 'some_lazy_range', doing:
begin(some_lazy_range(source)) -> the beginning of the lazy range end(some_lazy_range()) -> the 'end' iterator given the lazy range
How this is an unique property of lazy_range? This would work with all the range_ex proposals I have seen so far.
Without expression templates, you'll need extra temporaries (which is wasteful) to achieve things with merely lamda's.
Right. Maybe that's a bad example. Considering that we can make a reducer an external function and adapted using 'make_function_output_iterator', we can use it in std::copy (of course something like boost::copy(range, function) ):
reducer reducer_instance; copy ( source | take(10) | map(mapper()) , make_function_output_iterator(reducer_instance) );
A Reduce is a binary function: it takes the accumulated value and the i'th element to generate the new accumulated value (think std::accumulate). So, the above cannot work: an output iterator is logically unary, not binary, unless the reducer stores the accumulator value 'inside' (then you should be catching the result of copy to retrieve the accumulated result). In any case, you need to know the type of the accumulated value, and you are no better than in my example.
Okay, I confused the reducer as merely the function which took in values and performed the reduction or accumulation in a stateful manner.
In this case I really don't need to know the exact type of the range(s) created by the pipe syntax, because type deduction will do the trick for me.
You do not need to know the exact type of the range in my example either. Just the value of the type returned by the final reducer. A reducer doesn't usually (but not always) return a range.
Right.
Shouldn't the deforestation already have happened in the zipping and application of the function to yield another tuple?
let say you have the rewrite rule: map<map<Range,A>,B> -> map<Range, compose<A,B> >
If zip_with returns a map<zip<R1, R2>, A>, you can simplify the expression map(zip_with(r1, r2, a), b) as:
map<map<zip<R1, R2>, A>, B> -> map<zip<R1, R2>, compose<A,B> >
without any other special rule. If zip_with were to return a special zip<R1, R2, A>, you need a specific rule to handle this case. I.e. zip_with shouldn't be a primitive, but should be constructed on top of map and zip.
True, but zip_with can return a range: map<map<zip<transform<R1, F1>, transform<R2, F2> > >, A>, B> which can still be re-written.
eh? What is transform there? What are F1 and F2?
Given that zipped_with can result a zipped transform iterator, and F1 and F2 are the types of the functions used/provided with zip_with.
BTW, are the rewrite rules already there, or will this have to be something that has to still be written using Boost.MPL?
Of course not, for now this is just 'wouldn't it be great if...' :)
Which is precisely what can be implemented with lazy_range<...>. :D HTH -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
On Mon, Jul 7, 2008 at 6:29 AM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
Sorry this took a while.
On Thu, Jul 3, 2008 at 9:20 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Thu, Jul 3, 2008 at 2:32 PM, Dean Michael Berris
Now that allows you to chain operator|(r, f) infinitely, just making sure that along the way you create an instance of a specific type (lazy_range) which when applied a begin(...) or end(...) function will yield the appropriately constructed iterator.
what is the advantage with respect to having 'f' construct the lazy range instead? It just complicates the implementation of operator|.
The advantage would be that if operator| constructed the type by itself compared to having 'f' construct the type at runtime, then the compiler can (and usually would) perform optimizations on the code it generates for the type created by operator|.
Which optimizations cannot be performed by an 'f' that returns a lazy range?
This is the same reason why expression templates work very well for Blitz++ and the ublas library IIRC.
<snip blitz example>
If you use expression templates in this situation, you would do away with the temporary object(s) created in the intermediate multiplication -- and have the compiler optimize the code it creates for you because at compile time you're dealing first with types rather than values deferred at runtime. The final assignment in the above line would perform the necessary traversal(s) and assignments with the benefit of the compiler having optimized already everything it would have had optimized.
In the case of implementing operator| in this way, we are then able to compose/combine the chained lazy range implementation by building the type at compile time
This is not a property of operator|(). Any lazy range library I have seen does it.
-- where at runtime, the code would have benefitted from compiler opimizations.
More concretely, if 'f' created the lazy range, then you'd be passing the instance of the temporary range to the next 'f' in the chain, then the next, then the next... Then the temporaries you'd be creating are wasteful which you would avoid when using expression templates.
The lazy range temporaries returned by the various 'f' are the equivalent expression templates. Lazy ranges are meant to be very lightweight (little more that two iterators). I see 0 advantages to adding another level of complication.
Having a 'lazy_range' allows you to apply your own re-write rules (like what you mention below) when the nested 'type' class is accessed to yield a real instance of the range.
You can do rewrite rules any way. <snip>
Also, how is the above different from mapped_range<filtered_range<taken_range<range_type>, Filter>,Mapper> which is still lazy?
Not much except the actual type yielded by 'lazy_range<T>::type' can be what mapped_range<filtered_range<taken_range<range_type>, Filter>,Mapper>.
The advantage is?
An instance of this is returned by the whole expression template (the argument to the construction of the instance would be the original source range), which we can call begin(...) or end(...) against, and get the correct iterator type. If we have the above typedefed as 'some_lazy_range', doing:
begin(some_lazy_range(source)) -> the beginning of the lazy range end(some_lazy_range()) -> the 'end' iterator given the lazy range
How this is an unique property of lazy_range? This would work with all the range_ex proposals I have seen so far.
Without expression templates, you'll need extra temporaries (which is wasteful) to achieve things with merely lamda's.
Why the extra temporaries will be wasteful? They will be the equivalent of iterator pairs. In fact they will be exactly the same thing as a node in an expression template. <snip>
[...] zip_with can return a range: map<map<zip<transform<R1, F1>, transform<R2, F2> > >, A>, B> which can still be re-written.
eh? What is transform there? What are F1 and F2?
Given that zipped_with can result a zipped transform iterator, and F1 and F2 are the types of the functions used/provided with zip_with.
I do not understand. Why you have both map and transform in the same expression? Map and transform are the same thing, why do you use two different names? IMHO zipWith(r1, r2, f) should return mapped_range< zipped_range<R1, R2>, F > Which is the same as map(zip(r1, r2), f).
BTW, are the rewrite rules already there, or will this have to be something that has to still be written using Boost.MPL?
Of course not, for now this is just 'wouldn't it be great if...' :)
Which is precisely what can be implemented with lazy_range<...>. :D
See attached for rewrite rules implemented *without* lazy_range. -- gpd
On Mon, Jul 7, 2008 at 5:43 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Mon, Jul 7, 2008 at 6:29 AM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
The advantage would be that if operator| constructed the type by itself compared to having 'f' construct the type at runtime, then the compiler can (and usually would) perform optimizations on the code it generates for the type created by operator|.
Which optimizations cannot be performed by an 'f' that returns a lazy range?
Removing the need to actually create temporary objects.
This is the same reason why expression templates work very well for Blitz++ and the ublas library IIRC.
<snip blitz example>
If you use expression templates in this situation, you would do away with the temporary object(s) created in the intermediate multiplication -- and have the compiler optimize the code it creates for you because at compile time you're dealing first with types rather than values deferred at runtime. The final assignment in the above line would perform the necessary traversal(s) and assignments with the benefit of the compiler having optimized already everything it would have had optimized.
In the case of implementing operator| in this way, we are then able to compose/combine the chained lazy range implementation by building the type at compile time
This is not a property of operator|(). Any lazy range library I have seen does it.
True, but these lazy ranges aren't lazy enough -- precisely because they rely on temporaries returned from the previously chained expressions.
-- where at runtime, the code would have benefitted from compiler opimizations.
More concretely, if 'f' created the lazy range, then you'd be passing the instance of the temporary range to the next 'f' in the chain, then the next, then the next... Then the temporaries you'd be creating are wasteful which you would avoid when using expression templates.
The lazy range temporaries returned by the various 'f' are the equivalent expression templates. Lazy ranges are meant to be very lightweight (little more that two iterators). I see 0 advantages to adding another level of complication.
When you have a chain of at least three operations, you then need to have at least two returned temporary objects -- I don't know about you but inside a tight loop that's not good especially if the compiler doesn't remove the temporaries for you (which I don't think there's a standard optimization for anyway, without move semantics or something similar that will eliminate the temporaries). The complication introduced should be offset by the efficiency of not having to instantiate and potentially destroy temporary objects when chaining using the compile-time built types. The temporary object construction, copy, and destruction is the killer that when you do it in a tight enough loop (or even enough times in the case of asynchronous massively parallel high performance applications) -- the only recourse is to create *by hand* the exact range type you want so that you don't rely on the temporary objects returned by chaining operations; which actually defeats the purpose of introducing the chaining using operator| in the first place (for ease of use). So, if operator| built the *type* at compile time without having to create temporaries, you get both the expressive power of operator| and the efficiency brought about by the expression templates.
Having a 'lazy_range' allows you to apply your own re-write rules (like what you mention below) when the nested 'type' class is accessed to yield a real instance of the range.
You can do rewrite rules any way.
True, but only if you knew the type beforehand and invoke the rewrite explicitly.
<snip>
Also, how is the above different from mapped_range<filtered_range<taken_range<range_type>, Filter>,Mapper> which is still lazy?
Not much except the actual type yielded by 'lazy_range<T>::type' can be what mapped_range<filtered_range<taken_range<range_type>, Filter>,Mapper>.
The advantage is?
The advantage is that you built the type using zero temporaries, and get the exact type you want at the cost of an extra nested type instantiation.
Without expression templates, you'll need extra temporaries (which is wasteful) to achieve things with merely lamda's.
Why the extra temporaries will be wasteful? They will be the equivalent of iterator pairs. In fact they will be exactly the same thing as a node in an expression template.
The node in an expression template will be encapsulated in the type you're building, rather than an actual temporary object that gets constructed and destroyed at run-time.
<snip>
[...] zip_with can return a range: map<map<zip<transform<R1, F1>, transform<R2, F2> > >, A>, B> which can still be re-written.
eh? What is transform there? What are F1 and F2?
Given that zipped_with can result a zipped transform iterator, and F1 and F2 are the types of the functions used/provided with zip_with.
I do not understand. Why you have both map and transform in the same expression? Map and transform are the same thing, why do you use two different names?
IMHO zipWith(r1, r2, f) should return
mapped_range< zipped_range<R1, R2>, F >
Which is the same as map(zip(r1, r2), f).
Actually, I was thinking more: zip_with(r1, r2, f1, f2) Where you get: zipped_range< mapped_range<R1, F1>, mapped_range<R2, F2> >. Without variadic templates though it would have to look like: zip_with( make_tuple(r1, r2), make_tuple(f1, f2) ); In turn, you get a zipped range, where each element is an iterator to a tuple that is the result of the operation f1(*r1_it) and f2(*r2_it), and so on.
Which is precisely what can be implemented with lazy_range<...>. :D
See attached for rewrite rules implemented *without* lazy_range.
I understand that this can be implemented without lazy_range, but the point of using expression templates to build the lazy_range is to be able to implement the rewrite rules as part of the lazy_range implementation. -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
On Mon, Jul 7, 2008 at 12:19 PM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
On Mon, Jul 7, 2008 at 5:43 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Mon, Jul 7, 2008 at 6:29 AM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
The advantage would be that if operator| constructed the type by itself compared to having 'f' construct the type at runtime, then the compiler can (and usually would) perform optimizations on the code it generates for the type created by operator|.
Which optimizations cannot be performed by an 'f' that returns a lazy range?
Removing the need to actually create temporary objects.
You would be substituting one light weight temporary (a lazy range) for another one, the expression template node.
This is the same reason why expression templates work very well for Blitz++ and the ublas library IIRC.
<snip blitz example>
If you use expression templates in this situation, you would do away with the temporary object(s) created in the intermediate multiplication -- and have the compiler optimize the code it creates for you because at compile time you're dealing first with types rather than values deferred at runtime. The final assignment in the above line would perform the necessary traversal(s) and assignments with the benefit of the compiler having optimized already everything it would have had optimized.
In the case of implementing operator| in this way, we are then able to compose/combine the chained lazy range implementation by building the type at compile time
This is not a property of operator|(). Any lazy range library I have seen does it.
True, but these lazy ranges aren't lazy enough -- precisely because they rely on temporaries returned from the previously chained expressions.
Why would that make them not lazy enough? I could say: "an expression template node is not lazy enough because it relies on temporaries returned from the previously chained expressions"
-- where at runtime, the code would have benefitted from compiler opimizations.
More concretely, if 'f' created the lazy range, then you'd be passing the instance of the temporary range to the next 'f' in the chain, then the next, then the next... Then the temporaries you'd be creating are wasteful which you would avoid when using expression templates.
The lazy range temporaries returned by the various 'f' are the equivalent expression templates. Lazy ranges are meant to be very lightweight (little more that two iterators). I see 0 advantages to adding another level of complication.
When you have a chain of at least three operations, you then need to have at least two returned temporary objects -- I don't know about you but inside a tight loop that's not good especially if the compiler doesn't remove the temporaries for you (which I don't think there's a standard optimization for anyway, without move semantics or something similar that will eliminate the temporaries).
Those temporaries are a bunch of bytes, no dynamic memory allocation, at most a couple of iterators. Did you see the example I attached in my other post? Which temporaries would you like to see removed and how would you remove them? Using rewrite rules (which is exactly like traversing an expression template to optimize it), you can simplify expressions containing more than one range, but after some tests I didn't actually measure any difference between a plain for loop, non-optimized range expressions and optimized range expressions. I.e. the compiler is smart enough to simplify it for you. May be I'm misunderstanding you; could you provide an example and show exactly where is the difference between expression templates and lazy ranges (IMHO they are exactly the same thing)?
The complication introduced should be offset by the efficiency of not having to instantiate and potentially destroy temporary objects when chaining using the compile-time built types. The temporary object construction, copy, and destruction is the killer that when you do it in a tight enough loop (or even enough times in the case of asynchronous massively parallel high performance applications)
eh? These temporaries are constructed of objects having trivial destructors: iterators (if at all), function objects and references. Any compiler written in the last 15 years can completely omit a call to such a destructor.
-- the only recourse is to create *by hand* the exact range type you want so that you don't rely on the temporary objects returned by chaining operations; which actually defeats the purpose of introducing the chaining using operator| in the first place (for ease of use).
I can't still measure any abstraction overhead with a modern compiler and at least filter_iterator and mapped_iterator, without any extra expression template. BTW, letting operator| do all magic, means that you get no benefit if you actually not use it. I would expect that the only difference between: v = reduce(map(filter(x, f), m), r) and v = x | map(_, m) | filter(_, f) | reduce(_, r) were only the syntactic sugar, not any performance difference (i.e. the translation between the two is trivial) <snip>
Having a 'lazy_range' allows you to apply your own re-write rules (like what you mention below) when the nested 'type' class is accessed to yield a real instance of the range.
You can do rewrite rules any way.
True, but only if you knew the type beforehand and invoke the rewrite explicitly.
There is no need to invoke the rule explicitly. It is magic :) Really, I think that we are talking basically about the same thing. We are just discussing about details. In my view, chains of lazy ranges *are* a form of expression templates, and I cannot see what a smart operator| would add to that: lazy ranges encode the same information that you would encode in an expression template, with the same (or lack of) penality. Maybe you could show a proof of concept implementation... -- gpd
Sorry this took a while again... I promise I'll be more prompt with responses next time an interesting discussion like this is going on. ;-) On Mon, Jul 7, 2008 at 10:48 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Mon, Jul 7, 2008 at 12:19 PM, Dean Michael Berris <mikhailberis@gmail.com> wrote: <snip>
Removing the need to actually create temporary objects.
You would be substituting one light weight temporary (a lazy range) for another one, the expression template node.
Right, but the expression template "node" will just contain references -- and actual iterators will only be built once with the exact type, instead of encapsulating and creating multiple range objects passed from function to function.
True, but these lazy ranges aren't lazy enough -- precisely because they rely on temporaries returned from the previously chained expressions.
Why would that make them not lazy enough? I could say: "an expression template node is not lazy enough because it relies on temporaries returned from the previously chained expressions"
I'm actually thinking of making it lazier than just using lazy ranges returned from functions in a chain: reduce( map( filter( source, f), m, r ) In this case when you have: template <class R, class F> result_of<F(R::iterator::value_type)>::type reduce(R const & r, F const & f) { return f(begin(r), end(r), I::value_type()); }; template <class R, class F> mapped_range<R, F> map(R const & r, F const & f) { return mapped_range<R, F>(r, f); }; template <class R, class F> filtered_range<R, F> filter(R const & r, F const & f) { return filtered_range<R, F>(r, f); }; Consider the number of temporaries you actually create when you chain them together, compared to something like this: source | filter(f) | map(m) | reduce(r) -> T In essence, what I was thinking is: template <class T> T create(T) { return T(); }; create(source | filter(f) | map(m) | reduce(r))(make_tuple(source, f, m, r)) where create(T) -> T() T()(sequence) -> actual range type So what should happen is, create(T (*)(_, _)) will create a single temporary object T (because the compiler would just want the type T generated from the chained pipe expression), which is polymorphic on the function operator overload that lazily creates the correct range type given the sequence argument. This is similar to the technique used in lambda's, but this time you just want the return type of the function invocation built by operator|(...) so that you can create the correct range generator type. I might be dreaming if this can be done, since I haven't had time to actually try it out. ;-)
When you have a chain of at least three operations, you then need to have at least two returned temporary objects -- I don't know about you but inside a tight loop that's not good especially if the compiler doesn't remove the temporaries for you (which I don't think there's a standard optimization for anyway, without move semantics or something similar that will eliminate the temporaries).
Those temporaries are a bunch of bytes, no dynamic memory allocation, at most a couple of iterators. Did you see the example I attached in my other post? Which temporaries would you like to see removed and how would you remove them?
Using rewrite rules (which is exactly like traversing an expression template to optimize it), you can simplify expressions containing more than one range, but after some tests I didn't actually measure any difference between a plain for loop, non-optimized range expressions and optimized range expressions. I.e. the compiler is smart enough to simplify it for you.
May be I'm misunderstanding you; could you provide an example and show exactly where is the difference between expression templates and lazy ranges (IMHO they are exactly the same thing)?
I tried above, but generally I think there's a difference between creating the type first at compile time then reducing the number of steps actually done at runtime (constructing and destroying objects or even a bunch of bytes). The idea brought about by the expression templates allows someone to implement considerably complex operations using syntactic sugar (or expressive notation) while at the same time getting the benefit of the efficiency afforded us by the compiler. So in essence, you'd want expression templates because if: reduce(map(filter(source, f), m), r) Is less efficient than: source | filter(f) | map(m) | reduce(r) Then that's a good reason to either: make the range generation functions as efficient as the expression template approach, or remove the range generation functions in favor of the more concise (and efficient) pipe syntax.
The complication introduced should be offset by the efficiency of not having to instantiate and potentially destroy temporary objects when chaining using the compile-time built types. The temporary object construction, copy, and destruction is the killer that when you do it in a tight enough loop (or even enough times in the case of asynchronous massively parallel high performance applications)
eh? These temporaries are constructed of objects having trivial destructors: iterators (if at all), function objects and references. Any compiler written in the last 15 years can completely omit a call to such a destructor.
Right, and I'd think 0 bytes of overhead is always better than some bytes of overhead. ;-) I'm nitpicking, but the idea really is if you can actually remove all the temporaries or minimize it to one (or none at all?), then we should think about that approach and see the merits.
-- the only recourse is to create *by hand* the exact range type you want so that you don't rely on the temporary objects returned by chaining operations; which actually defeats the purpose of introducing the chaining using operator| in the first place (for ease of use).
I can't still measure any abstraction overhead with a modern compiler and at least filter_iterator and mapped_iterator, without any extra expression template.
BTW, letting operator| do all magic, means that you get no benefit if you actually not use it. I would expect that the only difference between:
v = reduce(map(filter(x, f), m), r)
and
v = x | map(_, m) | filter(_, f) | reduce(_, r)
were only the syntactic sugar, not any performance difference (i.e. the translation between the two is trivial)
Right. Unless you can also optimize the range generation functions to work as fast as or as efficiently as the magical operator| implementation. ;-)
True, but only if you knew the type beforehand and invoke the rewrite explicitly.
There is no need to invoke the rule explicitly. It is magic :)
Oh, right!
Really, I think that we are talking basically about the same thing. We are just discussing about details. In my view, chains of lazy ranges *are* a form of expression templates, and I cannot see what a smart operator| would add to that: lazy ranges encode the same information that you would encode in an expression template, with the same (or lack of) penality. Maybe you could show a proof of concept implementation...
I think using operator|(...) to just create the type and not actually create the range(s) (like the above) would be something worth looking into at least. I think that's the way a lazy range generator or lazy range type should work -- much like how everything else in the lazy world works. About the proof of concept, I'll see what I can do. :-) Thanks, and I hope this helps! -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
On Thu, Jul 10, 2008 at 12:25 PM, Dean Michael Berris <mikhailberis@gmail.com> wrote:
On Mon, Jul 7, 2008 at 10:48 PM, Giovanni Piero Deretta <gpderetta@gmail.com> wrote:
On Mon, Jul 7, 2008 at 12:19 PM, Dean Michael Berris <mikhailberis@gmail.com> wrote: <snip>
Removing the need to actually create temporary objects.
You would be substituting one light weight temporary (a lazy range) for another one, the expression template node.
Right, but the expression template "node" will just contain references -- and actual iterators will only be built once with the exact type, instead of encapsulating and creating multiple range objects passed from function to function.
You can do the same thing with ranges (thus my claim that we are actually talking about the same thing:) ). If you look at the file I have attached a couple of posts ago, you'll find, for example, that the definition of transform_range (I call it map range) is: template<class R, class M> struct map_range { map_range(R const&r_, M m_) : r(r_), m(m_) {} typedef typename detail::make_transform_iterator<R, M>::type iterator; typedef iterator const_iterator; iterator begin() const { return iterator(detail::begin(r), m); } iterator end() const { return iterator(detail::end(r), m); } R r; M m; }; I.e. the iterators are built on demand. You can do a bit better by inheriting from compressed_pair<R, M>, just in case any of R or M is an empty class (uncommon for the former, but likely for the latter), but I didn't bother for this example. Note that both R and M are by value (otherwise some code might break horribly). If you want references, specify the template parameters as needed. (BTW, the above code is broken as a const_iterator is not necessary the same as iterator, also the const variant of begin and end should return a const_iterator: I converted the code from actually holding an iterator pair to holding the original range, and forgot to fix this mistake) Building iterators on demand has the advantage that complex range objects might be smaller, but means that you have to explicitly wrap the initial range object in a boost::sub_range to prevent it from being copied around, so it is not necessarily a win. Holding the original range by reference means that you risk dangling references. <snip>
May be I'm misunderstanding you; could you provide an example and show exactly where is the difference between expression templates and lazy ranges (IMHO they are exactly the same thing)?
I tried above, but generally I think there's a difference between creating the type first at compile time then reducing the number of steps actually done at runtime (constructing and destroying objects or even a bunch of bytes).
Have you ever measured this overhead? Did it matter? I user lazy ranges extensively and never measured any significant overhead the expression construction itself. <snip>
So in essence, you'd want expression templates because if:
reduce(map(filter(source, f), m), r)
Is less efficient than:
source | filter(f) | map(m) | reduce(r)
I claim that if a range library makes the former less efficient than the latter, the library is broken. In fact It would be take very hard work to make the first syntax inefficient :)
Then that's a good reason to either: make the range generation functions as efficient as the expression template approach, or remove the range generation functions in favor of the more concise (and efficient) pipe syntax.
The former syntax is more natural and can work with binders (if you make map, filter and friends function objects), so I'm against dropping it. Also neither of these two syntaxes: auto r = map(filter(range, f), m); auto r = range|map(_,m)|filter(_,f) should produce dangling references (use result_of for a c++03 variant). Which means that ranges should close by value over their arguments or use iterators internally, unless you want to add another step to do a deep copy. -- gpd
on Thu Jul 10 2008, "Giovanni Piero Deretta" <gpderetta-AT-gmail.com> wrote:
Have you ever measured this overhead? Did it matter? I user lazy ranges extensively and never measured any significant overhead the expression construction itself.
FWIW, I'm sorry that I planted this seed without really thinking about it. I'd be willing to believe what you say is true and there's not much if anything to be gained by attempting to make lazy ranges even lazier. As I said, it was just idle speculation on my part. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams skrev:
on Thu Jul 10 2008, "Giovanni Piero Deretta" <gpderetta-AT-gmail.com> wrote:
Have you ever measured this overhead? Did it matter? I user lazy ranges extensively and never measured any significant overhead the expression construction itself.
FWIW, I'm sorry that I planted this seed without really thinking about it. I'd be willing to believe what you say is true and there's not much if anything to be gained by attempting to make lazy ranges even lazier. As I said, it was just idle speculation on my part.
That said, it would be interesting enough to see if the generated assembler differs in any non-trivial way. -Thorsten
Giovanni Piero Deretta wrote: > On Mon, Jun 30, 2008 at 3:37 PM, David Abrahams <dave@boostpro.com> wrote: >>> On Thu, Jun 26, 2008 at 9:08 PM, David Abrahams <dave@boostpro.com> wrote: >>>> Here are the parts and topics that I see as being in the category: >>>> >>>> result_of >> * Too hard to define result<> templates for polymorphic function objects. > > I use a simple adaptor that makes it possible to use an mpl > expression to compute result types. Not publicly available in Boost >> * A whole spate of questions that came up in >> http://article.gmane.org/gmane.comp.lib.boost.devel/173370 were not >> answered with a page of documentation, utility library, or other >> facility. I still don't know what the upshot was > > The solution used by Eric is basically the same used by Egg::poly: > > http://tinyurl.com/64k4k6 > > I haven't really tried it so far, but it seems pretty good and simple. IIRC egg didn't pass its review. >>>> BOOST_TYPEOF >> * Doesn't coordinate with result_of > > Should it? How? We have this facility called result_of that is used to compute the result type of various "function" calls; it requires manual intervention for each function. We have this facility called BOOST_TYPEOF that works (or can be made to work) automatically on many important platforms and otherwise requires some manual intervention for each concrete user-defined type or template. It seems obvious to me that the default implementation of result_of could use BOOST_TYPEOF, and it would "just work" in any environment where BOOST_TYPEOF has native support or where types/templates are being registered explicitly. >>>> range_ex >> * Not in Boost. Should have pipe syntax. Maybe should use expression >> templates and be lazier. > > Does it really need expression templates? No, that probably doesn't rise to the same level of importance as the other things I've been mentioning. It was a bit of an idle thought. > That is, given the range > builders make_*_view [1], it should return a *_view (for example a > filtered_view). Combining multiple views, you get for example: > > filtered_view<mapped_view<taken_view<BaseRange> , Mapper>, Filter> > > > I.e. a series of range views are already an expression templates. > Now, if you make some (reasonable, and hopefully documented) > assumptions about Filter and Mapper, you should be always able to > convert sequences of nested views that contains only views known to > range_ex in a normal form: for example: > > taken_view<mapped_view<filtered_view<BaseRange, compose<Mapper, > Filter> >, Mapper> > > > You can of course collapse multiple mapped, filtered and taken range > in a single instance of each range. [2] Right, I was thinking of those kinds of optimizations, e.g. strided(strided(it,2),10) == strided(it,20) > A top level boost::for_each could be able to unwind the normalized > expression and apply a simple iteration over the Base range instead of > relying on the compiler of eliminating the abstraction overhead of > four different iterators. Not sure what you have in mind, but adding smarts to for_each doesn't seem very smart to me :-). You'd have to do the same for every other algorithm. > I think that the basic ranges that should (at least) be supported are: > > - mapped_range > - filtered_range > - taken_range (i.e. just the first n elements of a range) > - taken_while_range (i.e. the first elements such as a predicate is true) > - zipped_range > - unfold_range (with which you can pretty much express any other > range, see http://tinyurl.com/5mus25) > > An eventual 'drop' (take all the elements after the first n) and > 'drop_while' (drop the first elements such as a predicate is true) > probably need strict evaluation anyway and aren't worth supporting > explicitly. > > Finally, what do you mean that range_ex views should be lazier? Lazier > than what? Lazier than they are. If you avoid building the iterators until you have the whole chained view expression, you can do some optimizations like I mentioned above. Again, though, this probably isn't really important. > [1] Btw, I greatly dislike the verbose make_filtered_view, > make_transformed_view names. I use these builders a lot, and simply > call them 'map' and 'filter'. Agree. Who's using the long names? > [2] I think that this is quite similar to the deforestation technique > used by functional languages compilers. (see the wikipedia page and > the linked articles). In FP it is used mostly to eliminate > intermediate (lazily evaluated) lists, but the same technique can be > applied in c++. In fact the rewrite rules used map perfectly to c++ > templates. Maybe we're talking about the same thing. >>>> our own implementations of std::containers (in interprocess, IIRC) >> * Not well-known. >> * Not in a general place (bound to interprocess?) >> * Ought to support move semantics > > Yes please. And segmented iterators support for deque, And the hash containers > and the > guarantee that every deque subrange is contiguous in memory (it is > great for network programming). And inplace allocation, and... > > <snip> >>>> Boost.Iterator >> * Needs an update (e.g. "new iterator concepts" have to go -- C++0x >> resolves that differently by simply allowing proxies everywhere) > > What about older compilers? What about them? > Also it would be interesting to have a > mapping between the C++0x solution and the "new iterator concepts". I prefer not to haul that legacy around -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Tue, Jul 1, 2008 at 5:44 PM, David Abrahams <dave@boostpro.com> wrote: > Giovanni Piero Deretta wrote: >> On Mon, Jun 30, 2008 at 3:37 PM, David Abrahams <dave@boostpro.com> wrote: >>>> On Thu, Jun 26, 2008 at 9:08 PM, David Abrahams <dave@boostpro.com> wrote: >>>>> Here are the parts and topics that I see as being in the category: >>>>> >>>>> result_of >>> * Too hard to define result<> templates for polymorphic function objects. >> >> I use a simple adaptor that makes it possible to use an mpl >> expression to compute result types. > > Not publicly available in Boost I'm doing a clean up right now and I'll put it in the vault, just in case anybody wants to take a look. > >>> * A whole spate of questions that came up in >>> http://article.gmane.org/gmane.comp.lib.boost.devel/173370 were not >>> answered with a page of documentation, utility library, or other >>> facility. I still don't know what the upshot was >> >> The solution used by Eric is basically the same used by Egg::poly: >> >> http://tinyurl.com/64k4k6 >> >> I haven't really tried it so far, but it seems pretty good and simple. > > IIRC egg didn't pass its review. No surprise, I did write the only review ;). Still some pieces could find their way into boost. > >>>>> BOOST_TYPEOF >>> * Doesn't coordinate with result_of >> >> Should it? How? > > We have this facility called result_of that is used to compute the > result type of various "function" calls; it requires manual intervention > for each function. We have this facility called BOOST_TYPEOF that > works (or can be made to work) automatically on many important platforms > and otherwise requires some manual intervention for each concrete > user-defined type or template. It seems obvious to me that the default > implementation of result_of could use BOOST_TYPEOF, and it would "just > work" in any environment where BOOST_TYPEOF has native support or where > types/templates are being registered explicitly. I still do not see how can you 'fix' BOOST_TYPEOF to deduce const references correctly. Without that, you can't implement result_of with boost typeof. > >> That is, given the range >> builders make_*_view [1], it should return a *_view (for example a >> filtered_view). Combining multiple views, you get for example: >> >> filtered_view<mapped_view<taken_view<BaseRange> , Mapper>, Filter> > >> >> I.e. a series of range views are already an expression templates. >> Now, if you make some (reasonable, and hopefully documented) >> assumptions about Filter and Mapper, you should be always able to >> convert sequences of nested views that contains only views known to >> range_ex in a normal form: for example: >> >> taken_view<mapped_view<filtered_view<BaseRange, compose<Mapper, >> Filter> >, Mapper> > >> >> You can of course collapse multiple mapped, filtered and taken range >> in a single instance of each range. [2] > > Right, I was thinking of those kinds of optimizations, e.g. > > strided(strided(it,2),10) == strided(it,20) > >> A top level boost::for_each could be able to unwind the normalized >> expression and apply a simple iteration over the Base range instead of >> relying on the compiler of eliminating the abstraction overhead of >> four different iterators. > > Not sure what you have in mind, but adding smarts to for_each doesn't > seem very smart to me :-). You'd have to do the same for every other > algorithm. you can use rewrite rules to simplify ranges of ranges, but in the end, you want to do a smarter traversal to eliminate all abstraction overhead (exactly for the same reason you would use a segmented iterator-aware for_each). I shouldn't have called it for_each, but an optimizing range library should provide an internal iteration interface that takes care of eliminating as much abstraction as possible. It would in practice be a boost::for_each. Any algorithm that can be implemented on top of it (and I guess that there are many, consider that you can use lazy zip, map and filter) would benefit from it. > >> I think that the basic ranges that should (at least) be supported are: >> >> - mapped_range >> - filtered_range >> - taken_range (i.e. just the first n elements of a range) >> - taken_while_range (i.e. the first elements such as a predicate is true) >> - zipped_range >> - unfold_range (with which you can pretty much express any other >> range, see http://tinyurl.com/5mus25) >> >> An eventual 'drop' (take all the elements after the first n) and >> 'drop_while' (drop the first elements such as a predicate is true) >> probably need strict evaluation anyway and aren't worth supporting >> explicitly. >> >> Finally, what do you mean that range_ex views should be lazier? Lazier >> than what? > > Lazier than they are. If you avoid building the iterators until you > have the whole chained view expression, you can do some optimizations > like I mentioned above. Again, though, this probably isn't really > important. Ah, ok. Would this just be a matter of deferring the iterator construction only in at an explicit call to begin()/end() over, for example, a strided_range, or you have in mind something more complex? (I think the domain you are thinking of is numeric computations, mine is text processing) > >> [1] Btw, I greatly dislike the verbose make_filtered_view, >> make_transformed_view names. I use these builders a lot, and simply >> call them 'map' and 'filter'. > > Agree. Who's using the long names? I think that those name have been proposed for range_ex, but I might be mistaken. > >> [2] I think that this is quite similar to the deforestation technique >> used by functional languages compilers. (see the wikipedia page and >> the linked articles). In FP it is used mostly to eliminate >> intermediate (lazily evaluated) lists, but the same technique can be >> applied in c++. In fact the rewrite rules used map perfectly to c++ >> templates. > > Maybe we're talking about the same thing. I guess so :) -- gpd
Giovanni Piero Deretta wrote:
I still do not see how can you 'fix' BOOST_TYPEOF to deduce const references correctly. Without that, you can't implement result_of with boost typeof.
As written elsewhere, treating a const rvalue as though it were a const lvalue should almost never cause a serious problem. It wouldn't be perfect, but it would work in a vast majority of cases.
you can use rewrite rules to simplify ranges of ranges, but in the end, you want to do a smarter traversal to eliminate all abstraction overhead (exactly for the same reason you would use a segmented iterator-aware for_each).
Ah.
I shouldn't have called it for_each, but an optimizing range library should provide an internal iteration interface that takes care of eliminating as much abstraction as possible. It would in practice be a boost::for_each. Any algorithm that can be implemented on top of it (and I guess that there are many, consider that you can use lazy zip, map and filter) would benefit from it.
OK.
I think that the basic ranges that should (at least) be supported are:
- mapped_range - filtered_range - taken_range (i.e. just the first n elements of a range) - taken_while_range (i.e. the first elements such as a predicate is true) - zipped_range - unfold_range (with which you can pretty much express any other range, see http://tinyurl.com/5mus25)
An eventual 'drop' (take all the elements after the first n) and 'drop_while' (drop the first elements such as a predicate is true) probably need strict evaluation anyway and aren't worth supporting explicitly.
Finally, what do you mean that range_ex views should be lazier? Lazier than what? Lazier than they are. If you avoid building the iterators until you have the whole chained view expression, you can do some optimizations like I mentioned above. Again, though, this probably isn't really important.
Ah, ok. Would this just be a matter of deferring the iterator construction only in at an explicit call to begin()/end() over, for example, a strided_range, or you have in mind something more complex? (I think the domain you are thinking of is numeric computations, mine is text processing)
Probably something more complex, but I really don't want to drag this point out :-) -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Tue, Jul 1, 2008 at 8:32 PM, David Abrahams <dave@boostpro.com> wrote:
Giovanni Piero Deretta wrote:
I still do not see how can you 'fix' BOOST_TYPEOF to deduce const references correctly. Without that, you can't implement result_of with boost typeof.
As written elsewhere, treating a const rvalue as though it were a const lvalue should almost never cause a serious problem. It wouldn't be perfect, but it would work in a vast majority of cases.
Good point, it is possible to always deduce T& references correctly and collapase const T& to plain T. I was trying to come up with something a program that might break while working correctly with TR1 result_of, but the only thing that came to my mind is something like this, which is not a strong reason not to breaking compatibility: let 'first' be a result_of compatible function object that behaves like fusion::at_c<0>() typedef boost::tuple<int> tuple_t; tuple_t x(0); boost::result_of<filrst(tuple_t const&)>::type r = first(x); assert(&r == &boost::get<0>(x)); The assert will fail if result_of uses BOOST_TYPEOF. Just be safe, a result_of that uses typeof might have a different name. -- gpd
David Abrahams wrote:
Here are the parts and topics that I see as being in the category:
result_of
* Too hard to define result<> templates for polymorphic function objects.
Agreed. I've written my own Egg-like function object adaptor to do the "right thing" with lvalues and rvalues. It lives here: http://svn.boost.org/trac/boost/browser/branches/proto/v4/boost/proto/detail... It is a polymorphic function object wrapper around a template that generates monomorphic function objects. (So it's analagous to mpl::quoteN, which turns a metafunction into a metafunction class.) It forwards reference-wrapped arguments to the monomorphic implementation as lvalues, and all others as rvalues. TODO: replace all Proto's function objects with this wrapper to enforce consistent behavior. All function objects across Boost that need to make rvalue/lvalue distinctions would need to be examined for consistency in this regard. Joel, I'm looking in Fusion's general direction.
* A whole spate of questions that came up in http://article.gmane.org/gmane.comp.lib.boost.devel/173370 were not answered with a page of documentation, utility library, or other facility. I still don't know what the upshot was
I still don't know either, sadly, but I think a wrapper like the one above that makes it easy to author polymorphic function objects that consistently handle lvalues and rvalues is a good first step. <snip>
lambda/phoenix/bind
* Incompatible placeholders * Been waiting for the lambda/phoenix merger forever * Lambda has lots of limitations and gotchas that we can improve on now * Phoenix 2.0 not in Boost
I've already taken the first and (IMO) most important step here by porting both Lambda and Phoenix to Proto. My opinion on the unification of Lambda and Phoenix is that it shouldn't happen. There are too many subtle differences in semantics. Rather, Lambda should be retired and Phoenix should be promoted (after being reviewed, etc.). The placeholders should be Proto terminals shared across Phoenix, Bind, Xpressive, Spirit and any other library that needs numbered placeholders. This sort of unification was, after all, one of the major design goals of Proto in the first place. Boost.Lambda Proto port: http://svn.boost.org/trac/boost/browser/branches/proto/v4/libs/proto/example... Boost.Phoenix Proto port: http://svn.boost.org/trac/boost/browser/branches/proto/v4/boost/phoenix Still TODO: finish Proto v4 and get it in trunk. Improve the compile times of Phoenix/Proto and get it in the review queue. Then get Bind working with the Phoenix/Proto placeholders. <snip> -- Eric Niebler BoostPro Computing http://www.boostpro.com
Apologies for overquoting... On Jun 30, 2008, at 1:13 PM, Eric Niebler <eric@boost-consulting.com> wrote: > David Abrahams wrote: >>>> Here are the parts and topics that I see as being in the category: >>>> >>>> result_of >> * Too hard to define result<> templates for polymorphic function >> objects. > > Agreed. I've written my own Egg-like function object adaptor to do > the "right thing" with lvalues and rvalues. It lives here: > > http://svn.boost.org/trac/boost/browser/branches/proto/v4/boost/proto/detail/poly_function.hpp > > It is a polymorphic function object wrapper around a template that > generates monomorphic function objects. (So it's analagous to > mpl::quoteN, which turns a metafunction into a metafunction class.) > It forwards reference-wrapped arguments to the monomorphic > implementation as lvalues, and all others as rvalues. > > TODO: replace all Proto's function objects with this wrapper to > enforce consistent behavior. All function objects across Boost that > need to make rvalue/lvalue distinctions would need to be examined > for consistency in this regard. Joel, I'm looking in Fusion's > general direction. > > >> * A whole spate of questions that came up in >> http://article.gmane.org/gmane.comp.lib.boost.devel/173370 were not >> answered with a page of documentation, utility library, or other >> facility. I still don't know what the upshot was > > > I still don't know either, sadly, but I think a wrapper like the one > above that makes it easy to author polymorphic function objects that > consistently handle lvalues and rvalues is a good first step. Great, but one important thing is missing: it isn't in boost as far as users are concerned. > > <snip> >>>> lambda/phoenix/bind >> * Incompatible placeholders >> * Been waiting for the lambda/phoenix merger forever >> * Lambda has lots of limitations and gotchas that we can improve on >> now >> * Phoenix 2.0 not in Boost > > I've already taken the first and (IMO) most important step here by > porting both Lambda and Phoenix to Proto. Kewl! > My opinion on the unification of Lambda and Phoenix is that it > shouldn't happen. There are too many subtle differences in semantics. I guess you're in a position to know > Rather, Lambda should be retired and Phoenix should be promoted > (after being reviewed, etc.). The placeholders should be Proto > terminals shared across Phoenix, Bind, Xpressive, Spirit and any > other library that needs numbered placeholders. As long as they're objects and not types ;-) > This sort of unification was, after all, one of the major design > goals of Proto in the first place. > > Boost.Lambda Proto port: > http://svn.boost.org/trac/boost/browser/branches/proto/v4/libs/proto/example/lambda.hpp > > Boost.Phoenix Proto port: > http://svn.boost.org/trac/boost/browser/branches/proto/v4/boost/ > phoenix > > Still TODO: finish Proto v4 and get it in trunk. Improve the compile > times of Phoenix/Proto and get it in the review queue. Then get Bind > working with the Phoenix/Proto placeholders. > On old compilers, too for bind... Should be fun. So does this mean you're taking on the functional programming part of the amorphous issue I raised?
Eric Niebler wrote:
David Abrahams wrote:
Here are the parts and topics that I see as being in the category:
result_of
* Too hard to define result<> templates for polymorphic function objects.
Agreed. I've written my own Egg-like function object adaptor to do the "right thing" with lvalues and rvalues. It lives here:
http://svn.boost.org/trac/boost/browser/branches/proto/v4/boost/proto/detail...
It is a polymorphic function object wrapper around a template that generates monomorphic function objects. (So it's analagous to mpl::quoteN, which turns a metafunction into a metafunction class.)
The analogy doesn't seem quite right, but OK
It forwards reference-wrapped arguments to the monomorphic implementation as lvalues, and all others as rvalues.
Very cool. I don't suppose it's possible to forward lvalues and const rvalues as lvalues, and mutable rvalues as rvalues? I haven't thought of a way, but that would be as close to "perfect" as we might expect to get in C++0x and it would be close enough for me :-)
TODO: replace all Proto's function objects with this wrapper to enforce consistent behavior. All function objects across Boost that need to make rvalue/lvalue distinctions would need to be examined for consistency in this regard. Joel, I'm looking in Fusion's general direction.
Yep. It's a big coordination job.
* A whole spate of questions that came up in http://article.gmane.org/gmane.comp.lib.boost.devel/173370 were not answered with a page of documentation, utility library, or other facility. I still don't know what the upshot was
I still don't know either, sadly, but I think a wrapper like the one above that makes it easy to author polymorphic function objects that consistently handle lvalues and rvalues is a good first step.
Great! Problem: it's not publicly available in Boost (yet)/
<snip>
lambda/phoenix/bind
* Incompatible placeholders * Been waiting for the lambda/phoenix merger forever * Lambda has lots of limitations and gotchas that we can improve on now * Phoenix 2.0 not in Boost
I've already taken the first and (IMO) most important step here by porting both Lambda and Phoenix to Proto. My opinion on the unification of Lambda and Phoenix is that it shouldn't happen. There are too many subtle differences in semantics. Rather, Lambda should be retired and Phoenix should be promoted (after being reviewed, etc.).
That's OK too, although Joel still seems to be intent on an eventual merger.
The placeholders should be Proto terminals shared across Phoenix, Bind, Xpressive, Spirit and any other library that needs numbered placeholders.
'xcept MPL
David Abrahams wrote: > Eric Niebler wrote: >> Agreed. I've written my own Egg-like function object adaptor to do >> the "right thing" with lvalues and rvalues. It lives here: >> >> http://svn.boost.org/trac/boost/browser/branches/proto/v4/boost/proto/detail/poly_function.hpp >> <snip> >> It forwards reference-wrapped arguments to the monomorphic >> implementation as lvalues, and all others as rvalues. > > Very cool. I don't suppose it's possible to forward lvalues and > const rvalues as lvalues, and mutable rvalues as rvalues? I haven't > thought of a way, but that would be as close to "perfect" as we might > expect to get in C++0x and it would be close enough for me :-) We would need 2^N overloads, and it would be unsound because it would incorrectly forward const rvalues as lvalues. It wouldn't be useful for my purposes. What I took from the discussion about result_of and C++03 was that, for a function object that cares about lvalue/rvalue-ness to work consistently across C++03 and C++0x, assuming rvalue-ness and opting in for lvalue-ness with reference_wrapper<> is really the only option. But if you have an insight, I'd be happy to reopen that discussion. It was rather unsatisfying last time. <snip> >> I think a wrapper like the one above that makes it easy to author >> polymorphic function objects that consistently handle lvalues and >> rvalues is a good first step. > > Great! Problem: it's not publicly available in Boost (yet)/ A simple, light-weight utility such as this would be very helpful, and probably not too hard to get through on a fast-track review, if we could just agree on semantics. >> <snip> >>>>> lambda/phoenix/bind >>> * Incompatible placeholders * Been waiting for the lambda/phoenix >>> merger forever * Lambda has lots of limitations and gotchas that >>> we can improve on now * Phoenix 2.0 not in Boost >> I've already taken the first and (IMO) most important step here by >> porting both Lambda and Phoenix to Proto. My opinion on the >> unification of Lambda and Phoenix is that it shouldn't happen. >> There are too many subtle differences in semantics. Rather, Lambda >> should be retired and Phoenix should be promoted (after being >> reviewed, etc.). > > That's OK too, although Joel still seems to be intent on an eventual > merger. If we give up on backwards compatibility, a merger is still possible. I think Joel's suggestion is workable. I like the approach he and Hartmut are using in the Spirit 2 transition. >> The placeholders should be Proto terminals shared across Phoenix, >> Bind, Xpressive, Spirit and any other library that needs numbered >> placeholders. > > 'xcept MPL 'course. -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
Agreed. I've written my own Egg-like function object adaptor to do the "right thing" with lvalues and rvalues. It lives here:
http://svn.boost.org/trac/boost/browser/branches/proto/v4/boost/proto/detail...
<snip>
It forwards reference-wrapped arguments to the monomorphic implementation as lvalues, and all others as rvalues.
Very cool. I don't suppose it's possible to forward lvalues and const rvalues as lvalues, and mutable rvalues as rvalues? I haven't thought of a way, but that would be as close to "perfect" as we might expect to get in C++0x and it would be close enough for me :-)
We would need 2^N overloads,
That's OK for small N
and it would be unsound because it would incorrectly forward const rvalues as lvalues.
Can you show me an example of a "sound" function that treats a const rvalue differently from an lvalue, please?
It wouldn't be useful for my purposes. What I took from the discussion about result_of and C++03 was that, for a function object that cares about lvalue/rvalue-ness to work consistently across C++03 and C++0x, assuming rvalue-ness and opting in for lvalue-ness with reference_wrapper<> is really the only option. But if you have an insight, I'd be happy to reopen that discussion. It was rather unsatisfying last time.
I just think that since you don't have permission to mutate a const rvalue anyway, forwarding it as a const reference doesn't really hurt anyone.
<snip>
I think a wrapper like the one above that makes it easy to author polymorphic function objects that consistently handle lvalues and rvalues is a good first step.
Great! Problem: it's not publicly available in Boost (yet)/
A simple, light-weight utility such as this would be very helpful, and probably not too hard to get through on a fast-track review, if we could just agree on semantics.
Or we could stick it in utility without review, since I'm a maintainer of that "library" (gotta love these loopholes!)
If we give up on backwards compatibility, a merger is still possible. I think Joel's suggestion is workable. I like the approach he and Hartmut are using in the Spirit 2 transition.
Sounds like he changed his mind after all. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
Eric Niebler wrote:
David Abrahams wrote:
I don't suppose it's possible to forward lvalues and const rvalues as lvalues, and mutable rvalues as rvalues? I haven't thought of a way, but that would be as close to "perfect" as we might expect to get in C++0x and it would be close enough for me :-) We would need 2^N overloads,
That's OK for small N
boost::function and phoenix have argument limits of 10 by default. 2^10 is a big number. Can't wait for variadics and rvalue refs.
and it would be unsound because it would incorrectly forward const rvalues as lvalues.
Can you show me an example of a "sound" function that treats a const rvalue differently from an lvalue, please?
fusion::make_vector(), for instance. It returns a fusion::vector<> where the members can be held by value or by reference. Getting it wrong can lead to dangling references. Proto has similar function objects that need to be careful about the lifetime of temporaries.
It wouldn't be useful for my purposes. What I took from the discussion about result_of and C++03 was that, for a function object that cares about lvalue/rvalue-ness to work consistently across C++03 and C++0x, assuming rvalue-ness and opting in for lvalue-ness with reference_wrapper<> is really the only option. But if you have an insight, I'd be happy to reopen that discussion. It was rather unsatisfying last time.
I just think that since you don't have permission to mutate a const rvalue anyway, forwarding it as a const reference doesn't really hurt anyone.
When I've gotten it wrong, it hurts.
I think a wrapper like the one above that makes it easy to author polymorphic function objects that consistently handle lvalues and rvalues is a good first step. Great! Problem: it's not publicly available in Boost (yet)/ A simple, light-weight utility such as this would be very helpful, and
<snip> probably not too hard to get through on a fast-track review, if we could just agree on semantics.
Or we could stick it in utility without review, since I'm a maintainer of that "library" (gotta love these loopholes!)
Sneaky.
If we give up on backwards compatibility, a merger is still possible. I think Joel's suggestion is workable. I like the approach he and Hartmut are using in the Spirit 2 transition.
Sounds like he changed his mind after all.
He did? -- Eric Niebler BoostPro Computing http://www.boostpro.com
On Tue, Jul 1, 2008 at 6:57 PM, Eric Niebler <eric@boost-consulting.com> wrote:
David Abrahams wrote:
Eric Niebler wrote:
David Abrahams wrote:
I don't suppose it's possible to forward lvalues and const rvalues as lvalues, and mutable rvalues as rvalues? I haven't thought of a way, but that would be as close to "perfect" as we might expect to get in C++0x and it would be close enough for me :-)
We would need 2^N overloads,
That's OK for small N
boost::function and phoenix have argument limits of 10 by default. 2^10 is a big number. Can't wait for variadics and rvalue refs.
I use a wrapper that does perfect forwarding up to 5 arguments (and all ref/all const ref for the remaining 10). I never needed more than that and I do not think it has an important contribution to compile time (that is, heavy lambda usage will kill compile time way more than that).
and it would be unsound because it would incorrectly forward const rvalues as lvalues.
Can you show me an example of a "sound" function that treats a const rvalue differently from an lvalue, please?
fusion::make_vector(), for instance. It returns a fusion::vector<> where the members can be held by value or by reference. Getting it wrong can lead to dangling references. Proto has similar function objects that need to be careful about the lifetime of temporaries.
IMHO you can ever safely hold on a parameter by reference unless you somehow explicitly decorate that parameter (with reference_wrapper for example). I do not see how brute force forwarding or C++0x perfect forwarding change anything. I should probably read again the thread referenced by David Abrahams -- gpd
Giovanni Piero Deretta wrote:
On Tue, Jul 1, 2008 at 6:57 PM, Eric Niebler <eric@boost-consulting.com> wrote:
David Abrahams wrote:
Eric Niebler wrote:
David Abrahams wrote:
I don't suppose it's possible to forward lvalues and const rvalues as lvalues, and mutable rvalues as rvalues? I haven't thought of a way, but that would be as close to "perfect" as we might expect to get in C++0x and it would be close enough for me :-) We would need 2^N overloads, That's OK for small N boost::function and phoenix have argument limits of 10 by default. 2^10 is a big number. Can't wait for variadics and rvalue refs.
I use a wrapper that does perfect forwarding up to 5 arguments (and all ref/all const ref for the remaining 10). I never needed more than that and I do not think it has an important contribution to compile time (that is, heavy lambda usage will kill compile time way more than that).
My Phoenix/Proto port lets you do something similar. The first N arguments are handled "perfectly", and the rest are taken by const-ref (where N is configurable). I had to turn it off (N=0 by default) because even at N=3 it was slaughtering compile times. If you have to choose between 2^N overloads of operator(), you have to calculate the return types of all the overloads -- even the ones that won't get selected -- and that is hella expensive. I can't see a way around it, except to somehow normalize the const-ref-ness of each arg before doing the return type calculation, so that template memoization kicks in, but that would cause the return type calculation to be wrong.
and it would be unsound because it would incorrectly forward const rvalues as lvalues. Can you show me an example of a "sound" function that treats a const rvalue differently from an lvalue, please? fusion::make_vector(), for instance. It returns a fusion::vector<> where the members can be held by value or by reference. Getting it wrong can lead to dangling references. Proto has similar function objects that need to be careful about the lifetime of temporaries.
IMHO you can ever safely hold on a parameter by reference unless you somehow explicitly decorate that parameter (with reference_wrapper for example). I do not see how brute force forwarding or C++0x perfect forwarding change anything. I should probably read again the thread referenced by David Abrahams
IIRC, that was ultimately the conclusion reached in that thread. -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote: -- snip 2^10 overloads -- Yeah, that's probably too much, but I'm not sure we have to support forwarding for that many arguments. The only thing rvalue forwarding buys you (and it can be a big win) is the ability for the wrapped thing to make certain optimizations. It's a good idea when you can do it. If you can't do it, you can fall back to lvalues and in C++03 you've still done a credible job.
David Abrahams wrote:
Can you show me an example of a "sound" function that treats a const rvalue differently from an lvalue, please?
fusion::make_vector(), for instance. It returns a fusion::vector<> where the members can be held by value or by reference. Getting it wrong can lead to dangling references. Proto has similar function objects that need to be careful about the lifetime of temporaries.
I'm a little confused. Proper rvalue detection for *wrapper functions* or not, we have no way of writing make_vector so that it stores all rvalues and references lvalues. So in C++03, that function needs some hints about when to store values. I'm not convinced it doesn't need those hints in C++0x. After all, any time you build a vector of references from lvalues, those references (or copies thereof) can be made to dangle. That doesn't mean a wrapper function should not forward non-const rvalues to its wrapee.
It wouldn't be useful for my purposes. What I took from the discussion about result_of and C++03 was that, for a function object that cares about lvalue/rvalue-ness to work consistently across C++03 and C++0x, assuming rvalue-ness and opting in for lvalue-ness with reference_wrapper<> is really the only option. But if you have an insight, I'd be happy to reopen that discussion. It was rather unsatisfying last time.
I just think that since you don't have permission to mutate a const rvalue anyway, forwarding it as a const reference doesn't really hurt anyone.
When I've gotten it wrong, it hurts.
Maybe you're making assumptions that you really can't afford to make (that storing a reference to an lvalue, even when properly detected, is a good idea).
If we give up on backwards compatibility, a merger is still possible. I think Joel's suggestion is workable. I like the approach he and Hartmut are using in the Spirit 2 transition.
Sounds like he changed his mind after all.
He did?
I saw a message to Jaakko go by that I interpreted that way. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
Eric Niebler wrote:
-- snip 2^10 overloads --
Yeah, that's probably too much, but I'm not sure we have to support forwarding for that many arguments. The only thing rvalue forwarding buys you (and it can be a big win) is the ability for the wrapped thing to make certain optimizations. It's a good idea when you can do it. If you can't do it, you can fall back to lvalues and in C++03 you've still done a credible job.
Even 2^3 overloads has a *very* noticeable effect on compile times.
Can you show me an example of a "sound" function that treats a const rvalue differently from an lvalue, please? fusion::make_vector(), for instance. It returns a fusion::vector<> where
David Abrahams wrote: the members can be held by value or by reference. Getting it wrong can lead to dangling references. Proto has similar function objects that need to be careful about the lifetime of temporaries.
I'm a little confused.
Proper rvalue detection for *wrapper functions* or not, we have no way of writing make_vector so that it stores all rvalues and references lvalues. So in C++03, that function needs some hints about when to store values.
Of course.
I'm not convinced it doesn't need those hints in C++0x.
Right.
After all, any time you build a vector of references from lvalues, those references (or copies thereof) can be made to dangle.
That doesn't mean a wrapper function should not forward non-const rvalues to its wrapee.
The wrapper I wrote (poly_function) passes arguments to the wrapped function and also passes along information about whether the user specified that argument as an lvalue (reference-wrapped) or not. That's all. Whether the wrapped function *actually* accepts its (rvalue) arguments as "T" or "T const &" is a different issue. poly_function doesn't care. When you use poly_function, you basically write a template that generates mono-morphic function objects like this: template< class Arg0, class Arg1 > struct my_function { typedef ... result_type; result_type operator()(poly_arg<Arg0> a0, poly_arg<Arg1> a1) const { ... } }; If ArgN is a reference, then the user has reference-wrapped the argument. If it isn't a reference, it was passed unwrapped. What poly_function buys you is: 1) It implements the result_of protocol for you, correctly handling const, refs, and reference wrappers. 2) It unwraps any reference wrapped args before forwarding them. 3) It makes writing a polymorphic function object as simple as writing a monomorphic one.
It wouldn't be useful for my purposes. What I took from the discussion about result_of and C++03 was that, for a function object that cares about lvalue/rvalue-ness to work consistently across C++03 and C++0x, assuming rvalue-ness and opting in for lvalue-ness with reference_wrapper<> is really the only option. But if you have an insight, I'd be happy to reopen that discussion. It was rather unsatisfying last time. I just think that since you don't have permission to mutate a const rvalue anyway, forwarding it as a const reference doesn't really hurt anyone. When I've gotten it wrong, it hurts.
Maybe you're making assumptions that you really can't afford to make (that storing a reference to an lvalue, even when properly detected, is a good idea).
Please, I'm not trying to detect lvalue/rvalue-ness! I'm letting users opt-in with reference_wrapper. -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
-- snip 2^10 overloads --
Yeah, that's probably too much, but I'm not sure we have to support forwarding for that many arguments. The only thing rvalue forwarding buys you (and it can be a big win) is the ability for the wrapped thing to make certain optimizations. It's a good idea when you can do it. If you can't do it, you can fall back to lvalues and in C++03 you've still done a credible job.
Even 2^3 overloads has a *very* noticeable effect on compile times.
8 overloads hurts compile times?
After all, any time you build a vector of references from lvalues, those references (or copies thereof) can be made to dangle.
That doesn't mean a wrapper function should not forward non-const rvalues to its wrapee.
The wrapper I wrote (poly_function) passes arguments to the wrapped function and also passes along information about whether the user specified that argument as an lvalue (reference-wrapped) or not.
Understood
That's all. Whether the wrapped function *actually* accepts its (rvalue) arguments as "T" or "T const &" is a different issue. poly_function doesn't care.
Understood again. I don't get what you're driving at, though.
When you use poly_function, you basically write a template that generates mono-morphic function objects like this:
template< class Arg0, class Arg1 > struct my_function { typedef ... result_type;
result_type operator()(poly_arg<Arg0> a0, poly_arg<Arg1> a1) const { ... } };
If ArgN is a reference, then the user has reference-wrapped the argument. If it isn't a reference, it was passed unwrapped.
Pretty much what I expected. Hm, what is poly_arg<T> ?
What poly_function buys you is:
1) It implements the result_of protocol for you, correctly handling const, refs, and reference wrappers.
2) It unwraps any reference wrapped args before forwarding them.
3) It makes writing a polymorphic function object as simple as writing a monomorphic one.
All good things, although I think I contest point 3. It may be better, but it can't be _as simple_ if you need poly_arg, template arguments, and a poly_function wrapper.
I just think that since you don't have permission to mutate a const rvalue anyway, forwarding it as a const reference doesn't really hurt anyone. When I've gotten it wrong, it hurts.
Maybe you're making assumptions that you really can't afford to make (that storing a reference to an lvalue, even when properly detected, is a good idea).
Please, I'm not trying to detect lvalue/rvalue-ness!
I'm sorry if it sounded like you were being attacked; I was just trying to find an explanation for the hurt you described in the context of my understanding of the meaning of "const rvalue." Maybe where one really gets hurt is in dealing with return values: if you treat const rvalues as const lvalues you *will* dangle, and if you treat const lvalues as const rvalues you will cause the wrong semantics in way too many common cases.
I'm letting users opt-in with reference_wrapper.
I understand that's the scope you chose. I was only trying to explore whether that facility can usefully be pushed a little further so that we don't lose move semantics inside of forwarded arguments. Well, I guess we don't, if we assume everything's an rvalue by default :-) -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
-- snip 2^10 overloads --
Yeah, that's probably too much, but I'm not sure we have to support forwarding for that many arguments. The only thing rvalue forwarding buys you (and it can be a big win) is the ability for the wrapped thing to make certain optimizations. It's a good idea when you can do it. If you can't do it, you can fall back to lvalues and in C++03 you've still done a credible job. Even 2^3 overloads has a *very* noticeable effect on compile times.
8 overloads hurts compile times?
Yes, because for each function invocation you have to calculate the return type 8x in 8 different ways. That can cause 8x the number of templates to be instantiated.
After all, any time you build a vector of references from lvalues, those references (or copies thereof) can be made to dangle.
That doesn't mean a wrapper function should not forward non-const rvalues to its wrapee. The wrapper I wrote (poly_function) passes arguments to the wrapped function and also passes along information about whether the user specified that argument as an lvalue (reference-wrapped) or not.
Understood
That's all. Whether the wrapped function *actually* accepts its (rvalue) arguments as "T" or "T const &" is a different issue. poly_function doesn't care.
Understood again. I don't get what you're driving at, though.
When you use poly_function, you basically write a template that generates mono-morphic function objects like this:
template< class Arg0, class Arg1 > struct my_function { typedef ... result_type;
result_type operator()(poly_arg<Arg0> a0, poly_arg<Arg1> a1) const { ... } };
If ArgN is a reference, then the user has reference-wrapped the argument. If it isn't a reference, it was passed unwrapped.
Pretty much what I expected. Hm, what is poly_arg<T> ?
It's not actually needed, but it's handy. Think of it as a replacement for typename add_reference<typename add_const<ArgN>::type>::type.
What poly_function buys you is:
1) It implements the result_of protocol for you, correctly handling const, refs, and reference wrappers.
2) It unwraps any reference wrapped args before forwarding them.
3) It makes writing a polymorphic function object as simple as writing a monomorphic one.
All good things, although I think I contest point 3. It may be better, but it can't be _as simple_ if you need poly_arg, template arguments, and a poly_function wrapper.
OK.
I just think that since you don't have permission to mutate a const rvalue anyway, forwarding it as a const reference doesn't really hurt anyone. When I've gotten it wrong, it hurts. Maybe you're making assumptions that you really can't afford to make (that storing a reference to an lvalue, even when properly detected, is a good idea). Please, I'm not trying to detect lvalue/rvalue-ness!
I'm sorry if it sounded like you were being attacked; I was just trying to find an explanation for the hurt you described in the context of my understanding of the meaning of "const rvalue." Maybe where one really gets hurt is in dealing with return values: if you treat const rvalues as const lvalues you *will* dangle,
Yes.
and if you treat const lvalues as const rvalues you will cause the wrong semantics in way too many common cases.
Yes. I was trying to illustrate those points with the fusion::make_vector() example.
I'm letting users opt-in with reference_wrapper.
I understand that's the scope you chose. I was only trying to explore whether that facility can usefully be pushed a little further so that we don't lose move semantics inside of forwarded arguments. Well, I guess we don't, if we assume everything's an rvalue by default :-)
I'm not sure if changes are needed to better support move semantics, or what they would be. Feedback is welcome. -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
-- snip 2^10 overloads --
Yeah, that's probably too much, but I'm not sure we have to support forwarding for that many arguments. The only thing rvalue forwarding buys you (and it can be a big win) is the ability for the wrapped thing to make certain optimizations. It's a good idea when you can do it. If you can't do it, you can fall back to lvalues and in C++03 you've still done a credible job. Even 2^3 overloads has a *very* noticeable effect on compile times.
8 overloads hurts compile times?
Yes, because for each function invocation you have to calculate the return type 8x in 8 different ways.
Hmm. Why? Seems to me that only one specialization of the wrapped class template needs to be instantiated per invocation.
I just think that since you don't have permission to mutate a const rvalue anyway, forwarding it as a const reference doesn't really hurt anyone. When I've gotten it wrong, it hurts. Maybe you're making assumptions that you really can't afford to make (that storing a reference to an lvalue, even when properly detected, is a good idea). Please, I'm not trying to detect lvalue/rvalue-ness!
I'm sorry if it sounded like you were being attacked; I was just trying to find an explanation for the hurt you described in the context of my understanding of the meaning of "const rvalue." Maybe where one really gets hurt is in dealing with return values: if you treat const rvalues as const lvalues you *will* dangle,
Yes.
and if you treat const lvalues as const rvalues you will cause the wrong semantics in way too many common cases.
Yes. I was trying to illustrate those points with the fusion::make_vector() example.
I think it's maybe better illustrated as template <class F, class T> result_of<F(T&)>::type forwarder(F const& g, T& x); If g(x) returns an rvalue, forwarder had better also return an rvalue or the result will dangle. if g(x) returns a const lvalue and forwarder does not, you can get silent odd behavior when the value returned from forwarder turns out to be a different object from the one g(x) references. Such functions can only work right if result_of returns the proper "reference-ness." However, there's no problem AFAICS with a forwarder like this, if it could be implemented. // Made-up syntax follows template <class F, class T> decltype(g(x)) forwarder(F const& g, T& x) if (x is an lvalue or const rvalue) { g( x ); } template <class F, class T> decltype(g(move(x))) forwarder(F const& g, T x) if (x is a non-const rvalue) { g( move(x) ); } for any sane implementation of F. That's my only point. It's not a big deal that we can't tell const rvalues from const lvalues where function arguments are concerned.
I'm letting users opt-in with reference_wrapper.
I understand that's the scope you chose. I was only trying to explore whether that facility can usefully be pushed a little further so that we don't lose move semantics inside of forwarded arguments. Well, I guess we don't, if we assume everything's an rvalue by default :-)
I'm not sure if changes are needed to better support move semantics, or what they would be. Feedback is welcome.
Basically, if your wrappers copy arguments by default, then it's always safe for wrapped functions to move from arguments unless they are reference_wrappers. We could even make move(x) where x is a reference_wrapper return an lvalue, so it'd be transparent. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
-- snip 2^10 overloads --
Yeah, that's probably too much, but I'm not sure we have to support forwarding for that many arguments. The only thing rvalue forwarding buys you (and it can be a big win) is the ability for the wrapped thing to make certain optimizations. It's a good idea when you can do it. If you can't do it, you can fall back to lvalues and in C++03 you've still done a credible job. Even 2^3 overloads has a *very* noticeable effect on compile times. 8 overloads hurts compile times? Yes, because for each function invocation you have to calculate the return type 8x in 8 different ways.
Hmm. Why? Seems to me that only one specialization of the wrapped class template needs to be instantiated per invocation.
Take the simple case with 1 arg: template<typename T> typename result_of<F(T &)>::type fun( T & t ); template<typename T> typename result_of<F(T const &)>::type fun( T const & t ); Now I invoke fun(x). The compiler has to do overload resolution. First it has to determine function signatures. In so doing, it computes the return type for both overloads. For 3 arguments there are 8 overloads and the return type needs to be calculated 8x in 8 different ways, even though only one will ever be used. <snip the rest, which I need to think more about> -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
-- snip 2^10 overloads --
Yeah, that's probably too much, but I'm not sure we have to support forwarding for that many arguments. The only thing rvalue forwarding buys you (and it can be a big win) is the ability for the wrapped thing to make certain optimizations. It's a good idea when you can do it. If you can't do it, you can fall back to lvalues and in C++03 you've still done a credible job. Even 2^3 overloads has a *very* noticeable effect on compile times. 8 overloads hurts compile times? Yes, because for each function invocation you have to calculate the return type 8x in 8 different ways.
Hmm. Why? Seems to me that only one specialization of the wrapped class template needs to be instantiated per invocation.
Take the simple case with 1 arg:
template<typename T> typename result_of<F(T &)>::type fun( T & t );
template<typename T> typename result_of<F(T const &)>::type fun( T const & t );
Now I invoke fun(x). The compiler has to do overload resolution. First it has to determine function signatures. In so doing, it computes the return type for both overloads. For 3 arguments there are 8 overloads and the return type needs to be calculated 8x in 8 different ways, even though only one will ever be used.
Oh, sure. Technically compilers should be able to defer computing the return type until after the rest of the signature wins overload resolution, but that's not the way they work today. I *think* we're getting that for 0x, though. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
I think it's maybe better illustrated as
template <class F, class T> result_of<F(T&)>::type forwarder(F const& g, T& x);
If g(x) returns an rvalue, forwarder had better also return an rvalue or the result will dangle. if g(x) returns a const lvalue and forwarder does not, you can get silent odd behavior when the value returned from forwarder turns out to be a different object from the one g(x) references. Such functions can only work right if result_of returns the proper "reference-ness."
Correct. And the question raised in the "result_of and C++03" thread was: is the "reference-ness" of of result_of<F(T&)>::type allowed to be influenced by the "reference-ness" of the arguments in the function type. That is, what do these mean, are they different and if so, how? result_of<F(T)>::type // 1 result_of<F(T&)>::type // 2 result_of<F(T const&)>::type // 3 There be dragons there in C++03 because there's no way in C++03 to overload a function on T, T& and T const&. The only workable solution I've found so far is that these all mean the same thing: the result of calling F with an rvalue of type T, unless T is a reference_wrapper<U>, in which case, it's the result of calling F with an lvalue of type U. If your reaction is "yuk" then I agree, but I haven't yet thought of something better. Now, you might say that we can detect case (2) in C++03 and make it mean "lvalue of type T", but for functions that take N arguments, that leads us to 2^N overloads, and 2^N return type calculations, which I'm trying to avoid.
However, there's no problem AFAICS with a forwarder like this, if it could be implemented.
// Made-up syntax follows
template <class F, class T> decltype(g(x)) forwarder(F const& g, T& x) if (x is an lvalue or const rvalue) { g( x ); }
template <class F, class T> decltype(g(move(x))) forwarder(F const& g, T x) if (x is a non-const rvalue) { g( move(x) ); }
for any sane implementation of F. That's my only point. It's not a big deal that we can't tell const rvalues from const lvalues where function arguments are concerned.
Hm, I *think* I get it.
I'm letting users opt-in with reference_wrapper. I understand that's the scope you chose. I was only trying to explore whether that facility can usefully be pushed a little further so that we don't lose move semantics inside of forwarded arguments. Well, I guess we don't, if we assume everything's an rvalue by default :-) I'm not sure if changes are needed to better support move semantics, or what they would be. Feedback is welcome.
Basically, if your wrappers copy arguments by default,
They don't. They could, but I don't think they should, do you?
then it's always safe for wrapped functions to move from arguments unless they are reference_wrappers. We could even make move(x) where x is a reference_wrapper return an lvalue, so it'd be transparent.
I'm not seeing a win here, at least not for C++03. Little help? -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
David Abrahams wrote:
I'm letting users opt-in with reference_wrapper. I understand that's the scope you chose. I was only trying to explore whether that facility can usefully be pushed a little further so that we don't lose move semantics inside of forwarded arguments. Well, I guess we don't, if we assume everything's an rvalue by default :-) I'm not sure if changes are needed to better support move semantics, or what they would be. Feedback is welcome.
Basically, if your wrappers copy arguments by default,
They don't. They could, but I don't think they should, do you?
I'm not entirely sure
then it's always safe for wrapped functions to move from arguments unless they are reference_wrappers. We could even make move(x) where x is a reference_wrapper return an lvalue, so it'd be transparent.
I'm not seeing a win here, at least not for C++03. Little help?
One of the motivations for perfect forwarding in C++0x is that a forwarding function that simply turns everything into lvalues when it presents the arguments to the forwarded-to function kills most opportunities for move semantics (since the main opportunities arise with non-const rvalues). If we start supporting move semantics in Boost, it would be nice if our forwarding functions could avoid turning non-const rvalueness into some kind of lvalueness. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
Eric Niebler wrote:
If we give up on backwards compatibility, a merger is still possible. I think Joel's suggestion is workable. I like the approach he and Hartmut are using in the Spirit 2 transition. Sounds like he changed his mind after all. He did?
I saw a message to Jaakko go by that I interpreted that way.
If "he" is Joel, then I'd say yes. To be specific: 1) Ask for a phoenix review 2) Granted phoenix is accepted, it becomes a full fledged boost library 3) Merge phoenix and lambda codebase. - lambda will be lambda1 - phoenix will be lambda2. - Provide version switches and include forwarding (ala Spirit2) such that old client code will remain as-is and new code can use lambda2 (aka phoenix) Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
David Abrahams wrote:
Eric Niebler wrote:
TODO: replace all Proto's function objects with this wrapper to enforce consistent behavior. All function objects across Boost that need to make rvalue/lvalue distinctions would need to be examined for consistency in this regard. Joel, I'm looking in Fusion's general direction.
I hear ya.
Yep. It's a big coordination job.
I've already taken the first and (IMO) most important step here by porting both Lambda and Phoenix to Proto. My opinion on the unification of Lambda and Phoenix is that it shouldn't happen. There are too many subtle differences in semantics. Rather, Lambda should be retired and Phoenix should be promoted (after being reviewed, etc.).
That's OK too, although Joel still seems to be intent on an eventual merger.
No, Eric talked me out of it. I guess, with that in light, I'll ask for a Phoenix review then. Ahhh! Another bungee jumping adventure! (CC'ing Ronald Garcia) Ronald, please consider this a formal review request. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
Joel de Guzman wrote:
David Abrahams wrote:
Eric Niebler wrote:
TODO: replace all Proto's function objects with this wrapper to enforce consistent behavior. All function objects across Boost that need to make rvalue/lvalue distinctions would need to be examined for consistency in this regard. Joel, I'm looking in Fusion's general direction.
I hear ya.
Yep. It's a big coordination job.
I've already taken the first and (IMO) most important step here by porting both Lambda and Phoenix to Proto. My opinion on the unification of Lambda and Phoenix is that it shouldn't happen. There are too many subtle differences in semantics. Rather, Lambda should be retired and Phoenix should be promoted (after being reviewed, etc.).
That's OK too, although Joel still seems to be intent on an eventual merger.
No, Eric talked me out of it. I guess, with that in light, I'll ask for a Phoenix review then. Ahhh! Another bungee jumping adventure!
(CC'ing Ronald Garcia)
Ronald, please consider this a formal review request.
If nobody objects I would like to serve as the review manager. Regards Hartmut
On Wed, Jul 2, 2008 at 9:56 AM, Joel de Guzman <joel@boost-consulting.com> wrote:
No, Eric talked me out of it. I guess, with that in light, I'll ask for a Phoenix review then. Ahhh! Another bungee jumping adventure!
(CC'ing Ronald Garcia)
Ronald, please consider this a formal review request.
This is one of my most awaited formal review requests second maybe only to Boost.Asio. :D I'm definitely looking forward to this one. -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
Eric Niebler:
Then get Bind working with the Phoenix/Proto placeholders.
#include <boost/is_placeholder.hpp> namespace boost { template<int I> struct is_placeholder< phoenix::actor< phoenix::argument<I> >
{ enum _vt { value = I+1 }; }; } // namespace boost
On Mon, Jun 30, 2008 at 9:37 PM, David Abrahams <dave@boostpro.com> wrote:
Dean Michael Berris wrote:
I think it would be easy to agree that we'd love to have everything in Boost work like it new every other library in it, but that makes Boost more a Framework than a library collection. I don't necessarily think that would be a Bad Thing, but rather too close a coupling for my taste.
I don't think there's anything "coupled" about coordinating foundational parts so they interoperate properly.
Right, but I was stating this more in terms of dependency between/among components.
Here are the parts and topics that I see as being in the category:
result_of
* Too hard to define result<> templates for polymorphic function objects.
Isn't this one of the issues the EGG library is supposed to address? [snip]
BOOST_TYPEOF
* Doesn't coordinate with result_of
range_ex
* Not in Boost. Should have pipe syntax. Maybe should use expression templates and be lazier.
This is scheduled already for review right? [snip]
our own implementations of std::containers (in interprocess, IIRC)
* Not well-known.
I agree.
* Not in a general place (bound to interprocess?)
Should there be a Boost.Containers library where we distill all the containers implemented in a myriad of libraries? Multi-Index and Bimap come to mind...
* Ought to support move semantics
lambda/phoenix/bind
* Incompatible placeholders * Been waiting for the lambda/phoenix merger forever * Lambda has lots of limitations and gotchas that we can improve on now * Phoenix 2.0 not in Boost
Is Phoenix 2.0 not in the review queue yet?
move semantics
* Not supported by many foundational Boost types that could benefit (e.g. shared_ptr) * No general move semantics library in Boost
Should we borrow the ASL move library implementation?
Boost.Iterator
* Needs an update (e.g. "new iterator concepts" have to go -- C++0x resolves that differently by simply allowing proxies everywhere) * Several important fixes and features needed
Interesting... Do you have a list of important fixes and features needed?
Boost.Algorithm
* Should support ranges, move semantics, segmented iterators, property maps
I agree.
segmented iterator support (http://lafstern.org/matt/segmented.pdf) property maps
The above are just related to the rest of them.
Should Boost.Iterator have a 'segmented_iterator' type?
How about we first collect the problems you encounter and what other people already encounter with the specific libraries you've listed above -- and what precisely or roughly you'd like to achieve with coodinating the effort(s) of the above.
* More "it just works out of the box" experience * More efficiency!
When you say "just works out of the box", do you have a concrete use case that would generally shed some light as to a possible specific context you want things to "just work"?
Would it be safe to say you'd like to coordinate these to achieve true "expressive C++" with libraries?
Something like that. I guess efficiency is the other big concern.
Ok. -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
Dean Michael Berris wrote: > On Mon, Jun 30, 2008 at 9:37 PM, David Abrahams <dave@boostpro.com> wrote: >>>> lambda/phoenix/bind >> * Incompatible placeholders >> * Been waiting for the lambda/phoenix merger forever >> * Lambda has lots of limitations and gotchas that we can improve on now >> * Phoenix 2.0 not in Boost >> > > Is Phoenix 2.0 not in the review queue yet? CC'ing both Jaakko and Eric. One of the obstacles towards merger is that Lambda has some quirks of its own that makes it difficult to provide full backwards compatibility. Eric ported Phoenix 2.0 to proto, making it Phoenix 3.0. In the course of the development, Eric and I seem to both coming to the conclusion that the best route is to leave the Lambda codebase alone and make Phoenix 3.0 the new lambda (i.e. lambda 2.0). And, similar to what we did with Spirit2, we can have an interim release that bundles both the old lambda and the new. With this approach, code that uses Lambda should should not do anything special. Users who want to take advantage of the features of Lambda-2 (aka Phoenix) can upgrade with some minimal code tweaks. If this is an acceptable solution to all parties involved (Jaakko?), then we can do this as early as 1.37 (alas, it's too late for 1.36). Your comments very welcome! BTW, Phoenix 2.0 will be in 1.36 as a sub-library of Spirit2. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
On Tue, Jul 1, 2008 at 2:30 PM, Joel de Guzman <joel@boost-consulting.com> wrote: <snip>
One of the obstacles towards merger is that Lambda has some quirks of its own that makes it difficult to provide full backwards compatibility. Eric ported Phoenix 2.0 to proto, making it Phoenix 3.0. In the course of the development, Eric and I seem to both coming to the conclusion that the best route is to leave the Lambda codebase alone and make Phoenix 3.0 the new lambda (i.e. lambda 2.0). And, similar to what we did with Spirit2, we can have an interim release that bundles both the old lambda and the new. With this approach, code that uses Lambda should should not do anything special. Users who want to take advantage of the features of Lambda-2 (aka Phoenix) can upgrade with some minimal code tweaks. If this is an acceptable solution to all parties involved (Jaakko?), then we can do this as early as 1.37 (alas, it's too late for 1.36).
Your comments very welcome!
Yes please. The only thing that prevents me to use Phoenix every day is the fact that it is not an official boost library. If it were to become 'first class' personally I would be glad to ditch lambda. Still I do not think that having both libraries for only one release is enough, probably boost should carry both of them for a while (3, 4 releases), with explicit deprecation makers. Boost.lambda is a big and complex library, and it might take time to do a good conversion. Now, some questions: - Does Phoenix 3.0 (or 2.0) compile faster or slower than boost.lambda (on common expressions at least). Seen the care that Eric took to tweak Proto, I would say faster. - Last time I checked Phoenix only had a monomoprhic bind. If you have polymorphic functions you have to convert them to lazy functions. I think that adding a polymorphic bind (like lambda shouldn't be hard). - Would it be possible to tell Phoenix the default capture mode, like in C++0x lambdas? By default lambda[] ( or any implicit lambda expression) would capture by value, but something like lambda_r[ ] (_r for reference) would get rid of many ref/cref at least for upward funargs. I do not know how easy is to do it, but I guess that proto make it possible (in fact I think that the reverse would be easier: capture by reference by default and then a top level lambdac would traverse the expression and convert any implicitly captured reference to a copy). - What are exactly the quirks that makes it hard to have full backward compatibility? Well except for 'sig' of course. -- gpd
Giovanni Piero Deretta wrote:
Yes please. The only thing that prevents me to use Phoenix every day is the fact that it is not an official boost library. If it were to become 'first class' personally I would be glad to ditch lambda.
Still I do not think that having both libraries for only one release is enough, probably boost should carry both of them for a while (3, 4 releases), with explicit deprecation makers. Boost.lambda is a big and complex library, and it might take time to do a good conversion.
I agree.
Now, some questions:
- Does Phoenix 3.0 (or 2.0) compile faster or slower than boost.lambda (on common expressions at least). Seen the care that Eric took to tweak Proto, I would say faster.
At the moment, Phoenix 3 (with Proto) compiles slower than Phoenix 2. I haven't benchmarked either against Lambda. Both Phoenix 2 and Lambda are surprisingly lightweight, so porting them to Proto without incurring a compile-time perf hit is a challenge. But I've only just begun optimizing it for compile times.
- Last time I checked Phoenix only had a monomoprhic bind. If you have polymorphic functions you have to convert them to lazy functions. I think that adding a polymorphic bind (like lambda shouldn't be hard).
- Would it be possible to tell Phoenix the default capture mode, like in C++0x lambdas? By default lambda[] ( or any implicit lambda expression) would capture by value, but something like lambda_r[ ] (_r for reference) would get rid of many ref/cref at least for upward funargs. I do not know how easy is to do it, but I guess that proto make it possible (in fact I think that the reverse would be easier: capture by reference by default and then a top level lambdac would traverse the expression and convert any implicitly captured reference to a copy).
An interesting suggestion. Joel?
- What are exactly the quirks that makes it hard to have full backward compatibility? Well except for 'sig' of course.
Wish I had written them all down. One that sticks in my mind is whether variables are captured by value or by reference. With Phoenix, it's very consistent ... all variables are captured by value, always. Lambda tries to be a bit more helpful, and it makes the rules more confusing, IMO. Most of the time, variables are captured by value, but a variable on the left of an assignment is captured by reference. This includes all the assignment operators, like plus-assign. So the following is a valid lambda expression: int i = 0; i += _1; But in Phoenix, it would have to be: int i = 0; ref(i) += _1; -- Eric Niebler BoostPro Computing http://www.boostpro.com
On Tue, Jul 1, 2008 at 5:39 PM, Eric Niebler <eric@boost-consulting.com> wrote:
Giovanni Piero Deretta wrote:
- What are exactly the quirks that makes it hard to have full backward compatibility? Well except for 'sig' of course.
Wish I had written them all down. One that sticks in my mind is whether variables are captured by value or by reference. With Phoenix, it's very consistent ... all variables are captured by value, always. Lambda tries to be a bit more helpful, and it makes the rules more confusing, IMO. Most of the time, variables are captured by value, but a variable on the left of an assignment is captured by reference. This includes all the assignment operators, like plus-assign. So the following is a valid lambda expression:
int i = 0; i += _1;
But in Phoenix, it would have to be:
int i = 0; ref(i) += _1;
D'uh, I didn't know boost.lambda did that! I always stick a ref() in these cases. The only "help" that lambda provided, that I knew of, was with 'cout' (and similar) and array types. So this won't affect me at least. -- gpd
Eric Niebler wrote:
Giovanni Piero Deretta wrote:
Yes please. The only thing that prevents me to use Phoenix every day is the fact that it is not an official boost library. If it were to become 'first class' personally I would be glad to ditch lambda.
Still I do not think that having both libraries for only one release is enough, probably boost should carry both of them for a while (3, 4 releases), with explicit deprecation makers. Boost.lambda is a big and complex library, and it might take time to do a good conversion.
I agree.
Yep. Just like Spirit "classic" will stay around for quite some time still.
- Last time I checked Phoenix only had a monomoprhic bind. If you have polymorphic functions you have to convert them to lazy functions. I think that adding a polymorphic bind (like lambda shouldn't be hard).
Boost.Lambda has polymorphic bind? How can it do that? Bind is inherently monomorphic. What am I missing?
- Would it be possible to tell Phoenix the default capture mode, like in C++0x lambdas? By default lambda[] ( or any implicit lambda expression) would capture by value, but something like lambda_r[ ] (_r for reference) would get rid of many ref/cref at least for upward funargs. I do not know how easy is to do it, but I guess that proto make it possible (in fact I think that the reverse would be easier: capture by reference by default and then a top level lambdac would traverse the expression and convert any implicitly captured reference to a copy).
An interesting suggestion. Joel?
Good ideas. It would help to provide code snippets/use cases.
- What are exactly the quirks that makes it hard to have full backward compatibility? Well except for 'sig' of course.
Wish I had written them all down. One that sticks in my mind is whether variables are captured by value or by reference. With Phoenix, it's very consistent ... all variables are captured by value, always. Lambda tries to be a bit more helpful, and it makes the rules more confusing, IMO. Most of the time, variables are captured by value, but a variable on the left of an assignment is captured by reference. This includes all the assignment operators, like plus-assign. So the following is a valid lambda expression:
int i = 0; i += _1;
But in Phoenix, it would have to be:
int i = 0; ref(i) += _1;
Yep. I personally find this one rather quirky. Another is the ->* syntax, and yet another is the "protect" feature in lambda that's supplanted by the more powerful 'lambda[]' syntax and the 'let' feature (local variables) in phoenix. There are some more, but I have to go through the tests and code to list them down. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
On Wed, Jul 2, 2008 at 4:12 AM, Joel de Guzman <joel@boost-consulting.com> wrote:
Giovanni Piero Deretta wrote:
- Last time I checked Phoenix only had a monomoprhic bind. If you have polymorphic functions you have to convert them to lazy functions. I think that adding a polymorphic bind (like lambda shouldn't be hard).
Boost.Lambda has polymorphic bind? How can it do that? Bind is inherently monomorphic. What am I missing?
Uh? Even C++0x bind is polymorphic: struct plus_t { template<class A, class B> [] operator()(A a, B b) -> decltype(a + b) { return a + b; } }; auto plus = std::bind(plus_t(), _1, _2); int a = plus(1, 2); std::string b = plus(std::string("hello"), std::string("world")); With appropriate 'sig' magic in plus_t you can do the same thing with boost.lambda (but not with boost.bind). Or we are talking about different kind of polymorphism? -- gpd
Giovanni Piero Deretta wrote:
On Wed, Jul 2, 2008 at 4:12 AM, Joel de Guzman <joel@boost-consulting.com> wrote:
Giovanni Piero Deretta wrote:
- Last time I checked Phoenix only had a monomoprhic bind. If you have polymorphic functions you have to convert them to lazy functions. I think that adding a polymorphic bind (like lambda shouldn't be hard). Boost.Lambda has polymorphic bind? How can it do that? Bind is inherently monomorphic. What am I missing?
Uh? Even C++0x bind is polymorphic:
struct plus_t { template<class A, class B> [] operator()(A a, B b) -> decltype(a + b) { return a + b; } };
auto plus = std::bind(plus_t(), _1, _2);
int a = plus(1, 2);
std::string b = plus(std::string("hello"), std::string("world"));
With appropriate 'sig' magic in plus_t you can do the same thing with boost.lambda (but not with boost.bind). Or we are talking about different kind of polymorphism?
Uh oh. I misunderstood you. Well, in that case, phoenix has polymorphic bind for function objects. It is only monomorphic when binding free functions and member functions. Example (this example is in the distro): struct sqr { template <typename Arg> struct result { typedef Arg type; }; template <typename Arg> Arg operator()(Arg n) const { return n * n; } }; ... BOOST_TEST(bind(sqr(), arg1)(i5) == (i5*i5)); (Note that the return type deduction was from the original phoenix 1.0 and predates result_of. This is changed in the latest version). Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
Giovanni Piero Deretta wrote:
- Last time I checked Phoenix only had a monomoprhic bind. If you have polymorphic functions you have to convert them to lazy functions. I think that adding a polymorphic bind (like lambda shouldn't be hard).
I think that ideally, there should be a single bind function (Boost.Bind, Boost.Lambda, Phoenix, etc.) Why is there a need for bind to be library-specific? Overloading is possible to handle special cases.
On Tue, Jul 1, 2008 at 6:10 PM, Mathias Gaunard <mathias.gaunard@ens-lyon.org> wrote:
Giovanni Piero Deretta wrote:
- Last time I checked Phoenix only had a monomoprhic bind. If you have polymorphic functions you have to convert them to lazy functions. I think that adding a polymorphic bind (like lambda shouldn't be hard).
I think that ideally, there should be a single bind function (Boost.Bind, Boost.Lambda, Phoenix, etc.) Why is there a need for bind to be library-specific? Overloading is possible to handle special cases.
Agree about boost.lambda and phoenix (in fact if they are both based on proto, it will probably work out of the box). Boost.bind, on the other hand, should probably remain similar to TR1/C++0x, i.e. no fancy operator overloads (other than relational). It is simpler to learn and more light weight. -- gèd
Giovanni Piero Deretta wrote:
On Tue, Jul 1, 2008 at 6:10 PM, Mathias Gaunard <mathias.gaunard@ens-lyon.org> wrote:
Giovanni Piero Deretta wrote:
- Last time I checked Phoenix only had a monomoprhic bind. If you have polymorphic functions you have to convert them to lazy functions. I think that adding a polymorphic bind (like lambda shouldn't be hard). I think that ideally, there should be a single bind function (Boost.Bind, Boost.Lambda, Phoenix, etc.) Why is there a need for bind to be library-specific? Overloading is possible to handle special cases.
Agree about boost.lambda and phoenix (in fact if they are both based on proto, it will probably work out of the box).
Boost.bind, on the other hand, should probably remain similar to TR1/C++0x, i.e. no fancy operator overloads (other than relational). It is simpler to learn and more light weight.
That's OK with me as long as there aren't any surprising semantic differences between bind and the bind subset of phoenix. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Joel de Guzman wrote: > Dean Michael Berris wrote: >> On Mon, Jun 30, 2008 at 9:37 PM, David Abrahams <dave@boostpro.com> >> wrote: > >>>>> lambda/phoenix/bind >>> * Incompatible placeholders >>> * Been waiting for the lambda/phoenix merger forever >>> * Lambda has lots of limitations and gotchas that we can improve on now >>> * Phoenix 2.0 not in Boost >>> >> >> Is Phoenix 2.0 not in the review queue yet? > > CC'ing both Jaakko and Eric. Jaakko is at TAMU now; I can send you his address if you want. > One of the obstacles towards merger is that Lambda has some > quirks of its own that makes it difficult to provide full backwards > compatibility. Eric ported Phoenix 2.0 to proto, making it Phoenix > 3.0. In the course of the development, Eric and I seem to both > coming to the conclusion that the best route is to leave the > Lambda codebase alone and make Phoenix 3.0 the new lambda > (i.e. lambda 2.0). And, similar to what we did with Spirit2, > we can have an interim release that bundles both the old lambda > and the new. With this approach, code that uses Lambda should > should not do anything special. Users who want to take advantage > of the features of Lambda-2 (aka Phoenix) can upgrade with some > minimal code tweaks. If this is an acceptable solution to all > parties involved (Jaakko?) If it works for Jaakko, it works for me. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams <dave@boostpro.com> writes: > Joel de Guzman wrote: >> Dean Michael Berris wrote: >>> On Mon, Jun 30, 2008 at 9:37 PM, David Abrahams <dave@boostpro.com> >>> wrote: >> >>>>>> lambda/phoenix/bind >>>> * Incompatible placeholders >>>> * Been waiting for the lambda/phoenix merger forever >>>> * Lambda has lots of limitations and gotchas that we can improve on now >>>> * Phoenix 2.0 not in Boost >>>> >>> >>> Is Phoenix 2.0 not in the review queue yet? >> >> CC'ing both Jaakko and Eric. > > Jaakko is at TAMU now; I can send you his address if you want. > >> One of the obstacles towards merger is that Lambda has some >> quirks of its own that makes it difficult to provide full backwards >> compatibility. Eric ported Phoenix 2.0 to proto, making it Phoenix >> 3.0. In the course of the development, Eric and I seem to both >> coming to the conclusion that the best route is to leave the >> Lambda codebase alone and make Phoenix 3.0 the new lambda >> (i.e. lambda 2.0). And, similar to what we did with Spirit2, >> we can have an interim release that bundles both the old lambda >> and the new. With this approach, code that uses Lambda should >> should not do anything special. Users who want to take advantage >> of the features of Lambda-2 (aka Phoenix) can upgrade with some >> minimal code tweaks. If this is an acceptable solution to all >> parties involved (Jaakko?) > > If it works for Jaakko, it works for me. My first reply didn't make it to the list. Yes, this would be a good arrangement. Jaakko
Dean Michael Berris wrote:
When you say "just works out of the box", do you have a concrete use case that would generally shed some light as to a possible specific context you want things to "just work"?
Personally, I would have expected Boost.Lambda to work out of the box with range algorithms and the like. However, the sig template thingy that Boost.Lambda uses is not supported by these.
Mathias Gaunard wrote:
Dean Michael Berris wrote:
When you say "just works out of the box", do you have a concrete use case that would generally shed some light as to a possible specific context you want things to "just work"?
Personally, I would have expected Boost.Lambda to work out of the box with range algorithms and the like.
Phoenix does. All range algos and container intrinsics.
However, the sig template thingy that Boost.Lambda uses is not supported by these.
Yep. Another feature that makes backward compatibility imperfect. There's no sense in supporting that feature when Phoenix uses result_of and BOOST_TYPEOF. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
Joel de Guzman wrote:
Yep. Another feature that makes backward compatibility imperfect. There's no sense in supporting that feature when Phoenix uses result_of and BOOST_TYPEOF.
Can you explain how Phoenix uses these facilities together? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
Joel de Guzman wrote:
Yep. Another feature that makes backward compatibility imperfect. There's no sense in supporting that feature when Phoenix uses result_of and BOOST_TYPEOF.
Can you explain how Phoenix uses these facilities together?
You should ask Eric. He did that part when he ported it to proto. Phoenix used to have its own return type deduction which was a very primitive rendition of BOOST_TYPEOF using the sizeof trick I hacked a long time ago. Eric switched to result_of and BOOST_TYPEOF. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
Mathias Gaunard wrote:
Joel de Guzman wrote:
You should ask Eric. He did that part when he ported it to proto.
So that's only in Phoenix 3?
Yes, but I intend to backport that part back to Phx2 very soon. It might be that Phx2 is the one that'll be reviewed if Phx3 is not yet ready by then. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
Joel de Guzman wrote:
David Abrahams wrote:
Joel de Guzman wrote:
Yep. Another feature that makes backward compatibility imperfect. There's no sense in supporting that feature when Phoenix uses result_of and BOOST_TYPEOF.
Can you explain how Phoenix uses these facilities together?
You should ask Eric. He did that part when he ported it to proto. Phoenix used to have its own return type deduction which was a very primitive rendition of BOOST_TYPEOF using the sizeof trick I hacked a long time ago. Eric switched to result_of and BOOST_TYPEOF.
The Phoenix/Proto port in (Phoenix v3) uses BOOST_TYPEOF to deduce the the type of expressions involving operators, with the exception of the function call operator. For the function call operator, Phoenix v3 uses result_of to calculate the return type. To extend Phoenix v3 with a lazy function, your function object must conform to the result_of protocol. Phoenix lambda expressions are themselves function objects and conform to the result_of protocol. This is true for the (still immature) Phoenix v3. Phoenix v2 uses its own protocols. -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
Joel de Guzman wrote:
David Abrahams wrote:
Joel de Guzman wrote:
Yep. Another feature that makes backward compatibility imperfect. There's no sense in supporting that feature when Phoenix uses result_of and BOOST_TYPEOF.
Can you explain how Phoenix uses these facilities together?
You should ask Eric. He did that part when he ported it to proto. Phoenix used to have its own return type deduction which was a very primitive rendition of BOOST_TYPEOF using the sizeof trick I hacked a long time ago. Eric switched to result_of and BOOST_TYPEOF.
The Phoenix/Proto port in (Phoenix v3) uses BOOST_TYPEOF to deduce the the type of expressions involving operators, with the exception of the function call operator. For the function call operator, Phoenix v3 uses result_of to calculate the return type.
Interesting. Kinda makes sense, but could you explain the rationale anyway? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
Eric Niebler wrote:
The Phoenix/Proto port in (Phoenix v3) uses BOOST_TYPEOF to deduce the the type of expressions involving operators, with the exception of the function call operator. For the function call operator, Phoenix v3 uses result_of to calculate the return type.
Interesting. Kinda makes sense, but could you explain the rationale anyway?
You mean, the rationale for using result_of instead of BOOST_TYPEOF for the function call operator? BOOST_TYPEOF can't perfectly distinguish between lvalues and rvalues. I fudge it where I have to, but with function calls no fudging is necessary because result_of presumably gives the right answer. -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
The Phoenix/Proto port in (Phoenix v3) uses BOOST_TYPEOF to deduce the the type of expressions involving operators, with the exception of the function call operator. For the function call operator, Phoenix v3 uses result_of to calculate the return type.
Interesting. Kinda makes sense, but could you explain the rationale anyway?
You mean, the rationale for using result_of instead of BOOST_TYPEOF for the function call operator? BOOST_TYPEOF can't perfectly distinguish between lvalues and rvalues. I fudge it where I have to, but with function calls no fudging is necessary because result_of presumably gives the right answer.
Assuming it's implemented, yeah. It might be nice to be able to fudge when it isn't :-) -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
Eric Niebler wrote:
Eric Niebler wrote:
The Phoenix/Proto port in (Phoenix v3) uses BOOST_TYPEOF to deduce the the type of expressions involving operators, with the exception of the function call operator. For the function call operator, Phoenix v3 uses result_of to calculate the return type. Interesting. Kinda makes sense, but could you explain the rationale anyway? You mean, the rationale for using result_of instead of BOOST_TYPEOF for
David Abrahams wrote: the function call operator? BOOST_TYPEOF can't perfectly distinguish between lvalues and rvalues. I fudge it where I have to, but with function calls no fudging is necessary because result_of presumably gives the right answer.
Assuming it's implemented, yeah. It might be nice to be able to fudge when it isn't :-)
I don't know a sufficiently portable way to ask a type whether it has implemented the result_of protocol. In C++0x, result_of will simply use decltype, so at the very least Phoenix's approach is forward-compatible. -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
Eric Niebler wrote:
The Phoenix/Proto port in (Phoenix v3) uses BOOST_TYPEOF to deduce the the type of expressions involving operators, with the exception of the function call operator. For the function call operator, Phoenix v3 uses result_of to calculate the return type. Interesting. Kinda makes sense, but could you explain the rationale anyway? You mean, the rationale for using result_of instead of BOOST_TYPEOF for
David Abrahams wrote: the function call operator? BOOST_TYPEOF can't perfectly distinguish between lvalues and rvalues. I fudge it where I have to, but with function calls no fudging is necessary because result_of presumably gives the right answer.
Assuming it's implemented, yeah. It might be nice to be able to fudge when it isn't :-)
I don't know a sufficiently portable way to ask a type whether it has implemented the result_of protocol.
Me niether. Too bad (?) result_of wasn't designed with that in mind :(
In C++0x, result_of will simply use decltype, so at the very least Phoenix's approach is forward-compatible.
Sounds like about the best we can do. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Wed, Jul 2, 2008 at 8:40 PM, Eric Niebler <eric@boost-consulting.com> wrote:
David Abrahams wrote:
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
The Phoenix/Proto port in (Phoenix v3) uses BOOST_TYPEOF to deduce the the type of expressions involving operators, with the exception of the function call operator. For the function call operator, Phoenix v3 uses result_of to calculate the return type.
Interesting. Kinda makes sense, but could you explain the rationale anyway?
You mean, the rationale for using result_of instead of BOOST_TYPEOF for the function call operator? BOOST_TYPEOF can't perfectly distinguish between lvalues and rvalues. I fudge it where I have to, but with function calls no fudging is necessary because result_of presumably gives the right answer.
Assuming it's implemented, yeah. It might be nice to be able to fudge when it isn't :-)
I don't know a sufficiently portable way to ask a type whether it has implemented the result_of protocol. In C++0x, result_of will simply use decltype, so at the very least Phoenix's approach is forward-compatible.
With 'portable' you mean 'works with non completely compliant compilers'? Otherwise I think you can legally sfinae on the presence of both result<> and result_type. -- gpd
Giovanni Piero Deretta wrote:
On Wed, Jul 2, 2008 at 8:40 PM, Eric Niebler <eric@boost-consulting.com> wrote:
I don't know a sufficiently portable way to ask a type whether it has implemented the result_of protocol. In C++0x, result_of will simply use decltype, so at the very least Phoenix's approach is forward-compatible.
With 'portable' you mean 'works with non completely compliant compilers'? Otherwise I think you can legally sfinae on the presence of both result<> and result_type.
Demonstration, please. I don't know of a way to detect a nested template with SFINAE. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Thu, Jul 3, 2008 at 12:06 AM, David Abrahams <dave@boostpro.com> wrote:
Giovanni Piero Deretta wrote:
On Wed, Jul 2, 2008 at 8:40 PM, Eric Niebler <eric@boost-consulting.com> wrote:
I don't know a sufficiently portable way to ask a type whether it has implemented the result_of protocol. In C++0x, result_of will simply use decltype, so at the very least Phoenix's approach is forward-compatible.
With 'portable' you mean 'works with non completely compliant compilers'? Otherwise I think you can legally sfinae on the presence of both result<> and result_type.
Demonstration, please. I don't know of a way to detect a nested template with SFINAE.
Daniel Walker posted a patch sometime ago to add BOOST_HAS_TEMPLATE_XXX to mpl. Here is a link: http://svn.boost.org/trac/boost/ticket/861 I'm sure that Eric is aware of it (and you too actually :) ) but IIRC he feared the amount of compiler workarounds. I use a variant of this to dinstinguish between 'sig' and 'result' and work fairly well on gcc 3.3 and 4.x. Here is a short test: #include <boost/static_assert.hpp> typedef char true_type[2]; typedef char false_type[1]; false_type& test(...); template<class T, class T2> struct dummy{}; template<class T1, class T2> true_type& test(dummy<T1, T2>, typename T1:: template result<T2>* =0); struct foo {}; struct bar { template<typename T> struct result{}; }; struct baz { int result; }; struct barf { typedef int result; }; struct bas { template<class T1, class T2> struct result{} ; }; struct bax { template<class T1, class T2=void> struct result{} ; }; BOOST_STATIC_ASSERT(sizeof(test(dummy<foo, int()>())) == sizeof(false_type)); BOOST_STATIC_ASSERT(sizeof(test(dummy<bar, int()>())) == sizeof(true_type)); BOOST_STATIC_ASSERT(sizeof(test(dummy<baz, int()>())) == sizeof(false_type)); BOOST_STATIC_ASSERT(sizeof(test(dummy<barf, int()>())) == sizeof(false_type)); //BOOST_STATIC_ASSERT(sizeof(test(dummy<bas, int()>())) == sizeof(false_type)); BOOST_STATIC_ASSERT(sizeof(test(dummy<bax, int()>())) == sizeof(true_type)); Unfortunately it fails (with a compiler error, not with a misleading detection) if a class has a nested result with more than one (non defaulted) argument. I think that Daniel Walker found some tricks plus compier workarounds to overcome this problem for the general MPL_HAS_TEMPLATE_XXX, But even with the plain test, it is not a big stretch to assume that a function object will not have result<> nested template unless it is trying to implement the result protocol. HTH, -- gpd
Giovanni Piero Deretta wrote:
On Thu, Jul 3, 2008 at 12:06 AM, David Abrahams <dave@boostpro.com> wrote:
Giovanni Piero Deretta wrote:
On Wed, Jul 2, 2008 at 8:40 PM, Eric Niebler <eric@boost-consulting.com> wrote:
I don't know a sufficiently portable way to ask a type whether it has implemented the result_of protocol. In C++0x, result_of will simply use decltype, so at the very least Phoenix's approach is forward-compatible.
With 'portable' you mean 'works with non completely compliant compilers'? Otherwise I think you can legally sfinae on the presence of both result<> and result_type. Demonstration, please. I don't know of a way to detect a nested template with SFINAE.
Daniel Walker posted a patch sometime ago to add BOOST_HAS_TEMPLATE_XXX to mpl. Here is a link:
No patch attached. This probably came over from the sourceforge tracker, but that appears to be gone now so I can't even look there for the attachment. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Thu, Jul 3, 2008 at 1:33 AM, David Abrahams <dave@boostpro.com> wrote:
Giovanni Piero Deretta wrote:
On Thu, Jul 3, 2008 at 12:06 AM, David Abrahams <dave@boostpro.com> wrote:
Giovanni Piero Deretta wrote:
On Wed, Jul 2, 2008 at 8:40 PM, Eric Niebler <eric@boost-consulting.com> wrote:
I don't know a sufficiently portable way to ask a type whether it has implemented the result_of protocol. In C++0x, result_of will simply use decltype, so at the very least Phoenix's approach is forward-compatible.
With 'portable' you mean 'works with non completely compliant compilers'? Otherwise I think you can legally sfinae on the presence of both result<> and result_type. Demonstration, please. I don't know of a way to detect a nested template with SFINAE.
Daniel Walker posted a patch sometime ago to add BOOST_HAS_TEMPLATE_XXX to mpl. Here is a link:
No patch attached. This probably came over from the sourceforge tracker, but that appears to be gone now so I can't even look there for the attachment.
Righ! Hum, you can search the mailing list archives for has_template_xxx to find Daniel Walker posts dated end of 2007 with the patch attached. -- gpd
Eric Niebler wrote:
David Abrahams wrote:
Eric Niebler wrote:
The Phoenix/Proto port in (Phoenix v3) uses BOOST_TYPEOF to deduce the the type of expressions involving operators, with the exception of the function call operator. For the function call operator, Phoenix v3 uses result_of to calculate the return type.
Interesting. Kinda makes sense, but could you explain the rationale anyway?
You mean, the rationale for using result_of instead of BOOST_TYPEOF for the function call operator? BOOST_TYPEOF can't perfectly distinguish between lvalues and rvalues.
Yeah, that's my basic problem with BOOST_TYPEOF. I fudge it where I have to, Where do you fudge it? if_then expression? How? How about ordinary operators like a + b? Do you simply not allow weird uses like: X& operator+(A a, B b) return a reference?? Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
Joel de Guzman wrote:
Eric Niebler wrote:
BOOST_TYPEOF can't perfectly distinguish between lvalues and rvalues.
Yeah, that's my basic problem with BOOST_TYPEOF.
I fudge it where I have to,
Where do you fudge it? if_then expression? How? How about ordinary operators like a + b? Do you simply not allow weird uses like:
X& operator+(A a, B b)
return a reference??
I use BOOST_TYPEOF to get the type of an expression. Then, I check to see if the expression can bind to a non-const reference using the sizeof trick. If it does, then I treat the result as an lvalue and add a reference to the expression's type. It gets the answer wrong for const-qualified rvalues, but in the above example, it would correctly deduce the type of "A()+B()" to be "X&". -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
Joel de Guzman wrote:
Eric Niebler wrote:
BOOST_TYPEOF can't perfectly distinguish between lvalues and rvalues.
Yeah, that's my basic problem with BOOST_TYPEOF.
I fudge it where I have to,
Where do you fudge it? if_then expression? How? How about ordinary operators like a + b? Do you simply not allow weird uses like:
X& operator+(A a, B b)
return a reference??
I use BOOST_TYPEOF to get the type of an expression. Then, I check to see if the expression can bind to a non-const reference using the sizeof trick. If it does, then I treat the result as an lvalue and add a reference to the expression's type. It gets the answer wrong for const-qualified rvalues, but in the above example, it would correctly deduce the type of "A()+B()" to be "X&".
Ah. Makes perfect sense. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
Dean Michael Berris wrote: > On Mon, Jun 30, 2008 at 9:37 PM, David Abrahams <dave@boostpro.com> wrote: >> Dean Michael Berris wrote: >>> I think it would be easy to >>> agree that we'd love to have everything in Boost work like it new >>> every other library in it, but that makes Boost more a Framework than >>> a library collection. I don't necessarily think that would be a Bad >>> Thing, but rather too close a coupling for my taste. >> I don't think there's anything "coupled" about coordinating foundational >> parts so they interoperate properly. >> > > Right, but I was stating this more in terms of dependency > between/among components. I don't think there needs to be such dependency for the most part. You can support result_of without including its header. >>>> Here are the parts and topics that I see as being in the category: >>>> >>>> result_of >> * Too hard to define result<> templates for polymorphic function objects. >> > > Isn't this one of the issues the EGG library is supposed to address? perhaps. IIRC it was not accepted. > [snip] >>>> BOOST_TYPEOF >> * Doesn't coordinate with result_of >> >>>> range_ex >> * Not in Boost. Should have pipe syntax. Maybe should use expression >> templates and be lazier. >> > > This is scheduled already for review right? Perhaps > Should there be a Boost.Containers library where we distill all the > containers implemented in a myriad of libraries? Multi-Index and Bimap > come to mind... Definitely >> * Ought to support move semantics >> >>>> lambda/phoenix/bind >> * Incompatible placeholders >> * Been waiting for the lambda/phoenix merger forever >> * Lambda has lots of limitations and gotchas that we can improve on now >> * Phoenix 2.0 not in Boost >> > > Is Phoenix 2.0 not in the review queue yet? I don't know >>>> move semantics >> * Not supported by many foundational Boost types that could benefit >> (e.g. shared_ptr) >> * No general move semantics library in Boost >> > > Should we borrow the ASL move library implementation? No need to borrow; Adobe has told us we're welcome to move it into Boost >>>> Boost.Iterator >> * Needs an update (e.g. "new iterator concepts" have to go -- C++0x >> resolves that differently by simply allowing proxies everywhere) >> * Several important fixes and features needed >> > > Interesting... Do you have a list of important fixes and features needed? See Trac. > Should Boost.Iterator have a 'segmented_iterator' type? Not exactly, but I suppose any facility for supporting segmented iterators should live in the iterator lib, yeah -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Wed, Jul 2, 2008 at 12:03 AM, David Abrahams <dave@boostpro.com> wrote: > Dean Michael Berris wrote: >> On Mon, Jun 30, 2008 at 9:37 PM, David Abrahams <dave@boostpro.com> wrote: >>> Dean Michael Berris wrote: >>>> I think it would be easy to >>>> agree that we'd love to have everything in Boost work like it new >>>> every other library in it, but that makes Boost more a Framework than >>>> a library collection. I don't necessarily think that would be a Bad >>>> Thing, but rather too close a coupling for my taste. >>> I don't think there's anything "coupled" about coordinating foundational >>> parts so they interoperate properly. >>> >> >> Right, but I was stating this more in terms of dependency >> between/among components. > > I don't think there needs to be such dependency for the most part. You > can support result_of without including its header. > Ok. >>>>> Here are the parts and topics that I see as being in the category: >>>>> >>>>> result_of >>> * Too hard to define result<> templates for polymorphic function objects. >>> >> >> Isn't this one of the issues the EGG library is supposed to address? > > perhaps. IIRC it was not accepted. > Yeah. >> [snip] >>>>> BOOST_TYPEOF >>> * Doesn't coordinate with result_of >>> >>>>> range_ex >>> * Not in Boost. Should have pipe syntax. Maybe should use expression >>> templates and be lazier. >>> >> >> This is scheduled already for review right? > > Perhaps > I guess then this should be one of the important parts that really need to get in there. >> Should there be a Boost.Containers library where we distill all the >> containers implemented in a myriad of libraries? Multi-Index and Bimap >> come to mind... > > Definitely > >>> * Ought to support move semantics >>> >>>>> lambda/phoenix/bind >>> * Incompatible placeholders >>> * Been waiting for the lambda/phoenix merger forever >>> * Lambda has lots of limitations and gotchas that we can improve on now >>> * Phoenix 2.0 not in Boost >>> >> >> Is Phoenix 2.0 not in the review queue yet? > > I don't know > After reading some of the other emails in the thread, I'd think there are already some avenues worth exploring as far as getting Phoenix 2/3 is concerned -- which can only mean good things. :D >>>>> move semantics >>> * Not supported by many foundational Boost types that could benefit >>> (e.g. shared_ptr) >>> * No general move semantics library in Boost >>> >> >> Should we borrow the ASL move library implementation? > > No need to borrow; Adobe has told us we're welcome to move it into Boost > Nice. >>>>> Boost.Iterator >>> * Needs an update (e.g. "new iterator concepts" have to go -- C++0x >>> resolves that differently by simply allowing proxies everywhere) >>> * Several important fixes and features needed >>> >> >> Interesting... Do you have a list of important fixes and features needed? > > See Trac. > >> Should Boost.Iterator have a 'segmented_iterator' type? > > Not exactly, but I suppose any facility for supporting segmented > iterators should live in the iterator lib, yeah > Ok. -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
Dave sent me the following off list: ---- Hi Sean, I'm starting to notice more and more bits of ASL that we want to use in Boost, and it would be unfortunate if there was a cyclic dependency between these two libraries. Somehow I think this is related to the thread beginning here: http://news.gmane.org/find-root.php?message_id=%3c486394D0.3040500% 40boostpro.com%3e although I admit to not having a very clear view of the relationship. If you have any thoughts it would be great if you'd post them. --- I'm pleased that there are bits of ASL (Adobe Source Libraries) that folks would find useful in Boost - Currently we're struggling to get more documentation written and papers published, support from the Boost community could be one way to move ASL forward. From the outset of ASL I've known there would be overlap with Boost - but ASL has a research agenda and pushing adoption and standardization is not part of that agenda. However, we are 100% open to anyone who wants to take portions of ASL and incorporate them into Boost - we're also willing to provide guidance and feedback and assist with documentation and test cases to the level we would have to do so otherwise to complete the library for ASL. Someone in the boost community will have to "take point" on shepherding the library into boost though as I know from experience that exposure in Boost leads to far more feedback than my team can afford to process. To avoid circular dependencies I would suggest we simply move any item in ASL that Boost plans to adopt into the Boost namespace and directory structure - we already manage that fairly well for the GIL libraries. I don't usually monitor the boost lists so reply directly if you want me to see the response. Sean Parent
Mathias Gaunard wrote:
Steven Watanabe wrote:
Can't you use result_of?
As far as I can see, the current boost::result_of implementation doesn't use decltype or typeof. I suppose implementing decltype on top of typeof is not a big problem (you only need to check whether the expression is a reference or not) however both have problems with mangling when used in a return type, at least in GCC. That basically means you can't always rely on them working. I thought decltype didn't have those mangling problems, looks like it is a regression.
As for the correct way to do that, I suppose it should be possible to tell boost::result_type with a macro definition to use Boost.Typeof, which in turn can use either native or emulation.
However range_ex only forwards algorithms and adaptors to Boost.Iterator and maybe the standard library. Maybe it's these that ought to use boost::result_of in the first place?
I've been thinking for a long time that we need to develop a coordinated approach to developing some of these more "basic" facilities. We have too many great bits and pieces that don't quite interoperate as well as they could. This was driven home for me recently when I taught a course on Boost with hands-on workshops and the number of pitfalls that people (including me!) ran into was just too high. In order to really make the case for the power of library use, it needs to be easier to write expressive C++. I'm not sure exactly how to proceed from here. Maybe the first thing is to develop a list of things in (or that should be in) Boost that seem to fall into this category, because I don't really understand the rules my brain has used to decide the category boundaries. Maybe after that we need a "library domain maintainer" or something who can see to the coordination of these things... but let's not get ahead of ourselves. Here are the parts and topics that I see as being in the category: result_of BOOST_TYPEOF range_ex BOOST_FOREACH our own implementations of std::containers (in interprocess, IIRC) lambda/phoenix/bind move semantics Boost.Iterator Boost.Algorithm segmented iterator support (http://lafstern.org/matt/segmented.pdf) property maps Thoughts?
Neil Groves wrote:
Mathias Gaunard wrote:
Neil Groves wrote:
range_ex.zip has been updated in the root of the Vault to correct the folder structure.
I just downloaded your library and I was surprised to see the make_filtered_range-like syntax was dropped. While the pipe-like syntax is interesting, I think both interfaces should be there.
Thank you for this feedback. I hope to get a few more responses and then respond appropriately. I do not have a strong preference one way or another. If no one objects within this week then I shall reintroducing make_filtered_range et al. I expect to be able to upload by 6 June 2008.
Yes, I'd add a vote to see make_***_range variants of counting, transformed, filtered, reversed and indirect. Did you mean 6 July 2008? Thanks, Jeff
On Wed, Jun 25, 2008 at 3:17 PM, Jeff Flinn <TriumphSprint2000@hotmail.com> wrote:
Neil Groves wrote:
Mathias Gaunard wrote:
Neil Groves wrote:
range_ex.zip has been updated in the root of the Vault to correct the folder structure.
I just downloaded your library and I was surprised to see the make_filtered_range-like syntax was dropped. While the pipe-like syntax is interesting, I think both interfaces should be there.
Yes, I'd add a vote to see make_***_range variants of counting, transformed, filtered, reversed and indirect.
Did you mean 6 July 2008?
I am sorry, I do indeed mean 6 July 2008.
Thanks, Jeff
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Jeff Flinn wrote:
Yes, I'd add a vote to see make_***_range variants of counting, transformed, filtered, reversed and indirect.
Thanks, Jeff
An update to the Boost.RangeEx proposal has been uploaded to the vault. Please take note that Eric has moved his older version of Boost.RangeEx (upon which much of the later work is based). I have therefore placed the new version in the Algorithms folder of the vault and removed the version in the root. The updated code is to allow make_***_range as requested by a few users of RangeEx. Feedback is welcomed. Neil Groves
On Sun, Jun 29, 2008 at 19:43, Neil Groves <neilgroves@googlemail.com>wrote:
An update to the Boost.RangeEx proposal has been uploaded to the vault. Please take note that Eric has moved his older version of Boost.RangeEx (upon which much of the later work is based). I have therefore placed the new version in the Algorithms folder of the vault and removed the version in the root.
The updated code is to allow make_***_range as requested by a few users of RangeEx.
Feedback is welcomed.
Dear Neil, I've got a couple of questions. They might just highlight my ignorance. I'm sorry if I have missed available documentation and would be happy to be pointed to it where I have. What's the good thing about output iterators? I always regarded them as a clumsy way of returning a range. For example, shouldn't generate_n() just return a lazy range? If I understand correctly, the pipe operator is the main modus operandi. Why? "range | filtered(pred)" does not look as familiar to C++ readers as "filter(range, predicate)". The functional form also allows for operations on two ranges, like "merge(range1, range2)". I have found this useful in the past. (I would be happy to contribute an implementation of this "merge" that I have lying around.) If operator| is so cute it needs to be provided (which is a valid argument), is it possible to turn it into something more general? I can now get an iterator to the last element of a vector v with "prior(end(v))". Why not "v | end | prior"? Cheers, Rogier
Mathias Gaunard wrote:
I'm writing some piece of software working that does a lot of data structures traversal and that really needed filtering of traversal etc.
I thought I'd get that from Boost.Range, but I was surprised to see Boost.Range provides very little: all the good stuff (the range-based algorithms and adaptors) are in another library, range_ex, which isn't in the main distribution yet.
Will this library be included in Boost soon? It's really great.
As for the open issues, here are my opinions: - Output ranges: don't adaptors already output ranges? - Two input ranges, or a range and an iterator?: two ranges can do more - For any algorithm foo(container, args...), if foo(args...) is valid then it should be called - Null-termination of C-style strings: Isn't that up to Boost.Range to decide whether it's a range or not?
It sounds like you're referring to my old range_ex library in the Algorithms directory in the Boost File Vault, is that right? Somewhat confusingly, there is a *different* range_ex.zip in the Boost File Vault, but at the top-level, here: http://www.boostpro.com/vault/ That one appears to be an extension of my earlier work (which itself was an extension of the work of several others ... range algorithms and adaptors have been a long time in coming). The good news is that this new range_ex library is currently in the review queue. From the review schedule (http://www.boost.org/community/review_schedule.html), it looks like we're still looking for a review manager. Care to volunteer? At the very least, you should have a look at this new version and let us know if it meets your needs. Thanks, -- Eric Niebler BoostPro Computing http://www.boostpro.com
participants (18)
-
Daniel Walker -
David Abrahams -
Dean Michael Berris -
Eric Niebler -
Giovanni Piero Deretta -
Hartmut Kaiser -
jarvi＠cs.tamu.edu -
Jeff Flinn -
Joel de Guzman -
Maik Beckmann -
Mathias Gaunard -
Neil Groves -
Neil Groves -
Peter Dimov -
Rogier van Dalen -
Sean Parent -
Steven Watanabe -
Thorsten Ottosen