[mpl] Nested Scopes w/Placeholders

I've recently been doing a lot mpl::folding (in boost 1.47), and I was disappointed when I realized that a nested fold wouldn't work with a lambda expression passed to the inner call, as in this example which extracts elements of sequences: using namespace mpl; using namespace mpl::placeholders; typedef vector< vector<char>, vector<int> > vec_of_vecs; typedef vector< char, int > expected_result; typedef fold< vec_of_vecs, vector<>, fold< _2, _1, *push_back< _1, _2 >* // wrong - refers to outer arguments >
::type result;
BOOST_MPL_ASSERT(( equal<result, expected_result> )); After looking around for a solution I came across an old thread<http://lists.boost.org/boost-users/2004/12/9269.php> on the same subject, where it was suggested using mpl::protect might work. Indeed, the following modification does what I want, without any changes to the library: typedef fold< vec_of_vecs, vector<>, fold< _2, _1, *protect< lambda< push_back< _1, _2 > >::type
* // correct > ::type result;
Of course, this is a little on the verbose side. In the old thread, Daniel Wallin suggested suggested changing the definition of lambda to yield syntax more in line with the runtime version of boost::bind, but that would apparently break other code. What about instead adding a specialization to protect for placeholder expressions? Indeed, the source in protect.hpp seems to indicate such a thing might've been in the works at one point, but I didn't see any more references to it: template< typename BOOST_MPL_AUX_NA_PARAM(T) * , int not_le_ = 0 // not lambda expression?* > struct protect : T { ... typedef protect type; }; What about changing it to something like this? template< typename BOOST_MPL_AUX_NA_PARAM(T) * , typename not_le_ = typename is_lambda_expression< T >::type* > struct protect : T { ... typedef protect type; }; *template< typename T >* *struct protect< T, mpl::true_ > : lambda<T>* *{* *...* *typedef protect type;* *};* Doing so would remove the inner lambda<...>::type expression, yielding the expected syntax. Could that be done without messing up everything else? Thanks, *Nick Kitten* Software Engineer Center for Video Understanding Excellence ObjectVideo, Inc.

On 01/13/12 19:44, Kitten, Nicholas wrote:
I've recently been doing a lot mpl::folding (in boost 1.47), and I was disappointed when I realized that a nested fold wouldn't work with a lambda expression passed to the inner call, as in this example which extracts elements of sequences:
using namespace mpl; using namespace mpl::placeholders;
typedef vector< vector<char>, vector<int> > vec_of_vecs; typedef vector< char, int > expected_result;
typedef fold< vec_of_vecs, vector<>, fold< _2, _1, *push_back< _1, _2 >* // wrong - refers to outer arguments >
::type result;
BOOST_MPL_ASSERT(( equal<result, expected_result> ));
After looking around for a solution I came across an old thread<http://lists.boost.org/boost-users/2004/12/9269.php> on the same subject, where it was suggested using mpl::protect might work. Indeed, the following modification does what I want, without any changes to the library:
typedef fold< vec_of_vecs, vector<>, fold< _2, _1, *protect< lambda< push_back< _1, _2 > >::type
* // correct > ::type result;
Hi Nick. I copied your code (removing the * delimiters, which I guess you use just to emphasize the important parts), but couldn't get either to compile. The code I used is attached. Use the #define USE_LAM_PRO to select which version to compile. Could you modify the attached so that it does compile? I'm using gcc-4.7. The error messages I'm getting with gcc-4.7 include: /home/evansl/prog_dev/boost-svn/ro/trunk/boost/mpl/fold.hpp:39:18: required from 'struct boost::mpl::fold<boost::mpl::vector<boost::mpl::vector<char>, boost::mpl::vector<int> >, boost::mpl::vector<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::mpl::fold<mpl_::arg<2>, mpl_::arg<1>, boost::mpl::protect<boost::mpl::lambda<boost::mpl::push_back<mpl_::arg<1>, mpl_::arg<2> > > > > >' nested_placeholders.cpp:45:6: required from here /home/evansl/prog_dev/boost-svn/ro/trunk/boost/mpl/apply_wrap.hpp:141:1: error: no class template named 'apply' in 'struct boost::mpl::protect<boost::mpl::lambda<boost::mpl::push_back<mpl_::arg<1>, mpl_::arg<2> > > >' In file included from /home/evansl/prog_dev/boost-svn/ro/trunk/boost/preprocessor/iteration/detail/iter/forward1.hpp:52:0, from /home/evansl/prog_dev/boost-svn/ro/trunk/boost/mpl/aux_/fold_impl_body.hpp:67, from /home/evansl/prog_dev/boost-svn/ro/trunk/boost/mpl/aux_/fold_impl.hpp:40, from /home/evansl/prog_dev/boost-svn/ro/trunk/boost/mpl/fold.hpp:20, from nested_placeholders.cpp:11: /home/evansl/prog_dev/boost-svn/ro/trunk/boost/mpl/aux_/fold_impl_body.hpp: In instantiation of 'struct boost::mpl::aux::fold_impl<1, boost::mpl::v_iter<boost::mpl::vector<char>, 0l>, boost::mpl::v_iter<boost::mpl::vector<char>, 1l>, boost::mpl::vector<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::mpl::protect<boost::mpl::lambda<boost::mpl::push_back<mpl_::arg<1>, mpl_::arg<2> > > > >': /home/evansl/prog_dev/boost-svn/ro/trunk/boost/mpl/aux_/has_type.hpp:20:1: recursively required from 'const bool boost::mpl::aux::has_type<boost::mpl::fold<boost::mpl::vector<char>, boost::mpl::vector<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::mpl::protect<boost::mpl::lambda<boost::mpl::push_back<mpl_::arg<1>, mpl_::arg<2> > > > >, mpl_::bool_<true> >::value' -regards, Larry

Hi Nick.
I copied your code (removing the * delimiters, which I guess you use just to emphasize the important parts), but couldn't get either to compile. The code I used is attached. Use the #define USE_LAM_PRO to select which version to compile.
Could you modify the attached so that it does compile? I'm using gcc-4.7.
Hi Larry, Sorry about the extra formatting - in Gmail that's bolded, but I guess I should stick to plain text... I modified the code on ideone <https://ideone.com/MVYDj> and it compiles fine (gcc 4.3.4) - my guess is it won't give you any trouble. The main thing I changed was moving the ::type access from outside protect to inside (it should refer to the inner lambda), although I also added a couple includes for form and a main function so it would work online. -Nick Kitten Software Engineer Center for Video Understanding Excellence ObjectVideo, Inc. http://www.objectvideo.com

On 01/16/12 09:08, Kitten, Nicholas wrote: [snip]
I modified the code on ideone <https://ideone.com/MVYDj> and it compiles fine (gcc 4.3.4) - my guess is it won't give you any trouble. The main thing I changed was moving the ::type access from outside protect to inside (it should refer to the inner lambda), although I also added a couple includes for form and a main function so it would work online.
-Nick Kitten Thanks Nick. It compiles & runs OK now with my gcc4.7 when macro USE_LAM_PRO is #defined.
-Larry

Hi Nicholas, On Fri, 13 Jan 2012 19:44:07 -0600, Kitten, Nicholas <nkitten@objectvideo.com> wrote:
I've recently been doing a lot mpl::folding (in boost 1.47), and I was disappointed when I realized that a nested fold wouldn't work with a lambda expression passed to the inner call, as in this example which extracts elements of sequences:
using namespace mpl; using namespace mpl::placeholders;
typedef vector< vector<char>, vector<int> > vec_of_vecs; typedef vector< char, int > expected_result;
typedef fold< vec_of_vecs, vector<>, fold< _2, _1, *push_back< _1, _2 >* // wrong - refers to outer arguments >
::type result;
BOOST_MPL_ASSERT(( equal<result, expected_result> ));
After looking around for a solution I came across an old thread<http://lists.boost.org/boost-users/2004/12/9269.php> on the same subject, where it was suggested using mpl::protect might work. Indeed, the following modification does what I want, without any changes to the library:
typedef fold< vec_of_vecs, vector<>, fold< _2, _1, *protect< lambda< push_back< _1, _2 >
::type * // correct > ::type result;
Actually, you don't need 'protect' here, plain 'lambda< push_back< _1, _2
::type' would suffice, the resulting Metafunction Class is already shielded from being interpreted as a placeholder expression. Or you can just go with simple 'quote2<push_back>'.
Of course, this is a little on the verbose side. In the old thread, Daniel Wallin suggested suggested changing the definition of lambda to yield syntax more in line with the runtime version of boost::bind, but that would apparently break other code.
Yes, Daniel's suggestion affects not only protect's syntax, but also its semantics.
What about instead adding a specialization to protect for placeholder expressions? Indeed, the source in protect.hpp seems to indicate such a thing might've been in the works at one point,
Nope :), please see below.
but I didn't see any more references to it:
template< typename BOOST_MPL_AUX_NA_PARAM(T) * , int not_le_ = 0 // not lambda expression?* > struct protect : T { ... typedef protect type; };
That parameter's name is somewhat misleading; what it actually means is something like "this is an non-type template parameter that prevents 'protect' from being treated as a placeholder expression".
What about changing it to something like this?
template< typename BOOST_MPL_AUX_NA_PARAM(T) * , typename not_le_ = typename is_lambda_expression< T >::type* > struct protect : T { ... typedef protect type; };
*template< typename T >* *struct protect< T, mpl::true_ > : lambda<T>* *{* *...* *typedef protect type;* *};*
If we are try this, it would need to be more along the lines of template< typename BOOST_MPL_AUX_NA_PARAM(T) , bool not_le_ = is_lambda_expression<T>::value > struct protect : T { typedef protect type; }; template< typename T > struct protect<T,true> : lambda<T>::type { typedef protect type; }; .. but I'm pretty sure it's still going to break at least some code. Hmm, let me try this quick... yep, breaks the library itself; check out the following code in "equal.hpp": template< typename Predicate , typename LastIterator1 , typename LastIterator2 > struct equal_pred // Metafunction class! { template< typename Iterator2 , typename Iterator1 > struct apply { ... }; }; template< typename Sequence1 , typename Sequence2 , typename Predicate > struct equal_impl { typedef typename begin<Sequence1>::type first1_; typedef typename begin<Sequence2>::type first2_; typedef typename end<Sequence1>::type last1_; typedef typename end<Sequence2>::type last2_; typedef aux::iter_fold_if_impl< first1_ , first2_ , next<> , protect< aux::equal_pred<Predicate,last1_,last2_> > // <---- here , void_ , always<false_> > fold_; See how the suggested specialization changes the meaning of the highlighted line? For it to work with the new 'protect' definition, the line needs to be changed to , aux::equal_pred< protect<Predicate>,last1_,last2_> > and the 'equal_pred' rewritten to take this into account. Doable, but the change is definitely not backward compatible. The thing is, 'protect' was really conceived w/ bind and Metafunction classes in mind, and it actually works as you'd expect in that context (see Example in http://www.boost.org/doc/libs/1_48_0/libs/mpl/doc/refmanual/protect.html). It might be possible to retrofit it to lambda expressions, but it'll require quite a bit of work that I'm not sure is worth it. Personally, I'd rather work on a more general scoping mechanism along the lines of http://article.gmane.org/gmane.comp.lib.boost.devel/116000 HTH, -- Aleksey Gurtovoy MetaCommunications Engineering

On Mon, Jan 23, 2012 at 2:30 AM, Aleksey Gurtovoy <agurtovoy@meta-comm.com> wrote:
Hi Nicholas,
Hi Aleksey, <snip>
Actually, you don't need 'protect' here, plain 'lambda< push_back< _1, _2 > >::type' would suffice, the resulting Metafunction Class is already shielded from being interpreted as a placeholder expression. Or you can just go with simple 'quote2<push_back>'.
Both of those make sense, and that's certainly the kind of short addition I was hoping for; the quote2 version is especially nice, since I rarely swizzle the arguments anyway - thanks!
template< typename BOOST_MPL_AUX_NA_PARAM(T) , int not_le_ = 0 // not lambda expression?
> struct protect : T { ... typedef protect type; };
That parameter's name is somewhat misleading; what it actually means is something like "this is an non-type template parameter that prevents 'protect' from being treated as a placeholder expression".
I realized that was probably the case a while after posting, but it's good to have confirmation.
If we are try this, it would need to be more along the lines of
template< typename BOOST_MPL_AUX_NA_PARAM(T) , bool not_le_ = is_lambda_expression<T>::value > struct protect : T
{ typedef protect type; };
template< typename T > struct protect<T,true> : lambda<T>::type { typedef protect type; };
.. but I'm pretty sure it's still going to break at least some code. Hmm, let me try this quick... yep, breaks the library itself; check out the following code in "equal.hpp": <snip>
Well, even after you cleaned up my errors, I guess that was too hopeful...
Doable, but the change is definitely not backward compatible.
The thing is, 'protect' was really conceived w/ bind and Metafunction classes in mind, and it actually works as you'd expect in that context (see Example in http://www.boost.org/doc/libs/1_48_0/libs/mpl/doc/refmanual/protect.html). It might be possible to retrofit it to lambda expressions, but it'll require quite a bit of work that I'm not sure is worth it.
I think what gets me (and others before me) is the fact that MPL mirrors the runtime boost::lambda in so many ways, making it surprising when a function with the same name doesn't serve the same purpose (especially when it superficially appears to). However, for a library as old and fundamental as MPL, I agree that backwards compatibility is probably more important in this particular case. I think forcing instantiation of inner lambdas and/or the use of quoten are fine solutions for my use case - they're just subtle for neophytes like me in the land of metaprogramming, and might warrant a piece of documentation, just as both Phoenix <http://www.boost.org/doc/libs/1_48_0/libs/phoenix/doc/html/phoenix/modules/scope/lambda.html> and Lambda <http://www.boost.org/doc/libs/1_48_0/doc/html/lambda/le_in_details.html#lambda.nested_stl_algorithms> have sections on nesting.
Personally, I'd rather work on a more general scoping mechanism along the lines of http://article.gmane.org/gmane.comp.lib.boost.devel/116000
Alright, that seems reasonable, so I'll share my thoughts. Looking at the options, if you're concerned about backwards compatibility, then it seems like any kind of implicit local scoping (for library algorithms or lambda expressions) is out. That leaves explicit scopes used with either local variable declarations (Phoenix's let[] solution) or something similar to the outer() syntax you suggested:
// outer(arg1) template< int n > outer_argument<1,n> outer(arg<n>);
// outer(... outer(arg1)) template< int scope, int n > outer_argument<scope+1,n> outer(outer_argument<scope,n>);
// outer<n>(arg1) template< int scope, int n > outer_argument<scope,n> outer(arg<n>);
Okay, let's see what syntax might look like for both options. First, using Phoenix's simple use case: write a lambda expression that accepts: 1. a 2-dimensional container (e.g. vector<vector<int> >) 2. a container element (e.g. int) and pushes-back the element to each of the vector<int>. Here's my take on it: typedef vector< vector<char>, vector<char> > vec_of_vecs; typedef vector< vector<char,int>, vector<char,int> > expected_result; // scope and outer version typedef transform< _1, scope< push_back< _1, outer< _2 > > >
lambda_e1;
// let<> version, with let declared as: // template< LetExpression, typename a = _1, typename b = _2, ... > struct let; <--- LetExpression includes _a, _b, etc. typedef transform< _1, let< push_back< _1, _a >, _2 // <-- declares _a to be outer _2 >
lambda_e2;
BOOST_MPL_ASSERT(( equal< apply2< lambda_e1, vec_of_vecs, int
::type, expected_result, equal<_1, _2> > )); BOOST_MPL_ASSERT(( equal< apply2< lambda_e2, vec_of_vecs, int ::type, expected_result, equal<_1, _2> > ));
Here, let<>, like bind<>, reduces readability somewhat as you have to look after the expression for declarations, so I think the outer<> form would be the better of the two for MPL. Now, a more complex example: write a lambda expression that accepts 3 sequences and outputs a vector containing the Cartesian product of all 3, assuming non-empty vectors (similar to BOOST_PP_SEQ_FOR_EACH_PRODUCT <http://www.boost.org/doc/libs/1_47_0/libs/preprocessor/doc/ref/seq_for_each_product.html> ) typedef vector<bool, char> v1; typedef vector<short, int> v2; typedef vector<float> v3; typdef vector< vector< bool, short, float >, vector< bool, int, float >, vector< char, short, float >, vector< char, int, float >,
expected_result;
typedef fold< _1, // <--- scope 1 vector<>, scope< // <--- scope 2 fold< outer< _2 >, _1, scope< // <--- scope 3 fold< outer<2, _3 >, _1, scope< // <--- scope 4 push_back< _1, vector< outer<2, _2>, outer<_2>, _2 > > > > > > >
lambda_e1;
BOOST_MPL_ASSERT(( equal< apply3< lambda_e1, v1, v2, v3 >::type, expected_result, equal<_1, _2> > )); Not too bad, I suppose. Since scopes are explicit here, you wouldn't need a specialization for every algorithm, as you pointed out for implicit scopes, right?
It's interesting how compile-time and run-time lambdas differ here: in MPL, there is no way to automatically determine the implicit scope:
// sums a sequence of sequence of numbers typedef fold< _1, int_<0> , fold< _2, _1, plus<_1,_2> > // ^^^^^^^^^^ third scope, same question // ^^^^^^^^^^^^^^^^^^^^^^^^^^^ second scope, but how do we know?
f;
, at least without some additional metafunction's markup which would indicate that 'fold's third argument is a predicate, e.g.:
template< typename T1, typename T2, typename T3 > struct scope< fold<T1,T2,T3> > { typedef fold<T1,T2,scope<T3> > type; };
How does this sound?
HTH, -- Aleksey Gurtovoy MetaCommunications Engineering
You were indeed helpful :) Thanks, Nick Kitten Software Engineer Center for Video Understanding Excellence ObjectVideo, Inc. http://www.objectvideo.com

On Thu, 26 Jan 2012 09:09:36 -0600, Kitten, Nicholas <nkitten@objectvideo.com> wrote:
On Mon, Jan 23, 2012 at 2:30 AM, Aleksey Gurtovoy <agurtovoy@meta-comm.com> wrote:
Personally, I'd rather work on a more general scoping mechanism along the lines of http://article.gmane.org/gmane.comp.lib.boost.devel/116000
Alright, that seems reasonable, so I'll share my thoughts. Looking at the options, if you're concerned about backwards compatibility, then it seems like any kind of implicit local scoping (for library algorithms or lambda expressions) is out. That leaves explicit scopes used with either local variable declarations (Phoenix's let[] solution) or something similar to the outer() syntax you suggested:
// outer(arg1) template< int n > outer_argument<1,n> outer(arg<n>);
// outer(... outer(arg1)) template< int scope, int n > outer_argument<scope+1,n> outer(outer_argument<scope,n>);
// outer<n>(arg1) template< int scope, int n > outer_argument<scope,n> outer(arg<n>);
Okay, let's see what syntax might look like for both options. First, using Phoenix's simple use case:
write a lambda expression that accepts:
1. a 2-dimensional container (e.g. vector<vector<int> >) 2. a container element (e.g. int)
and pushes-back the element to each of the vector<int>.
Here's my take on it:
typedef vector< vector<char>, vector<char> > vec_of_vecs; typedef vector< vector<char,int>, vector<char,int> > expected_result;
// scope and outer version typedef transform< _1, scope< push_back< _1, outer< _2 > > >
lambda_e1;
// let<> version, with let declared as: // template< LetExpression, typename a = _1, typename b = _2, ... > struct let; <--- LetExpression includes _a, _b, etc. typedef transform< _1, let< push_back< _1, _a >, _2 // <-- declares _a to be outer _2 >
lambda_e2;
BOOST_MPL_ASSERT(( equal< apply2< lambda_e1, vec_of_vecs, int
::type, expected_result, equal<_1, _2> > )); BOOST_MPL_ASSERT(( equal< apply2< lambda_e2, vec_of_vecs, int ::type, expected_result, equal<_1, _2> > ));
Here, let<>, like bind<>, reduces readability somewhat as you have to look after the expression for declarations, so I think the outer<> form would be the better of the two for MPL.
That's my current preference as well. Now all we need is for somebody to implement this :). -- Aleksey Gurtovoy MetaCommunications Engineering
participants (3)
-
Aleksey Gurtovoy
-
Kitten, Nicholas
-
Larry Evans