
Eric, Several newbie questions here. I'm trying to understand how the proto::arg transform works. Looking at the Mixed example, I see this: template<typename Grammar> struct begin : Grammar { template<typename Expr, typename State, typename Visitor> struct apply : proto::terminal< iterator_wrapper< typename proto::result_of::arg<Expr>::type::const_iterator > > {}; template<typename Expr, typename State, typename Visitor> static typename apply<Expr, State, Visitor>::type call(Expr const &expr, State const &state, Visitor &visitor) { return proto::as_expr(cbegin(proto::arg(expr))); } }; Not knowing exactly what cbegin does, I think this makes sense to me. Given a grammar like this: // Here is the grammar for if_ statements // matches if_(e1)[e2] struct IfGrammar : boost::proto::subscript< boost::proto::unary_expr< tag::If, ExpressionGrammar >, StatementGrammar > {}; I want to create an AST node representing the statement. So I wrote a custom transform like this: // Transform a two-operand node template<typename Grammar, typename NodeType> struct ConstructBinaryLeftNested { : Grammar { template<typename Expr, typename State, typename Visitor> struct apply { typedef typename Ptr<NodeType>::type type; }; template<typename Expr, typename State, typename Visitor> static typename apply<Expr, State, Visitor>::type call(Expr const &expr, State const &state, Visitor &visitor) { return(new typename apply<Expr, State, Visitor>::type( Grammar::call(proto::arg(proto::left(expr)),state,visitor), Grammar::call(proto::right(expr),state,visitor))); } } } And use it like this: struct StatementGrammar : boost::proto::or< ConstructBinaryLeftNested<IfGrammar, IfStatement>, ... Essentially, I want to generate code like this: new IfStatement( transformed arg of unary_expr, // The condition transformed arg of subscript); // The statement block So several questions arise: - Is this use of arg/left allowed and correct? - If not, do I have to use identity instead to pass up the grandchild? - If not, what should I do? Something's telling me I'm not understanding something really fundamental. In particular, I don't understand the use of arg in the run-time code of Mixed vs. the use of arg as a compile-time transform. Some annotated examples of how proto transforms work with runtime code would be helpful. The calc transform examples are all pure compile-time code. Also, how do I get the arity of an expression at compile time? I need to be able to construct an AST node representing a function call which could have any number of arguments: struct CallGrammar : boost::proto::function<ExpressionGrammar, boost::proto::vararg<ExpressionGrammar> > {}; I need a transform that looks something like this: // Transform an n-ary node template<typename Grammar, typename NodeType> struct ConstructNary { : Grammar { template<typename Expr, typename State, typename Visitor> struct apply { typedef typename Ptr<NodeType>::type type; }; template<typename Expr, typename State, typename Visitor> static typename apply<Expr, State, Visitor>::type call(Expr const &expr, State const &state, Visitor &visitor) { func = new typename apply<Expr, State, Visitor>::type( // Function name Grammar::call(proto::arg<0>(expr),state,visitor)); // Generate func->addArgument(arg<i>(expr)) for each child ct_for<numargs(expr)-1>::exec(expr, func); return(func); } } } Does this make sense? Is it possible? Thanks for your help. -Dave

David A. Greene wrote:
Eric,
Several newbie questions here.
I'm trying to understand how the proto::arg transform works.
Looking at the Mixed example, I see this:
template<typename Grammar> struct begin : Grammar { template<typename Expr, typename State, typename Visitor> struct apply : proto::terminal< iterator_wrapper< typename proto::result_of::arg<Expr>::type::const_iterator > > {};
This says, reach into Expr and pull out the 0-th argument. If Expr is terminal<vector<T> >::type, then arg<Expr>::type::const_iterator is vector<T>::const_iterator. Then wrap it in an iterator_wrapper and put it back into a terminal node.
template<typename Expr, typename State, typename Visitor> static typename apply<Expr, State, Visitor>::type call(Expr const &expr, State const &state, Visitor &visitor) { return proto::as_expr(cbegin(proto::arg(expr))); } };
This is just the runtime equivalent.
Not knowing exactly what cbegin does, I think this makes sense to me.
cbegin() only ensures that the vector is const-qualified before calling begin() so we get a const_iterater back instead of an iterator.
Given a grammar like this:
// Here is the grammar for if_ statements // matches if_(e1)[e2] struct IfGrammar : boost::proto::subscript< boost::proto::unary_expr< tag::If, ExpressionGrammar >, StatementGrammar > {};
I want to create an AST node representing the statement. So I wrote a custom transform like this:
// Transform a two-operand node template<typename Grammar, typename NodeType> struct ConstructBinaryLeftNested { -----------------------------------------^ typo?
: Grammar { template<typename Expr, typename State, typename Visitor> struct apply { typedef typename Ptr<NodeType>::type type; };
Suspicious. Your return type doesn't seem to depend on Grammar, Expr, State or Visitor. Did you intend that? Transforms nest, so here, you'll typically want to be invoking the Grammar::apply<Expr,State,Visitor> metafunction and doing something with the result, but not always.
template<typename Expr, typename State, typename Visitor> static typename apply<Expr, State, Visitor>::type call(Expr const &expr, State const &state, Visitor &visitor) { return(new typename apply<Expr, State, Visitor>::type( Grammar::call(proto::arg(proto::left(expr)),state,visitor), Grammar::call(proto::right(expr),state,visitor)));
This is certainly not right. I don't do a good job explaining this yet in the docs. Let me explain ... When you write a transform like this ... template<typename Grammar> struct my_transform : Grammar { template<typename Expr, typename State, typename Visitor> struct apply { // do something with Expr here }; You can be secure in knowing that, whatever Expr may be, matches<Expr, Grammar> is true; that is, Expr matches Grammar. Call it the Transform Protocol. And the same is true for the call() static member function ... template<typename Expr, typename State, typename Visitor> static typename apply<Expr, State, Visitor>::type call(Expr const &expr, State const &state, Visitor &visitor) { // do something with expr here } }; As before, you can be sure that matches<Expr, Grammar> is true here. It's the Transform Protocol. So, how can I look at code like the following and know that it's wrong?
Grammar::call(proto::right(expr),state,visitor)
Because it violates the Transform Protocol. Grammar::call() invokes the transform associated with Grammar. expr matches Grammar, but right(expr) probably doesn't. To pass right(expr) to Grammar::call() would be to pass it an expression it doesn't know how to handle. A nasty compile error is sure to result. Perhaps what you mean to say is:
return(new typename apply<Expr, State, Visitor>::type( proto::arg(proto::left(Grammar::call(expr,state,visitor))), proto::right(Grammar::call(expr,state,visitor))));
But it's hard to know without seeing more of your code.
} } }
And use it like this:
struct StatementGrammar : boost::proto::or< ConstructBinaryLeftNested<IfGrammar, IfStatement>, ...
Essentially, I want to generate code like this:
new IfStatement( transformed arg of unary_expr, // The condition transformed arg of subscript); // The statement block
So several questions arise:
- Is this use of arg/left allowed and correct? - If not, do I have to use identity instead to pass up the grandchild? - If not, what should I do?
I don't have enough context to tell you what you need to do, but perhaps the above will get you going in the right direction.
Something's telling me I'm not understanding something really fundamental. In particular, I don't understand the use of arg in the run-time code of Mixed vs. the use of arg as a compile-time transform.
transform::arg is implemented in terms of proto::arg, and its implementation is simple and perhaps enlightening. Here is the implementation of transform::arg<Grammar,N>::call(expr,state,visitor) return proto::arg<N>(Grammar::call(expr, state, visitor)); It follows the pattern of invoking Grammar's transform, then manipulating the result. That is how the chaining of transforms works. Many of the transforms are just that simple. Once you really grok the Transform Protocol, it won't seem so mysterious. I hope. :-)
Some annotated examples of how proto transforms work with runtime code would be helpful. The calc transform examples are all pure compile-time code.
Also, how do I get the arity of an expression at compile time? I need to be able to construct an AST node representing a function call which could have any number of arguments:
struct CallGrammar : boost::proto::function<ExpressionGrammar, boost::proto::vararg<ExpressionGrammar> > {};
I need a transform that looks something like this:
// Transform an n-ary node template<typename Grammar, typename NodeType> struct ConstructNary { : Grammar { template<typename Expr, typename State, typename Visitor> struct apply { typedef typename Ptr<NodeType>::type type; };
template<typename Expr, typename State, typename Visitor> static typename apply<Expr, State, Visitor>::type call(Expr const &expr, State const &state, Visitor &visitor) { func = new typename apply<Expr, State, Visitor>::type( // Function name Grammar::call(proto::arg<0>(expr),state,visitor)); // Generate func->addArgument(arg<i>(expr)) for each child ct_for<numargs(expr)-1>::exec(expr, func); return(func); } } }
Does this make sense? Is it possible?
Yes, it's possible, but not documented anywhere. My fault. You can say Expr::proto_arity::value, which is a compile-time integral constant. Nicer would be a proto::arity<> metafunction. I'll add it. -- Eric Niebler Boost Consulting www.boost-consulting.com The Astoria Seminar ==> http://www.astoriaseminar.com

On Wednesday 11 July 2007 01:47, Eric Niebler wrote:
So, how can I look at code like the following and know that it's wrong?
Grammar::call(proto::right(expr),state,visitor)
Because it violates the Transform Protocol. Grammar::call() invokes the transform associated with Grammar. expr matches Grammar, but right(expr) probably doesn't. To pass right(expr) to Grammar::call() would be to pass it an expression it doesn't know how to handle. A nasty compile error is sure to result.
Ok, that makes sense, I think.
Perhaps what you mean to say is:
return(new typename apply<Expr, State, Visitor>::type( proto::arg(proto::left(Grammar::call(expr,state,visitor))), proto::right(Grammar::call(expr,state,visitor))));
But it's hard to know without seeing more of your code.
Ok, I think I see what you're getting at. Yes, I think this is what I want. I was getting confused about transform::call() vs. Grammar::call(), thinking that what you suggest here would be infinite recursion. But I see that's not the case. In essence the code here is "walking up" the inheritance tree from subclass (transform) to superclass (Grammar), yes? And Grammar::call will do the same with whatever it's inherited from. And by default built-in proto expressions do a pass-through. Because proto uses CRTP, "walking up" the inheritance tree is in effect "walking down" the proto expression tree, and since Grammar::call is always invoked first, transforms are built bottom-up. I hope I've got that right.
transform::arg is implemented in terms of proto::arg, and its implementation is simple and perhaps enlightening. Here is the implementation of transform::arg<Grammar,N>::call(expr,state,visitor)
return proto::arg<N>(Grammar::call(expr, state, visitor));
It follows the pattern of invoking Grammar's transform, then manipulating the result. That is how the chaining of transforms works.
Many of the transforms are just that simple. Once you really grok the Transform Protocol, it won't seem so mysterious. I hope. :-)
Yes, this explanation was extremely helpful. I'm not at the level of grok yet but I'm getting there. I had to stare at what you wrote here for a few minutes, but it makes good sense.
Does this make sense? Is it possible?
Yes, it's possible, but not documented anywhere. My fault. You can say Expr::proto_arity::value, which is a compile-time integral constant. Nicer would be a proto::arity<> metafunction. I'll add it.
Cool. -Dave

David A. Greene wrote:
Ok, I think I see what you're getting at. Yes, I think this is what I want. I was getting confused about transform::call() vs. Grammar::call(), thinking that what you suggest here would be infinite recursion. But I see that's not the case. In essence the code here is "walking up" the inheritance tree from subclass (transform) to superclass (Grammar), yes? And Grammar::call will do the same with whatever it's inherited from. And by default built-in proto expressions do a pass-through.
Yes, that's mostly correct. The types proto::shift_right and proto::unary_expr, for example, do a pass-through transform. Other types like proto::if_ and proto::expr have an identity transform. The docs describe the difference between these two, and which types have which transform.
Because proto uses CRTP, "walking up" the inheritance tree is in effect "walking down" the proto expression tree, and since Grammar::call is always invoked first, transforms are built bottom-up.
I hope I've got that right.
Well, you're getting warmer. :-) When you chain transforms by invoking Grammar::call(), the transforms are successively applied to the *same node* in the expression tree. Recursing into child nodes is the job of special transforms like pass-through (often the default) and fold. Here's how the pass-through transform works. Take a grammar like right_shift<A,B> and an expression like right_shift<X,Y>::type. (Assume that X matches A and Y matches B.) Applying the grammar's transform to the expression results in a new expression with this type: right_shift< A::apply<X,S,V>::type ,B::apply<Y,S,V>::type
::type
Here S and V are the types of the state and visitor parameters that just come along for the ride. So, calling Grammar::call() doesn't recurse to the children in general *unless* Grammar is one of these special transforms like pass-through. <snip>
Yes, it's possible, but not documented anywhere. My fault. You can say Expr::proto_arity::value, which is a compile-time integral constant. Nicer would be a proto::arity<> metafunction. I'll add it.
Cool.
I added it, and it's called proto::arity_of<>. -- Eric Niebler Boost Consulting www.boost-consulting.com The Astoria Seminar ==> http://www.astoriaseminar.com

On Saturday 14 July 2007 17:12, Eric Niebler wrote:
Yes, that's mostly correct. The types proto::shift_right and proto::unary_expr, for example, do a pass-through transform. Other types like proto::if_ and proto::expr have an identity transform. The docs describe the difference between these two, and which types have which transform.
Yep, I think I understand the difference.
Here S and V are the types of the state and visitor parameters that just come along for the ride. So, calling Grammar::call() doesn't recurse to the children in general *unless* Grammar is one of these special transforms like pass-through.
Yes, my (unstated) assumption was that Grammar was a pass-through transform. I'm about to tackle the n-ary case, which will be interesting. I'm sure I'll have more questions but armed with the help you've given, I've a much better shot now. Thanks. -Dave
participants (2)
-
David A. Greene
-
Eric Niebler