[proto] I'm trying to write a vector class

Eric, I spent a few hours the other week trying to convert my last matrix-vector class library over to proto. It implemented expression trees, but I’d never gotten around to adding some features that proto should make much, much easier to implement such as expression optimization. I figured it would be a good way to experiment with the library to review. I eventually gave up on refactoring my class towards proto and instead tried refactoring your lazy_vector example towards my class. It's raised some questions/points. I'll illustrate with questions that are based on your lazy_vector example. 1) lazy_vector seemed like it could derive from lazy_vector_expr in CRTP fashion. I figured your use of std::vector<T> as the terminal type was just to make initialization easier; however, using CRTP fails to compile. Did I miss this in the documentation or is it not explicitly mentioned? 2) How can I handle scalar values transparently? My attempts to accept "v1=v2*5" have failed. I can't get the 5 to implicitly construct into a scalar_vector type I have defined. I had that working in my old one, but I can't seem to figure out the correct way of setting that up in proto. 3) My vector class' signature is very simple: template<typename T> class Vector : VectorExpr<terminal<T> > . The type T contains a number of traits defining the vector: the type of the elements, the dimension, how the data is stored. I can't figure out how to specify the terminals in my grammar. In your example, you use terminal<std::vector<_> > as your terminal. The std::vector part is unknown to me. That's my point of variation in my vector concept thus I can't know all the different types that could be used in the terminal. I tried deriving my storage classes from a common base class; however, that failed. Is there any way to encode some sort of restriction in the grammar for this situation? Right now, I'm using _, but that's clearly not a great solution. Proto looks very interesting. I was quite excited the first I saw it mentioned. There seems to be a lot of power there. I just need to figure out how to use it. You have a lot of documentation, but it seems to be attacking the problem from the opposite direction I'm approaching from, so it's been a little slow going. I hate asking questions about something I can investigate on my own, but I'd like to get far enough along that I can properly review it. Thanks, Noah No virus found in this outgoing message. Checked by AVG. Version: 7.5.519 / Virus Database: 269.22.0/1342 - Release Date: 3/25/2008 10:26 AM

Noah Stein wrote:
Eric,
I spent a few hours the other week trying to convert my last matrix-vector class library over to proto. It implemented expression trees, but I’d never gotten around to adding some features that proto should make much, much easier to implement such as expression optimization. I figured it would be a good way to experiment with the library to review. I eventually gave up on refactoring my class towards proto and instead tried refactoring your lazy_vector example towards my class. It's raised some questions/points. I'll illustrate with questions that are based on your lazy_vector example.
1) lazy_vector seemed like it could derive from lazy_vector_expr in CRTP fashion. I figured your use of std::vector<T> as the terminal type was just to make initialization easier;
??? It's a lazy_vector. My use of std::vector<> has nothing to do with initialization. It's to hold a vector of elements.
however, using CRTP fails to compile. Did I miss this in the documentation or is it not explicitly mentioned?
What failed to compile? Can you show me the code?
2) How can I handle scalar values transparently? My attempts to accept "v1=v2*5" have failed. I can't get the 5 to implicitly construct into a scalar_vector type I have defined. I had that working in my old one, but I can't seem to figure out the correct way of setting that up in proto.
In proto, literals like 5 are converted to terminal<int>::type by default. It sounds like what you want is for int terminals to be treated as a vector of scalars. There are a couple of ways to approach this. 1) At some point, you'll need to evaluate the expression, perhaps with proto::eval() and a custom context class. In your context, you can add a handler for int terminals and treat them like scalar vectors. This is what I would do. 2) Before you evaluate an expression that may contain int terminals, you can apply a proto transform that replaces all int terminals with scalar vectors. This is probably not the ideal solution for you, unless you need to do other transforms, too (like expression optimization). 3) You can cause int terminals to be converted into scalar vectors on expression construction by using a custom generator. Think of a generator as a very simple transform that is applied to every new expression that proto creates for you. In the lazy_vector example, the lazy_vector_domain is defined like this: // Tell proto that in the lazy_vector_domain, all // expressions should be wrapped in laxy_vector_expr<> struct lazy_vector_domain : proto::domain< proto::generator<lazy_vector_expr> , LazyVectorGrammar > {}; proto::generator<> is a simple generator that takes any expression type and wraps it in lazy_vector_expr<>. You could write one that first turns scalar terminals into scalar vectors, and then wraps them in lazy_vector_expr<>. (3) is workable, but I would discourage it (see below). (2) is the Proto Way (tm), but might be tricky for someone who doesn't yet know transforms. (1) is also the Proto Way (tm) and might be the simplest thing to do.
3) My vector class' signature is very simple: template<typename T> class Vector : VectorExpr<terminal<T> > .
You probably mean VectorExpr<terminal<T>::type>, right?
The type T contains a number of traits defining the vector: the type of the elements, the dimension, how the data is stored. I can't figure out how to specify the terminals in my grammar. In your example, you use terminal<std::vector<_> > as your terminal. The std::vector part is unknown to me. That's my point of variation in my vector concept thus I can't know all the different types that could be used in the terminal. I tried deriving my storage classes from a common base class; however, that failed. Is there any way to encode some sort of restriction in the grammar for this situation? Right now, I'm using _, but that's clearly not a great solution.
If you have derived your storage classes from a common base, then try this to match your terminals: proto::and_< proto::terminal<_> , proto::if_< is_base_and_derived< common_base, _arg >() >
Proto looks very interesting. I was quite excited the first I saw it mentioned. There seems to be a lot of power there. I just need to figure out how to use it. You have a lot of documentation, but it seems to be attacking the problem from the opposite direction I'm approaching from, so it's been a little slow going. I hate asking questions about something I can investigate on my own, but I'd like to get far enough along that I can properly review it.
Yes, Proto comes at the problem differently than other expression template libraries, because it's not really an expression template library. It's a library for building domain-specific languages. In Proto, the expression template itself, and the operator overloads that build it, are rather incidental. The focus of Proto is what you *do* with the expression once you have it. So rather than asking, "How can I get Proto to build the type I want," you might ask, "How can I get the behavior I want from the types Proto builds." The answer for that is usually either: (a) write a context, or (b) write a transform. While you can certainly get Proto to build you the type that you want, that's not Proto's natural mode of operation. By default, Proto builds an abstract tree representing the expression, with little or no domain-specific information in it. All the domain-specific knowledge is centralized in your domain's contexts or transforms. The reason I did it this way was to facilitate cross-domain interoperability. The same expression can mean one thing in one domain and something else in a different one. That allows expressions from different domains to be intermingled freely, as with semantic actions (lambda domain) in a grammar (parser domain). Or consider the placeholder terminal _1. You would like that to mean different things to bind, lambda, spirit, karma, phoenix, xpressive, etc. If it were a proto terminal, all these libraries could use the same terminal -- it's just syntax -- and the semantics could vary by library. I hope this gets you a little farther along. -- Eric Niebler Boost Consulting www.boost-consulting.com

I was worried that I wouldn't properly express the issues. At least I know I'm an accurate judge of myself. :-) I had hoped to spend some time yesterday working with proto, but I have a deadline Friday. I'm still hoping to get in a few hours tonight, but I wanted to at least post a little follow up even if I can't respond with something more in-depth. I've already written this reply twice to try to ensure that I'm clearly communicating.
1) lazy_vector seemed like it could derive from lazy_vector_expr in CRTP fashion. I figured your use of std::vector<T> as the terminal type was just to make initialization easier;
??? It's a lazy_vector. My use of std::vector<> has nothing to do with initialization. It's to hold a vector of elements.
however, using CRTP fails to compile. Did I miss this in the documentation or is it not explicitly mentioned?
What failed to compile? Can you show me the code?
In your lazy_vector example, you have the following line: V1 += v2 - v3; When I see that line, from the view of the types involved I think, "lazy_vector plus-equals lazy_vector minus lazy_vector". And that's correct. Thus in the expression, I believe that the terminals in this expression are lazy_vector. That's wrong. The terminals in this case are of type std::vector<double>. My first stab at forming a vector type was, roughly: template<typename T> class vector : vector_expr< typename proto::terminal< vector<T> >::type > This fails to compile. So it would appear that the terminals of my expressions cannot be terminals in expressions (so much for clarity in this post!). Thus I need to split my class into two parts: one that stores the data and one that defines the DSEL. I guess my lack of understanding in this case boils down to the question: Why isn't a lazy_vector a terminal in expressions composed of lazy_vectors? I don't see the difference conceptually.
While you can certainly get Proto to build you the type that you want, that's not Proto's natural mode of operation. By default, Proto builds an abstract tree representing the expression, with little or no domain-specific information in it. All the domain-specific knowledge is centralized in your domain's contexts or transforms. The reason I did it this way was to facilitate cross-domain interoperability. The same expression can mean one thing in one domain and something else in a different one. That allows expressions from different domains to be intermingled freely, as with semantic actions (lambda domain) in a grammar (parser domain). Or consider the placeholder terminal _1. You would like that to mean different things to bind, lambda, spirit, karma, phoenix, xpressive, etc. If it were a proto terminal, all these libraries could use the same terminal -- it's just syntax -- and the semantics could vary by library.
I definitely see the potential for handling so many domains. After iterating a few times to figure out how to handle expression trees properly for a single domain, I definitely saw the value in a general-purpose system. It's a gigantic step from a single custom-written system to a general system. I'm very excited to get to the stage where I can play around with transforms as expression optimization was something I wanted to play with before, but I just hadn't gotten around to it yet. The vector class was what I figured would be the gentlest introduction to your library, seeing as I've already written an expression tree handler for it. There are a lot of uses for proto that I see. I'd like to try adapting phoenix to create a library that implements co-routines. There are some interesting possibilities for state machines. I just need more time! Thanks, Noah No virus found in this outgoing message. Checked by AVG. Version: 7.5.519 / Virus Database: 269.22.1/1346 - Release Date: 3/27/2008 10:03 AM

Noah Stein wrote:
My first stab at forming a vector type was, roughly:
template<typename T> class vector : vector_expr< typename proto::terminal< vector<T> >::type >
This fails to compile.
Ah, I see now.
So it would appear that the terminals of my expressions cannot be terminals in expressions (so much for clarity in this post!). Thus I need to split my class into two parts: one that stores the data and one that defines the DSEL. I guess my lack of understanding in this case boils down to the question: Why isn't a lazy_vector a terminal in expressions composed of lazy_vectors? I don't see the difference conceptually.
Well, first let me address the compile error by analyzing what the above code means, piece by piece: typename proto::terminal< vector<T> >::type This is a proto terminal that holds a vector<T> by value. That is, inside this terminal there is a vector<T> object. vector_expr< typename proto::terminal< vector<T> >::type > As defined previously, vector_expr<E> "extends" the expression E. It behaves like an E and, as far as Proto is concerned, it *is* an E. vector_expr<E> holds an E object by value. template<typename T> class vector : vector_expr< typename proto::terminal< vector<T> >::type > This says that vector<T> is ... something that holds a vector<T> by value. That's clearly not going to work. The larger question is, should it be possible to define a type that is a proto terminal, but that doesn't extend some other proto terminal? That would be a good thing because it wouldn't force you to separate your data from your user-visible types. Another way to say this is, can I non-intrusively make a type a proto terminal? The answer is yes. There is an (undocumented) way to non-intrusively make non-proto types behave like proto terminals. Consider: #include <boost/proto/proto.hpp> using namespace boost; using namespace proto; template<typename T> struct vector { // vector impl here }; template<typename T> struct is_vector : mpl::false_ {}; template<typename T> struct is_vector<vector<T> > : mpl::true_ {}; BOOST_PROTO_DEFINE_OPERATORS(is_vector, default_domain) int main() { vector<int> vi; // OK, vi is a proto terminal! // This builds a proto expression tree: vi + 32; } The operator overloads that Proto defines requires at least one operand to be either expr<> or some type that actually extends expr<>. If neither is true for your terminal, as is the case with vector<T> above, you'll need to define your own operator overloads (with BOOST_PROTO_DEFINE_OPERATORS) and provide a Boolean metafunction that can be used to contrain the overloads (e.g., is_vector<>).
I definitely see the potential for handling so many domains. After iterating a few times to figure out how to handle expression trees properly for a single domain, I definitely saw the value in a general-purpose system. It's a gigantic step from a single custom-written system to a general system. I'm very excited to get to the stage where I can play around with transforms as expression optimization was something I wanted to play with before, but I just hadn't gotten around to it yet.
The vector class was what I figured would be the gentlest introduction to your library, seeing as I've already written an expression tree handler for it.
Right, and it turned out to be not-so-gentle because Proto does things differently.
There are a lot of uses for proto that I see. I'd like to try adapting phoenix to create a library that implements co-routines. There are some interesting possibilities for state machines. I just need more time!
Me too. -- Eric Niebler Boost Consulting www.boost-consulting.com
participants (2)
-
Eric Niebler
-
Noah Stein