Proto transform : fold & fold_tree use case
Hello, I'm currently trying to write some transforms but I think I horribly fail because I must be missing an important point. Let's say we have a grammar defined as : struct skeleton : or_< terminal< bp::_ > , bitwise_and<skeleton,skeleton> > {}; What I need to do is build a type out of a given expression. This type is in fact a structure containing a tuple of class that represents MPI process. So my idea was to write transform that do the following : * for terminals, extract the argument (which is in fact a functor class) and create a process network class instanciation with the argument and a PID (basically a mpl::long_<>). * for the & operator, transform the left and right argument, and build the tuple out of both results by merging the contents of both into a unique process network. My biggest concern is to have the PID value to be kept across transform call and correctly increased for each terminal. To do so, am I forced to use fold (or fold_tree cause in the real code I have more than the sole & operator) or is there a simple way to say transform left and right and get the resulting types ? Then, how is State kept through fold_tree or transform iteration ? Should I kept the PID current value in an external parameters ? Thanks in advance. PS : if more detailed code is needed, I'll try to see what can be publicly dispatched here.
Joel Falcou wrote:
Hello,
I'm currently trying to write some transforms but I think I horribly fail because I must be missing an important point. Let's say we have a grammar defined as :
<snip>
PS : if more detailed code is needed, I'll try to see what can be publicly dispatched here.
Hi Joel, I would have an easier time understanding what you were trying to accomplish if you could give an example of an expression you'd like to transform and the desired result of the transform. Thanks, -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler a écrit :
Hi Joel, I would have an easier time understanding what you were trying to accomplish if you could give an example of an expression you'd like to transform and the desired result of the transform.
Yeah I kinda guessed after posting this. Here is a link to a paper presenting what I am currently porting to proto : http://www.ief.u-psud.fr/~falcou/pub/falcou-PARCO-2007.pdf Quaff is a parallel programming DSL that maps user defined functor class to a process network structure. Basically, take the principles of this paper and just say that instead of having pipe() and par() function, you have operator | and &. The old version of quaff ran on a hand-made expression template system that was - well - a piece of crap. Now, I want to use proto to leverage most of the ET + semantic code. I tried to do so in a straight forward way (aka making a apply_rule transform that mapped a proto expression contents to my semantic rule meta-function) but it failed cause when I was mixing pipe and par (ie | and &), the numbering of PID and insertion of elements went awkward. I also guess i don't have to use mpl::vector then build a boost::tupel out of them thanks to fusion, but considering my first failure, I stepped back. If more info than the paper is needed, I'll have to check with my co-worker to know which code is "safe" to publish here ;) Thanks for the help. -- Joel FALCOU Research Engineer @ Institut d'Electronique Fondamentale Université PARIS SUD XI France
Some more infos to get ideas straight. Given a pool of user defined functions or functor, I'll be able to write code like : run( seq<Foo>() | seq<Bar>() ); and have this code generate a parallel pipeline using MPI. To do so, I define a semantic rule for each parallel construction (called skeleton) that turns a sub-expression into a process network. A process_network is a static datatype defined by a triplet <P,I,O> where : - P : a list of processus - I : a list of input nodes - O : a list of outputs node All those 'list' are in fact mpl::vector of types OR mpl::long_<> process_network code is roughly this : template<class P,class I,class O> struct process_network { typedef P process; typedef I inputs; typedef O outputs; typedef typename bm::size<process>::type cardinal; static inline void run(); }; So P helds a list of process. A process is formally defined by : - the processus unique PID; the processus descriptor.</li> - a descriptor which is a list of "macro-instructions" for the processus to perform at runtime. template<class ID,class DESC> struct processus { typedef ID pid; typedef DESC descriptor; typedef processus<ID,DESC> type; typedef typename descriptor::type code; static inline void run(); }; Finally, the descriptor is formally defined by : - a lis of PID of preceding processus - a list of PID of following processus - a list of macro-instructions template<class IPID, class OPID,class CODE> struct descriptor { typedef IPID i_pids; typedef OPID o_pids; typedef CODE macro; }; Supported skeletons are : pipeline (operator|) parallel execution (operator&) farming which si tied to a special function called farm adn used like : farm<N>( some skeleton expression ) All of these have to be built from the expression above. To do so I have a set of rules. A rules take 2 process network and do stuff with its processus liss. For example, pipeline two process network concatenates the list of processus, add a Send macro instructions to all 'outputs' processus and a Receive macro instructions to all 'inputs' processus. Finally, the global out/input of the new network is set accordingly. Currently, I do the following (and yes it is ugly) : template<class X, class ID> struct apply_rule< bp::expr<tag::bitwise_or,X,2>, ID > { typedef expr<tag::bitwise_and,X,2> base; typedef typename result_of::arg_c<base, 0>::type::expr arg1; typedef typename result_of::arg_c<base, 1>::type::expr arg2; typedef typename apply_rule<arg1,ID>::type pn1; typedef typename apply_rule<arg1,ID>::pid next_pid; typedef typename apply_rule<arg2,next_pid>::type pn2; typedef typename apply_rule<arg2,next_pid>::pid pid; typedef typename rule_seq<pn1, pn2>::type type; }; basically, i extarct the arguments of a proto expression, evaluates each child of the expression to get their respective process_network,retrieves the next available PID and apply the corresponding rule to those intermediate result. Doing so works well for all my rules even in complex cases. When using proto transform and fold_tree, it doesn't. In fact, it even looks like the next PID and temporary network are badly computed, thus leading me to thinks that I badly use state and such. -- Joel FALCOU Research Engineer @ Institut d'Electronique Fondamentale Université PARIS SUD XI France
Joel FALCOU wrote:
Some more infos to get ideas straight. <snip>
Doing so works well for all my rules even in complex cases. When using proto transform and fold_tree, it doesn't. In fact, it even looks like the next PID and temporary network are badly computed, thus leading me to thinks that I badly use state and such.
I'm starting to get the idea, but I'm a little dense. And you've only shown a fragment of the kind of transformation you're doing. Since you have (ugly) code that already does what you want to do, can you post it so I can clearly see what you're doing? I can try to find a cleaner solution for you. -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler a écrit :
I'm starting to get the idea, but I'm a little dense. And you've only shown a fragment of the kind of transformation you're doing. Since you have (ugly) code that already does what you want to do, can you post it so I can clearly see what you're doing? I can try to find a cleaner solution for you.
Here is two zip files of the current state of work. Should compile with boost 1.35 + proto. If MPI inclusion is a problem just remove them altogetehr, they're not compelte yet. http://www.ief.u-psud.fr/~falcou/cpp/niebler/nice_not_working.zip http://www.ief.u-psud.fr/~falcou/cpp/niebler/ugly_working.zip Tell me when they're downloaded so i can remove them ;) First is what I tried to do using transform but failed to work properly. Second one works as intended by my semantic but is "ugly" (aka explicit use of inner form of proto:::expr). The use-case that fails is the followiong : cout << probe( (seq<Foo>() & seq<Foo>()) | (seq<Foo>() & seq<Foo>())); it supposed to write something like [0] : Function call send to 2 [1] : Function call send to 3 [2] : Recv from 0 Function call [3] : Recv from 1 Function call But in fact it outputs : [0] : Function call [1] : Function call [0] : Function call send to 2 [1] : Function call send to 3 [2] : Recv from 0 Function call [3] : Recv from 1 Function call
participants (3)
-
Eric Niebler
-
Joel Falcou
-
Joel FALCOU