"Joel de Guzman" <joel@boost-consulting.com> wrote in message news:ef1t83$bhv$1@sea.gmane.org...

Andy Little wrote:

...

"David Abrahams" <dave@boost-consulting.com> wrote in message news:87fyenkjh9.fsf@pereiro.luannocracy.com...

...

AFAICT the name ftag should be changed to something more descriptive and more certainly unique, e.g. boost_fusion_iterator_tag. Is there a reason it needs to be so short?

Why make such a trivial change to the interface post review? AFAICS now is too late, and will cause unnecessary pain to users.. like myself.

Well, actually, it shouldn't be a part of the interface. Here, I'm used to calling it the f#%$tag :) It's not supposed to be for public consumption, and I intend to enforce that. Either way, it's not a good name (especially for minors. ;) and it must be changed.

Well... what about mpl integral_c::tag then? ought to be changed to something more descriptive ASAP IMO. regards Andy Little

Andy Little wrote:

BTW Fusion ROCKS!!!!!

Gee, thanks! Comments like that makes those long sleepless coding hours a lot more enjoyable :) Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

"Joel de Guzman" <joel@boost-consulting.com> wrote in message news:ef1t83$bhv$1@sea.gmane.org...

Andy Little wrote:

...

"David Abrahams" <dave@boost-consulting.com> wrote in message news:87fyenkjh9.fsf@pereiro.luannocracy.com...

...

AFAICT the name ftag should be changed to something more descriptive and more certainly unique, e.g. boost_fusion_iterator_tag. Is there a reason it needs to be so short?

Why make such a trivial change to the interface post review? AFAICS now is too late, and will cause unnecessary pain to users.. like myself.

Well, actually, it shouldn't be a part of the interface. Here, I'm used to calling it the f#%$tag :) It's not supposed to be for public consumption, and I intend to enforce that. Either way, it's not a good name (especially for minors. ;) and it must be changed.

And BTW its pretty public in the 'make your own iterator" part of the docs. So far I have made 5 fusion style iterators. Next is the matrix minor iterator, which extracts minors from a matrix, as a prelude to extracting cofactors. Fusion::Views are wonderful things .... regards Andy little

"Joel de Guzman" <joel@boost-consulting.com> wrote in message news:45148B4C.3030804@boost-consulting.com...

Andy Little wrote:

...

"Joel de Guzman" <joel@boost-consulting.com> wrote in message news:ef1t83$bhv$1@sea.gmane.org...

...

Andy Little wrote:

...

"David Abrahams" <dave@boost-consulting.com> wrote in message news:87fyenkjh9.fsf@pereiro.luannocracy.com...

...

AFAICT the name ftag should be changed to something more descriptive and more certainly unique, e.g. boost_fusion_iterator_tag. Is there a reason it needs to be so short? Why make such a trivial change to the interface post review? AFAICS now is too late, and will cause unnecessary pain to users.. like myself. Well, actually, it shouldn't be a part of the interface. Here, I'm used to calling it the f#%$tag :) It's not supposed to be for public consumption, and I intend to enforce that. Either way, it's not a good name (especially for minors. ;) and it must be changed.

And BTW its pretty public in the 'make your own iterator" part of the docs.

Yeah. That's unfortunate. I asked Dan to fix it. Anyway, I suggest you do the right thing: specialize tag_of.

...

So far I have made 5 fusion style iterators. Next is the matrix minor iterator, which extracts minors from a matrix, as a prelude to extracting cofactors.

Cool!

...

Fusion::Views are wonderful things ....

Views are wonderful things. If you remember, I mentioned this idea about a const string library using Fusion like views. I'm betting such a library will rock in terms of performance.

hmmm.. Fusion is a bit wasted on strings ( I only need them for output, what is slow as a dog anyway) AFAICS. Seems to be "quite" good at inlining generic matrix concatenation though ... Actually I can't see how I would speed the output of my current efforts, once optimised by VC8 express, even if I coded each instantiation by hand :-) There do seem to be limits to when the compiler will optimise a view though, It may have something to do with copying references(aka joint-view) as raw bytes, after which AFAICS the compiler will treat them as pointers. that said that is only an impression so far. regards Andy Little regards Andy Little

Andy Little wrote:

There do seem to be limits to when the compiler will optimise a view though, It may have something to do with copying references(aka joint-view) as raw bytes, after which AFAICS the compiler will treat them as pointers. that said that is only an impression so far.

Ok, now I understand. You sent me an email about stacked fold of stacked joint_view created by push_back being difficult for the compiler to optimize. Here's why (and there's a solution): joint_view is a prime example of a segmented sequence. The resulting iterators from it will have some overhead. That is inevitable. See joint_view_iterator so you'll know what I mean. However, Eric Niebler did some work on segmented algorithms. His initial testing (for proto and early spirit-2 experiments) shows some exciting results. Basically, a segmented sequence is a sequence of sequences (for each segment). Instead of creating an iterator that "weaves" through each segment (like joint_view_iterator), segmented algorithms apply the algorithm recursively to each of the segments. For example, a fold of a joint_view will be 2 calls to fold, one for the left join and one for the right. I know this is a bit hand-wavy, but once you see it in action, it really makes a lot of sense. I'm CCing Eric. I think now is the right time to incorporate his work into the fusion code base. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

"Joel de Guzman" <joel@boost-consulting.com> wrote in message news:4517F285.2010806@boost-consulting.com...

Andy Little wrote:

...

There do seem to be limits to when the compiler will optimise a view though, It may have something to do with copying references(aka joint-view) as raw bytes, after which AFAICS the compiler will treat them as pointers. that said that is only an impression so far.

Ok, now I understand. You sent me an email about stacked fold of stacked joint_view created by push_back being difficult for the compiler to optimize. Here's why (and there's a solution):

joint_view is a prime example of a segmented sequence. The resulting iterators from it will have some overhead. That is inevitable. See joint_view_iterator so you'll know what I mean.

However, Eric Niebler did some work on segmented algorithms. His initial testing (for proto and early spirit-2 experiments) shows some exciting results. Basically, a segmented sequence is a sequence of sequences (for each segment). Instead of creating an iterator that "weaves" through each segment (like joint_view_iterator), segmented algorithms apply the algorithm recursively to each of the segments. For example, a fold of a joint_view will be 2 calls to fold, one for the left join and one for the right.

I know this is a bit hand-wavy, but once you see it in action, it really makes a lot of sense. I'm CCing Eric. I think now is the right time to incorporate his work into the fusion code base.

Yep. I don't understand,but it sounds great :-) OK. FWIW I found that creating a sequence by pushing_back result_types to an empty vector is a great way to 'create' a result sequence at compile time. It kind of creates something out of nothing. At runtime however I found I could employ a different approach by using recursive functors(that is still compile time rather than runtime recursion, but much faster to compile and using less resources, whereas joint_view can be a resource hog), because now the type of the result sequence is known. Basically the compile time and runtime stages are doing different jobs. The compile time fold push_back is working out what the final sequence will be. Now the runtime's job is to figure out the most efficient way to initialise the sequence, that it already knows about. I havent yet figured how to create an initialiser so that the compiler will initialise the sequence right in the constructor(not that its impossible, just havent got round to trying), but I guess that would be ideal. The compiler is seeming to optimise away the default ctor, followed by the fill anyway, but it would be satisfying if I could express that intent more directly. I guess this might involve making a view, where each element contains an individual view of the function for initialising that element. Anyway it might be worth thinking about not copying too slavishly the compile time behaviour at runtime, as maybe these are very different situations. Second my analysis of copying is based on not too much science. IN VC8 the assembler code gets pretty big and later functions tend to join the output of earlier functions and with 20 Mb assembler files I tended to have ideas half way through reading it and never get to the end of the story to see what the compiler finally did! If that makes any sense. However it may well be that the compiler does keep track of the stuff its copying. It just gets a bit frightening seeing rep movsd instructions on bigger and bigger joint_views (where these might have typically 16 elemnts each with 2 referneces), with bigger and bigger arguments, but OTOH I suspect once it has to copy then the game is up. The other question is whether Fusion scales. My guess is that the compile time should flatten out once the compiler starts seeing the same types, but OTOH this business of accumulating the result type may be 'interesting', but it requires writing some applications to find out, so that is what I'm currently about. regards Andy Little

Joel de Guzman wrote:

Andy Little wrote:

...

OK. FWIW I found that creating a sequence by pushing_back result_types to an empty vector is a great way to 'create' a result sequence at compile time. It kind of creates something out of nothing.

Oh, and BTW, have you considered using fusion::cons? If you are doing lots of sequence building, then cons is a very viable option. For example, with expression templates, cons can build the ET tree more efficiently than push_back/vector. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

"Joel de Guzman" <joel@boost-consulting.com> wrote in message news:ef9mui$nhb$1@sea.gmane.org...

Andy Little wrote:

...

OK. FWIW I found that creating a sequence by pushing_back result_types to an empty vector is a great way to 'create' a result sequence at compile time. It kind of creates something out of nothing. At runtime however I found I could employ a different approach by using recursive functors(that is still compile time rather than runtime recursion, but much faster to compile and using less resources, whereas joint_view can be a resource hog)

A resource hog? At compile time? At runtime? I'd like to know how you arrived at that observation.

FWIW the whole story is available in Quan CVS In the same directory with the following files, (though much of it is tedious to wade through) Here is v5: http://tinyurl.com/g3xlf In that I basically follow at runtime what I did at compile time with Fusion.

From memory it Blew VC7.1 to 'compiler out of resources" in debug mode.

In V6 I figured I'd try a different apprioach: http://tinyurl.com/l5q6t Here I use my own homerolled recursion at runtime and compiled much faster and completed and ran. Final version is not much different, but did some experiments on the evaluation order in homerolled version, which makes a huge difference toi the output. http://tinyurl.com/gkbbt But yeah. Don't take it too seriously. To analyse it properly would take a long time and I was having too much "fun with Fusion" at that time. :-) regards Andy Little You can use result_of::push_back to

build your sequence at compile time and that (AFAICT) is quite cheap.

, because now the type of

...

the result sequence is known. Basically the compile time and runtime stages are doing different jobs. The compile time fold push_back is working out what the final sequence will be. Now the runtime's job is to figure out the most efficient way to initialise the sequence, that it already knows about. I havent yet figured how to create an initialiser so that the compiler will initialise the sequence right in the constructor(not that its impossible, just havent got round to trying), but I guess that would be ideal. The compiler is seeming to optimise away the default ctor, followed by the fill anyway, but it would be satisfying if I could express that intent more directly. I guess this might involve making a view, where each element contains an individual view of the function for initialising that element.

If it helps, you can also use mpl to build your type. Fusion can take in MPL sequences. Example:

as_vector(an_mpl_sequence)

...

Anyway it might be worth thinking about not copying too slavishly the compile time behaviour at runtime, as maybe these are very different situations.

You mean: fusion should not copy the compile time behavior at runtime? Hmmm... but of course it does not. Let me get this straight: push_back *IS* cheap. It is the unfolding (iterating over) the push-backed view that is less than ideal. This can be made efficient by segmented techniques.

...

Second my analysis of copying is based on not too much science. IN VC8 the assembler code gets pretty big and later functions tend to join the output of earlier functions and with 20 Mb assembler files I tended to have ideas half way through reading it and never get to the end of the story to see what the compiler finally did! If that makes any sense. However it may well be that the compiler does keep track of the stuff its copying. It just gets a bit frightening seeing rep movsd instructions on bigger and bigger joint_views (where these might have typically 16 elemnts each with 2 referneces), with bigger and bigger arguments, but OTOH I suspect once it has to copy then the game is up.

The other question is whether Fusion scales. My guess is that the compile time should flatten out once the compiler starts seeing the same types, but OTOH this business of accumulating the result type may be 'interesting', but it requires writing some applications to find out, so that is what I'm currently about.

Ok.

Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Andy Little wrote:

"Joel de Guzman" <joel@boost-consulting.com> wrote in message news:ef9mui$nhb$1@sea.gmane.org...

...

Andy Little wrote:

...

OK. FWIW I found that creating a sequence by pushing_back result_types to an empty vector is a great way to 'create' a result sequence at compile time. It kind of creates something out of nothing. At runtime however I found I could employ a different approach by using recursive functors(that is still compile time rather than runtime recursion, but much faster to compile and using less resources, whereas joint_view can be a resource hog) A resource hog? At compile time? At runtime? I'd like to know how you arrived at that observation.

FWIW the whole story is available in Quan CVS In the same directory with the following files, (though much of it is tedious to wade through)

[snips]

But yeah. Don't take it too seriously. To analyse it properly would take a long time and I was having too much "fun with Fusion" at that time. :-)

I see. It seems you'll indeed have a perf-boost when we get the segmented stuff in! :) Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

"Andy Little" <andy@servocomm.freeserve.co.uk> writes:

...

I know this is a bit hand-wavy, but once you see it in action, it really makes a lot of sense. I'm CCing Eric. I think now is the right time to incorporate his work into the fusion code base.

Yep. I don't understand,but it sounds great :-)

Read http://lafstern.org/matt/segmented.pdf which does the same for STL iterators, and you will understand. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:

Joel de Guzman <joel@boost-consulting.com> writes:

...

joint_view is a prime example of a segmented sequence. The resulting iterators from it will have some overhead. That is inevitable. See joint_view_iterator so you'll know what I mean.

I don't get it. Fusion iterators know their position at compile-time. Why should there be any overhead?

joint_view_iterator needs to store these info: a) an iterator from the first sequence (left) b) an iterator from the second sequence (right) then, if you have a stacked joint_view_iterators, the outermost joint_view_iterator will be quite hefty. I wonder if there's a better formulation other than segmented iterators. Hmmm... yeah, there might be room for optimization. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

"David Abrahams" <dave@boost-consulting.com> wrote in message news:87bqp3nt9k.fsf@pereiro.luannocracy.com...

Joel de Guzman <joel@boost-consulting.com> writes:

...

joint_view is a prime example of a segmented sequence. The resulting iterators from it will have some overhead. That is inevitable. See joint_view_iterator so you'll know what I mean.

I don't get it. Fusion iterators know their position at compile-time. Why should there be any overhead?

FWIW The various sequences and iterators are here. This is all just a first hit. I am sure that there is much room for improvement. However currently I want to move on to other things, but I have done enough here ( and in the links to examples previous in the thread) to prove that a fusion matrix would be a very worthy addition to Fusion. I reckon with the use of static_values that it would be the fastest thing around. And BTW sorry to be rude about MTL4 but unfortunately I can't see that my comments are wrong, and if all just wait for the rollout then nothing is going to happen, is it? The matrix definition: http://tinyurl.com/kwfvd The matrix row and columns , sequences of and iterators: http://quan.cvs.sourceforge.net/quan/quan-trunk/quan/fusion/boost/ BTW , with latest mods this *should* work with CVS and Review version of Fusion... but no guarantess. regards Andy Little

"Joel de Guzman" <joel@boost-consulting.com> wrote in message news:45148B4C.3030804@boost-consulting.com...

Andy Little wrote:

...

"Joel de Guzman" <joel@boost-consulting.com> wrote in message news:ef1t83$bhv$1@sea.gmane.org...

...

Andy Little wrote:

...

"David Abrahams" <dave@boost-consulting.com> wrote in message news:87fyenkjh9.fsf@pereiro.luannocracy.com...

...

AFAICT the name ftag should be changed to something more descriptive and more certainly unique, e.g. boost_fusion_iterator_tag. Is there a reason it needs to be so short? Why make such a trivial change to the interface post review? AFAICS now is too late, and will cause unnecessary pain to users.. like myself. Well, actually, it shouldn't be a part of the interface. Here, I'm used to calling it the f#%$tag :) It's not supposed to be for public consumption, and I intend to enforce that. Either way, it's not a good name (especially for minors. ;) and it must be changed.

And BTW its pretty public in the 'make your own iterator" part of the docs.

Yeah. That's unfortunate. I asked Dan to fix it. Anyway, I suggest you do the right thing: specialize tag_of.

AFAICS specialising tag_of don't seem to work with current CVS. I seem to need to put the intrusive fusion_tag F*tag in, then things work OK. Also should I be specialising category_of or using a category F*tag ? Keep rockin'......... regards Andy Little

"Joel de Guzman" <joel@boost-consulting.com> wrote >

Keep rollin'..........

I'm sure rollin'. I'm also doin' a touch of scrollin' :-) Keep rockin', but a little less F'taggin' might keep you from flaggin'. Zounds! We English are great poets ain't we ? And if the mods look hard enough this isnt even O.T... I wonder what will happen when Fusion hits ConceptGCC? More Poetry ? WhoooEE .. SsshZamm! ... Nurse ! NURSE!! :-) regards Andy Little

"Joel de Guzman" <joel@boost-consulting.com> wrote

Joel de Guzman wrote:

...

nyway, I also have an iterator-adapter (iterator_facade), sequence_facade and view_facade in the works. With it, it is easier to write your own iterators, sequences and views. These will be built on top of the tag dispatching scheme.

Keep rollin'..........

Ok, I added iterator_facade and sequence_facade. fusion/sequence/adapted/array/array_iterator.hpp is the first example to migrate to the facade. I'm attaching it here for comments.

OK This looks a lot easier. Just need to test it out, and will try to do that over the next day or so. Regards Andy Little