[review] Fusion review begins

The review of Joel de Guzman's Fusion library begins today, May 1, 2006, and continues through May 10, 2006. :Download: http://spirit.sourceforge.net/dl_more/fusion_v2.zip :Description: Fusion is a library of heterogenous containers and views and algorithms. A set of heterogenous containers (vector, list, set and map) is provided out of the box along with view classes that present various composable views over the data. The containers and views follow a common sequence concept with an underlying iterator concept that binds it all together, suitably making the algorithms fully generic over all sequence types. The architecture is somewhat modeled after MPL which in turn is modeled after STL. It is code-named "fusion" because the library is the "fusion" of compile time metaprogramming with runtime programming. Review questions ================ Please always explicitly state in your review, whether you think the library should be accepted into Boost. You might want to comment on the following questions: - What is your evaluation of the design? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - What is your evaluation of the potential usefulness of the library? - Did you try to use the library? With what compiler? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? Cheers, Ron

Ronald Garcia wrote:
The review of Joel de Guzman's Fusion library begins today, May 1, 2006, and continues through May 10, 2006.
:Download: http://spirit.sourceforge.net/dl_more/fusion_v2.zip
Are the docs viewable somewhere?

Ronald Garcia escreveu:
The review of Joel de Guzman's Fusion library begins today, May 1, 2006, and continues through May 10, 2006.
:Download: http://spirit.sourceforge.net/dl_more/fusion_v2.zip
:Description: Fusion is a library of heterogenous containers and views and algorithms. A set of heterogenous containers (vector, list, set and map) is provided out of the box along with view classes that present various composable views over the data. The containers and views follow a common sequence concept with an underlying iterator concept that binds it all together, suitably making the algorithms fully generic over all sequence types.
The architecture is somewhat modeled after MPL which in turn is modeled after STL. It is code-named "fusion" because the library is the "fusion" of compile time metaprogramming with runtime programming.
I would like to see some "motivating example", like this one from the MPL documentation: http://www.boost.org/libs/mpl/doc/tutorial/tutorial-metafunctions.html This is, of course, not to say the library seems useless: it is I that don't know what to do with it. -- Pedro Lamarão Desenvolvimento Intersix Technologies S.A. SP: (55 11 3803-9300) RJ: (55 21 3852-3240) www.intersix.com.br Your Security is our Business

Pedro Lamarão wrote:
Ronald Garcia escreveu:
I would like to see some "motivating example", like this one from the MPL documentation:
http://www.boost.org/libs/mpl/doc/tutorial/tutorial-metafunctions.html
This is, of course, not to say the library seems useless: it is I that don't know what to do with it.
Sure that would be nice, but I'm afraid that would not be possible in the short amount of time given for the review. MPL did not have the Dimensional Analysis tutorial when it was first reviewed. What you are seeing now is actually a part of the book that Aleksey and Dave wrote. Back then, there was a paper titled "The Boost C++ Metaprogramming Library" also by Dave and Aleksey. I guess we could have written something similar, but then, Fusion is supposed to be based on the foundations of MPL. A lot of the basics of MPL should also apply to Fusion. The concepts of metaprogramming, type sequences, metafunctions, etc., are all based on the foundations already laid by MPL. You might even want to think of Fusion as an extension of MPL. In a sense, it is probably safe to say that a prerequisite for using Fusion is a good grasp of the basics of MPL. If you use MPL and wish to work on types *and* values, that's where Fusion comes in. To give you an idea of the applications of Fusion, Phoenix (http://tinyurl.com/6crgp), Spirit-2(http://spirit.sf.net/) xpressive(to be released with Boost very soon now) use Fusion in various degrees in its foundations. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

Joel de Guzman
To give you an idea of the applications of Fusion, Phoenix (http://tinyurl.com/6crgp), Spirit-2(http://spirit.sf.net/) xpressive(to be released with Boost very soon now) use Fusion in various degrees in its foundations.
Joel, I think showing the zip_iterator example I've been using would be very instructive, if not entirely representative of the library's full power: http://tinyurl.com/mrgub You could also link to these diffs to show how much simpler zip_iterator gets when we use fusion: http://tinyurl.com/qslyu I'll send you a slide set that you could use for inspiration if you like. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:
Joel de Guzman
writes: To give you an idea of the applications of Fusion, Phoenix (http://tinyurl.com/6crgp), Spirit-2(http://spirit.sf.net/) xpressive(to be released with Boost very soon now) use Fusion in various degrees in its foundations.
Joel, I think showing the zip_iterator example I've been using would be very instructive, if not entirely representative of the library's full power: http://tinyurl.com/mrgub
You could also link to these diffs to show how much simpler zip_iterator gets when we use fusion: http://tinyurl.com/qslyu
I'll send you a slide set that you could use for inspiration if you like.
Oh man, that would be awesome! Please! To be honest, I've been squeezing my brains inside out on an example like this to present. I thought about the properties code, event handling, etc., but all those are either not straight forward enough to be digested in one sitting or do not quite touch on the importance of the algorithms. Something like xpressive's "proto" library might have been a good candidate, but expression templates still seem to be too esoteric for general consumption. With materials like that and the slides you mention, I could write a motivational example in no time. Thank you! Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

"Pedro Lamarão" wrote
I would like to see some "motivating example", like this one from the MPL documentation:
http://www.boost.org/libs/mpl/doc/tutorial/tutorial-metafunctions.html
If the above is a motivating example for mpl, then I dont think it is the best
one. The problem is even highlighted in:
http://www.boost.org/libs/mpl/doc/tutorial/implementing.html
At the point in the text where it says;
"
However, if we try to write:
quantity

Andy Little wrote:
"Pedro Lamarão" wrote
I would like to see some "motivating example", like this one from the MPL documentation:
http://www.boost.org/libs/mpl/doc/tutorial/tutorial-metafunctions.html
If the above is a motivating example for mpl, then I dont think it is the best one. The problem is even highlighted in:
http://www.boost.org/libs/mpl/doc/tutorial/implementing.html
At the point in the text where it says;
" However, if we try to write:
quantity
f = m * a; we'll run into a little problem. ".
The problem is that mpl doesnt work as the (naive?) user expects, but worse the actual type of the quantity resulting from a multiplication isnt exactly defined. All that is known about it is that its dimension is comprised of "some sequence". In practise I found this, which was described as a convenience feature actually caused a great deal more work in the implementation. It also causes problems with trying to figure what to register with Boost.Typeof and I suspect that it would continue to throw up similar problems whenever the type is required to be known. The fact is that an interface must have exactly predictable types. The interface in this case ( The visible part. e.g for use in function signatures etc) is in the template signature of the class. IMO mpl isnt the best way to implement the dimension of a quantity for this reason.
I am sorry to go on about this, but I am expecting to have a boost review of my pqs library shortly and the issue is bound to come up. I suspect that the fact that it doesnt use mpl will be seen by the naive ( in the same sense as used above) user as a shortcoming of pqs. I believe that for this useage the shortcoming is rather in mpl for this particular useage, which the above example demonstrates.
So MPL is not sequence preserving after all. transform merely returns a sequence with the same properties as the original. This makes perfect sense. In Fusion, algorithms are not sequence preserving by design. However, we do have conversion functions/metafunctions to get a *true* vector/list/set/map from an arbitrary sequence. I think that's all that MPL needs to provide to solve the problem you mention. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

Joel de Guzman
So MPL is not sequence preserving after all.
It really depends what you mean by "sequence preserving." If it
returned the same type, it couldn't really do anything. So you have
to decide how many guarantees you're going to give about the result
type. Andy wants it to be spelled as a specialization of the same
template, so transform operating on mpl::vector
transform merely returns a sequence with the same properties as the original. This makes perfect sense. In Fusion, algorithms are not sequence preserving by design. However, we do have conversion functions/metafunctions to get a *true* vector/list/set/map from an arbitrary sequence.
No, I don't think that would satisfy Andy, because you're surely not
guaranteeing that the result type will be spelled
fusion::list

Ronald Garcia
- What is your evaluation of the design?
It's very good overall. I find the dispatching and function selection mechanisms cumbersome and inelegant, but I'm not sure if there's a better solution. Maybe since usually iterator intrinsics have to be implemented all together, it would be best to allow them to be defined in one big class template, rather than being forced to piecemeal create a bunch of little nested templates. I'm a bit surprised to see static call() functions used instead of operator() for the implementation of a function call, and also to see the use of nested apply<> metafunctions for computing the result types of functions. I thought Joel had decided to use something compatible with boost::result_of. I'm surprised and a little concerned about what I perceive to be redundancy in the value_of metafunction and the deref_impl metafunction class. In the extension example, I see repeatedly the same patter transferring the cv-ref qualification from one type to another. Can't that be made simpler for extenders? typedef typename mpl::if_< is_const<Sequence>, std::string const&, std::string&>::type type;
- What is your evaluation of the implementation?
What I've seen is excellent.
- What is your evaluation of the documentation?
A decent start, but needs more attention. Forgive me for being too
pedantic (I know that I am):
* The top-level heading "Support" is misleading at best.
* "As with MPL and STL iterators describe positions, and provide..."
^ move this comma-----^
^--- to here
* "A Forward Iterator traverses a Sequence allowing movement"
^^
You seem to have double spaces------------^^ where commas should be.
As with most English writing, the missing comma is much less common
than the extraneous one in this doc. I'm just not going to point out
all the extra ones.
* How does an "expression requirement" differ from a "requirement?"
* IMO there's no need for novel approaches to concept documentation
here [unless you are going to start using ConceptGCC notation ;-)].
The expression semantics should be an additional column in the
regular requirements table.
* IMO the doc formatting needs to better reflect the structure. For
example, at libs/fusion/doc/html/fusion/iterators/functions.html
there's no indication that the "Functions" section I'm in is a
subsection of "Iterators" rather than "Sequences" Maybe we should
show
Fusion > Iterators > Functions
above the horizontal rule on that page, for example.
* On that note, the section title "Adapted" under Sequences is
unacceptably non-descriptive without context.
* why do sequences have "intrinsics" while iterators only have
"functions?"
* IMO too many of the pages
At libs/fusion/doc/html/fusion/extension.html:
* Is all the information in this section available in
* is "ftag" a typo? Was "tag" intended? If not, what is the "f" for?
* "As it is straightforward to do, we are going to provide a random
access iterator in our example."
Doesn't providing random access imply providing a whole bunch of
iterator comparison and offsetting functionality that you wouldn't
otherwise have to? Why complicate the example? Oh, you have the
iterator claim it's random access, but then you refer the reader
to the example. Hm.
* "Notice we need both const and non const partial specialization."
There's no such thing; this is confusing. Reword
* it's "metafunction," not "meta function"
* "A few details need explaining here:
1. The iterator is parameterized by the type it is iterating
over, and the index of the current element."
2. The typedef struct_type provides the type being iterated
over, and the typedef index provides the index of the current
element. These will become useful later in our
implementation."
Aren't these points redundant with one another? What details are
being explained? Just the meaning of the template parameters?
The intro set me up for something a lot less self-evident.
...
5. All that remains is a constructor that initializes a
reference to the example_struct being iterated over.
"All that remains" doesn't seem right here, and there's plenty in
the example that bears explanation but that you've left out.
Maybe you should just add comments numbering the interesting lines
template
- What is your evaluation of the potential usefulness of the library?
Very useful.
- Did you try to use the library? With what compiler?
Yes, with several; I don't remember which exactly.
Did you have any problems?
No.
- How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain?
Very. I think this library should be accepted. However 1. the documentation should undergo an editorial review, to remove ambiguity, clarify the line between formal and informal, and with Strunk & White's mantra to "omit needless words" (and commas ;->) in mind. I only scratched the surface here. This is not necessarily a criticism of this particular library. All of our submissions need that. :) 2. I would like the authors to have a transition plan in place for replacement of the existing boost tuples. As long as the boost tuple library is there in its current form it will be the official tuple library of boost and I won't be able to use any fusion algorithms in my libraries because people will want to pass me boost::tuple. Making boost::tuple a valid fusion sequence would probably be sufficient. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams
- What is your evaluation of the documentation?
A decent start, but needs more attention.
It would be really nice for reviewers if there were a website where the doc was posted (with your edits, if you like -- the one in the vault is stable) so that we can refer to it in our postings. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:
David Abrahams
writes: - What is your evaluation of the documentation? A decent start, but needs more attention.
It would be really nice for reviewers if there were a website where the doc was posted (with your edits, if you like -- the one in the vault is stable) so that we can refer to it in our postings.
Ok, we'll set one up after we do some preliminary edits from your review. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

David Abrahams wrote:
David Abrahams
writes: - What is your evaluation of the documentation? A decent start, but needs more attention.
It would be really nice for reviewers if there were a website where the doc was posted (with your edits, if you like -- the one in the vault is stable) so that we can refer to it in our postings.
Done. Here's the current snapshot of the latest changes (mostly from Dan). I shall upload snapshots regularly here: http://spirit.sourceforge.net/dl_more/fusion_v2_review/ The docs can be read online here: (http://tinyurl.com/em6wg) http://spirit.sourceforge.net/dl_more/fusion_v2_review/libs/fusion/doc/html/... Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

David Abrahams
I object to this policy:
Unless otherwise noted, components are in namespace boost::fusion. For the sake of simplicity, code in this quick start implies using directives for the fusion components we will be using.
because it is utterly ambiguous whether some of these functions are supposed to be called without qualification and found via ADL or not.
I also note in fusion's tests, constructs like this:
void test()
{
using namespace boost::fusion;
using namespace boost;
{ // Testing deref, next, prior, begin, end
char const* s = "Hello";
typedef FUSION_SEQUENCE

David Abrahams wrote:
David Abrahams
writes: I object to this policy:
Unless otherwise noted, components are in namespace boost::fusion. For the sake of simplicity, code in this quick start implies using directives for the fusion components we will be using.
because it is utterly ambiguous whether some of these functions are supposed to be called without qualification and found via ADL or not.
Understood.
I also note in fusion's tests, constructs like this:
void test() { using namespace boost::fusion; using namespace boost;
{ // Testing deref, next, prior, begin, end
char const* s = "Hello"; typedef FUSION_SEQUENCE
seq_type; seq_type v(1, 'x', 3.3, s); result_of::begin ::type i(v); ^^^^^^^^^ which might (depending on whether ADL is intended) fail to test some important things in the library, and are certain to fail compilation once boost/utility/result_of.hpp gets included by a few more boost headers.
Right. Ok, noted. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

David Abrahams wrote: I'll try to reply to some of Dave's comments here. I think it's better if I do it a part at a time starting with design.
- What is your evaluation of the design?
It's very good overall.
I find the dispatching and function selection mechanisms cumbersome and inelegant, but I'm not sure if there's a better solution. Maybe since usually iterator intrinsics have to be implemented all together, it would be best to allow them to be defined in one big class template, rather than being forced to piecemeal create a bunch of little nested templates.
That's a possibility. One advantage of that approach is that we can subclass from a "facade" class that implements most of the basic infrastructure whereby limiting the things that are needed to be implemented. It might also be a good idea to implement some form of iterator adapters. I'll spend some time thinking about that. I think a similar approach might also be useful for implementing the sequence intrinsics and correspondingly, the view adapters.
I'm a bit surprised to see static call() functions used instead of operator() for the implementation of a function call, and also to see the use of nested apply<> metafunctions for computing the result types of functions. I thought Joel had decided to use something compatible with boost::result_of.
Well, I think we already have that covered in our chat a while ago. Anyway, the answer is that: unlike Fusion1, Fusion2 algorithms are not objects. Fusion1 used to have algorithm objects which are basically functors. This is not the case for Fusion2. One reason is because of the need to have the syntax: at_c<N>(x) or find<T>(x), etc. If at_c was an object, the interface would have to be at_c<N>()(x). We can get away with that, but we definitely can't get away with get<N>(x), which is part of the TR1 interface.
I'm surprised and a little concerned about what I perceive to be redundancy in the value_of metafunction and the deref_impl metafunction class.
result_of::value_of returns the actual type in the sequence it is analogous to tuple_element. result_of::deref returns the exact type returned by deref(i) its purpose is mainly to be able to write forwarding functions that call deref(i). deref(i) may or may not add a reference to the actual type in the sequence (e.g. for efficiency-- vector iterator returns a reference, mpl sequence iterator returns a value). IOTW, it is not possible (or at least awkward) to write forwarding functions with result_of::value_of alone.
In the extension example, I see repeatedly the same patter transferring the cv-ref qualification from one type to another. Can't that be made simpler for extenders?
typedef typename mpl::if_< is_const<Sequence>, std::string const&, std::string&>::type type;
Can you point me to an example in the library, please? Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

David Abrahams wrote:
Also I think the cover page credits me for Doug Gregor's work :)
My bad! Well, I first got that idea from you. It was not until a little bit later that I learned about Doug's work. Anyway, please excuse me. I'll definitely give credit where credit is due. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

On 05/01/2006 08:49 AM, Ronald Garcia wrote:
The review of Joel de Guzman's Fusion library begins today, May 1,
In extension.htm, there's a section:
Designing a suitable iterator
which has code:
template

Based on demand, the review period for the Fusion library has been extended to May 13th. For convenience I have quoted the original message below. Cheers, ron On May 1, 2006, at 9:49 AM, Ronald Garcia wrote:
The review of Joel de Guzman's Fusion library begins today, May 1, 2006, and continues through May 10, 2006.
:Download: http://spirit.sourceforge.net/dl_more/fusion_v2.zip
:Description: Fusion is a library of heterogenous containers and views and algorithms. A set of heterogenous containers (vector, list, set and map) is provided out of the box along with view classes that present various composable views over the data. The containers and views follow a common sequence concept with an underlying iterator concept that binds it all together, suitably making the algorithms fully generic over all sequence types.
The architecture is somewhat modeled after MPL which in turn is modeled after STL. It is code-named "fusion" because the library is the "fusion" of compile time metaprogramming with runtime programming.
Review questions ================
Please always explicitly state in your review, whether you think the library should be accepted into Boost.
You might want to comment on the following questions:
- What is your evaluation of the design? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - What is your evaluation of the potential usefulness of the library? - Did you try to use the library? With what compiler? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain?
Cheers, Ron

On 05/01/2006 08:49 AM, Ronald Garcia wrote:
The review of Joel de Guzman's Fusion library begins today, May 1, 2006, and continues through May 10, 2006. The boost/fusion/iterator/equal_to.hpp has:
operator==(Iter1 const&, Iter2 const&)
{
return result_of::equal_to

Hi! This is my review of the Fusion library. Sorry for the late review... I should probably state my bias, I've known Fusion since it was bundled with Spirit and have wanted to see it in Boost ever since I started to grasp its usefulness (I'll admit it wasn't right away... docs were scarce back then ;-) ). I've commented on Fusion occasionally and have contributed patches and code.
Please always explicitly state in your review, whether you think the library should be accepted into Boost.
Yes, I think Fusion should be an official part of Boost. Fusion has been in development for a couple of years and is a mature library. It is unofficially part of Boost, it is available under the Spirit tree. Fusion is being used by a growing number of official and proposed Boost libraries. It is instrumental in the redesign of Spirit and Phoenix (in queue for review?) and also in the fork and generalization of Proto (out of Xpressive), as a general-purpose Expression Templates library. There's really no excuse for a library of this caliber to be relegated to an implementation detail. There are a few configuration macros whose default value is hardcoded to 10. They are, FUSION_MAX_UNPACK_ARG_SIZE FUSION_MAX_VECTOR_SIZE FUSION_MAX_SET_SIZE FUSION_MAX_LIST_SIZE FUSION_MAX_MAP_SIZE . I'd like to see a single macro to globally set the default value for all the others. Which also reminds me, all macros still need to be BOOST_ified.
- What is your evaluation of the design?
Fusion builds on top of the design of MPL and also of the STL so I don't have much to say in this regard. It is also a mature library that has been in development for some time. In the past I've noticed minor inconsistencies between MPL and Fusion, which I reported back (the lack of unpack_args was one of them). I think these two libraries, as much as possible, should be kept in sync. Apart from interface exposed to library users, another important aspect is the interface for extension. In Fusion-1, implementing a new iterator type required one to specialize not only Fusion intrinsics, but also mpl intrinsics. Things have improved since then, but I didn't look into this specifically for the review. I'll try to share any comments at another time. The algorithms also look straightforward to implement and the algorithms already in the library are a testament to that. On the directory structure used for the headers, I like the modular approach but sometimes get the feeling that it's too fragmented. Having more top-level reflections of individual components would ease my concerns. Another aspect is that this structure diverges from what is used in MPL. Given the parallel between the two libraries I think an effort should be made to keep the structure similar. In fact, this is also part of the user interface. Still, in the end, I recognize that they're two independent libraries and must each follow its path.
- What is your evaluation of the implementation?
The implementation is very readable and even simple to grasp, once one understands a few fundamental concepts. Although this aspect is not required of a Boost library, I think it is great when it can be achieved. Fusion is coded with the expectation that the library code will be optimized into very tight code, which seems to be a reasonable assumption with (any?) modern compilers. I would like to see more compile-time concept checks and "friendlier" error messages added to the implementation. Ideally, the user should never see errors beyond what s/he directly uses. Unfortunately, however, at places this will be at odds with the simplicity and readability of the code.
- What is your evaluation of the documentation?
I think I shouldn't be allowed to comment on this department... Anyway, I think the docs still need some work, but I generally like the direction. I'll try to offer Joel and Dan specific comments later. For the record, and while it is no excuse, it should be noted that Joel de Guzman is one of the lead developers of QuickBook (the other one being Eric Niebler), which makes it harder to write the docs... I mean, he can't simply complain about the tools. I'm sure Joel is taking notes and, in the end, QuickBook and the rest of us will benefit as well :-)
- What is your evaluation of the potential usefulness of the library?
I believe Fusion has great potential as a library building block but also as a more application-oriented library. On the library building block side, I believe it has the potential to become a fundamental/core piece of Boost. For application developers, Fusion takes tuples to a new level, allowing one to make the most of information that is available at compile-time.
- Did you try to use the library? With what compiler?
Yes. Mostly with gcc (3.4 and over), also with intel-9.0 (IIRC), both under linux.
Did you have any problems?
No. Joel and Dan have been very helpful and supportive for any doubts and issues I have had and given the readability of the code I have been known to provide fixes to typos and other minor issues.
- How much effort did you put into your evaluation? A glance? A quick reading? In-depth study?
I suppose it qualifies as an in-depth study for the code and a quick
reading for the docs. I've followed Fusion evolution since version 1 and
have perused the code thoroughly. I implemented unpack_args for fusion-1
for my own purposes and later ported and contributed it to fusion-2.
As a user of the library, I've privately used Fusion to re-implement and
extend Spirit's closures, to allow members to selected and aggregated
for return. Something like,
closure
- Are you knowledgeable about the problem domain?
I'm not sure I can categorize Fusion into a single and specific problem domain. I think the potential use cases are vast and the domain too broad. I can't claim to be knowledgeable in all the possibilities it allows. I'd like to congratulate the developers of the library on the great library and thank them for exploring the concepts and bringing it to us! I really expect to see Fusion as an official Boost library, soon ;-) Best regards, João
participants (8)
-
Andy Little
-
David Abrahams
-
Joel de Guzman
-
João Abecasis
-
Larry Evans
-
Neal Becker
-
Pedro Lamarão
-
Ronald Garcia