David Abrahams wrote:

...

struct A { typedef int M;

template <class M> void foo(void) { M m; // which M is this? } };

I know the C++ committe is discussing this issue at this moment. The argument would be that "M" names the typedef because it's "more stable" than the template parameter (which could get renamed in an out-of-class definition). See also http://gcc.gnu.org/PR13967 for a detailed discussion.

I'm sorry, but that's insane from a usability POV. C++ already has too many places where something far away can be chosen instead of the "obvious" alternative close by (see ADL).

I'm not advocating that, I'm just saying that it's how GCC currently works and it seems to be a gray area of the standard. My personal opinion is that GCC is wrong: I agree with you that the template parameter should be found on name lookup.

Introducing a typedef in an enclosing namespace should not affect the meaning or well-formedness of a use of a template parameter, especially because this sort of thing is liable to happen due to changes in #includes.

We're not speaking of namespace scope but class scope, though. It's a bit harder to change things at class scope. Anyway, we both agree that it's insane for C++ to behave like this, so never mind :) -- Giovanni Bajo

I apologize in advance for a really naive question, but as an unsophisticated C++ programmer (at least relative to the boost experts), why can't we simply ban this kind of code? In other words, if "M" has already been defined within class scope for something else, it should not be allowed as a template parameter name. Code like this just makes my head spin, so I can't see how allowing it is to anyone's benefit. And can't situations like this be avoided with coding practices that dictate different styles for naming typedefs from template parameters? I would, of course, still want the compiler to at least warn me if something like this occurs. Gabriel Dos Reis wrote:

"Giovanni Bajo" <giovannibajo@libero.it> writes:

| David Abrahams wrote: | | >> struct A { | >> typedef int M; | >> | >> template <class M> | >> void foo(void) { | >> M m; // which M is this? | >> } | >> }; | >> | >> I know the C++ committe is discussing this issue at this moment. The | >> argument would be that "M" names the typedef because it's "more | >> stable" than the template parameter (which could get renamed in an | >> out-of-class definition). See also http://gcc.gnu.org/PR13967 for a | >> detailed discussion. | > | > I'm sorry, but that's insane from a usability POV. C++ already has

Dave, I got the copy of the message you sent. While the rules as currently defined are not clear and not even what you would expect, I don't agree that "insane" (or such words) are qualifications that should be used if you wanted a technical discussion about your request.

| > too many places where something far away can be chosen instead of the | > "obvious" alternative close by (see ADL).

When it comes to name look up, I think "obvious" does not carry any meaning that would be of help -- I'm not just speaking of what C++ current rules say about name look-up.

| I'm not advocating that, I'm just saying that it's how GCC currently | works and it seems to be a gray area of the standard. My personal | opinion is that GCC is wrong: I agree with you that the template | parameter should be found on name lookup.

And it should be also pointed out that GCC is not the only compiler that interprets the construct differently from what you would consider "obvious". And the problem has to do with the way the current rules are worded (the least that can be said is that the rules are incomplete). We have had a long discussion about this on the CWG reflector. This undoubtely is going to be debated again at the next meeting.

One thing that should be kept in the fore is that, when you provide an in-class definition of a member function, that definition gets re-evaluated as an out-of-class definition; that is not a requirement but, it is a common strategy in order to take into account use of names that are declared "later" in the class. If the re-evaluation of the definition in the completed scope differs from the evaluation of the in-class definition, then the program is ill-formed and no diagnostic is required. Some compilers however, as a QoI, issues diagnostics on the cases they can catch. For example,

typedef int A; struct B { A x; void f(); struct A { }; };

is ill-formed and it is likely that your favorite compiler will complain.

Now, what is the relation with the member template case presented above? When you have something like

struct A { typedef int M;

template <class M> void foo(void) { M m; // which M is this? } };

it gets evaluated as if

struct A { typedef int M; template<class M> void f(); };

template<class M> void A::f() { M m; }

The name look-up rules as currently defined do not attach the template parameters to the lexical scope of the member template, so when in the body of A::f(), the scope stack looks like this:

namespace scope (global scope) template scope (contains class M) class A scope (contains typedef int M;) A::f()-body scope

Therefore, look-up for M finds the class-scope declaration "typedef int M;" -- which is the `"obvious" alternative close by' if I were to use your words I already disagree with.

In order to choose the other `"obvious" alternative close by', the out-of-class definition for member template syntax should have been

A:: // <-- Note the enclosing A template<class M> void f() { M m; }

which would yield the scope stack

namespace scope (global scope) class A scope (contains typedef int M;) template scope (contains class M) A::f()-body scope

The point of the discussion we reached last week (with no vote to evaluate consensus) was that, we would change/clarify the lookup rules so that when the compiler sees the out-of-class definition of a member-template it does a resuffling of the scope stack to make it look like what one would get did we have the "right" syntax (as shown above).

The case posted by David Vandevoorde on the newsgroups is not revolved by the change I mentioned above. It should also be pointed out the decision of have a member name hides an enclosing template-parameter name was a conscious decision, made after debate.

Now, just to illustrate the point "name look up" is not an "obvious" thing; let's condider the following example

template<class T> struct S { typedef char C; template<class> int f(); };

Consider the following two "possible" out-of-class definitions

// alternative #1

template<class C> template<class U> int S<C>::f() { return sizeof (C); }

// alternative #2 template<class T> template<class C> int S<T>::f() { return sizeof (C); }

As interpreted by at least 3 or 4 major compilers, the "C" in the body consistently refers to S<>::C. If changed abruptly to the "obvious" close by, then the two definitions are not equivalent and in the "C" in the body of #1 will not be the "obvious" close by (template-parameter). During the discussion on the CWG reflector, we introduced the notion of "valid template-parameter renaming" (which is similar to what FP community calls valid "alpha conversion"). I proposed that if an out-of-class definition of a member template uses an invalid template-parameter renaming, then the program is ill-formed. The idea is to avoid misunderstanding about the `"obvious" alternative close by' between programmer and compiler.

| > Introducing a typedef in an | > enclosing namespace should not affect the meaning or well-formedness | > of a use of a template parameter, especially because this sort of | > thing is liable to happen due to changes in #includes.

Dave, I can't find anything like the one you're describing in the example you gave. Class-scope declarations are "preserved" against outside perturbation -- that is what the re-evaluation rules is about.

-- Gaby _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Gabriel Dos Reis <gdr@integrable-solutions.net> writes:

"Giovanni Bajo" <giovannibajo@libero.it> writes:

| David Abrahams wrote: | | >> struct A { | >> typedef int M; | >> | >> template <class M> | >> void foo(void) { | >> M m; // which M is this? | >> } | >> }; | >> | >> I know the C++ committe is discussing this issue at this moment. The | >> argument would be that "M" names the typedef because it's "more | >> stable" than the template parameter (which could get renamed in an | >> out-of-class definition). See also http://gcc.gnu.org/PR13967 for a | >> detailed discussion. | > | > I'm sorry, but that's insane from a usability POV. C++ already has

Dave, I got the copy of the message you sent. While the rules as currently defined are not clear and not even what you would expect, I don't agree that "insane" (or such words) are qualifications that should be used if you wanted a technical discussion about your request.

I don't think I really was trying to start a technical discussion. I was just trying to register a strong opinion with someone (you) who is working on the problem.

| > too many places where something far away can be chosen instead of the | > "obvious" alternative close by (see ADL).

When it comes to name look up, I think "obvious" does not carry any meaning that would be of help -- I'm not just speaking of what C++ current rules say about name look-up.

| I'm not advocating that, I'm just saying that it's how GCC currently | works and it seems to be a gray area of the standard. My personal | opinion is that GCC is wrong: I agree with you that the template | parameter should be found on name lookup.

And it should be also pointed out that GCC is not the only compiler that interprets the construct differently from what you would consider "obvious". And the problem has to do with the way the current rules are worded (the least that can be said is that the rules are incomplete). We have had a long discussion about this on the CWG reflector. This undoubtely is going to be debated again at the next meeting.

One thing that should be kept in the fore is that, when you provide an in-class definition of a member function, that definition gets re-evaluated as an out-of-class definition; that is not a requirement but, it is a common strategy in order to take into account use of names that are declared "later" in the class. If the re-evaluation of the definition in the completed scope differs from the evaluation of the in-class definition, then the program is ill-formed and no diagnostic is required. Some compilers however, as a QoI, issues diagnostics on the cases they can catch. For example,

typedef int A; struct B { A x; void f(); struct A { }; };

is ill-formed and it is likely that your favorite compiler will complain.

Now, what is the relation with the member template case presented above? When you have something like

struct A { typedef int M;

template <class M> void foo(void) { M m; // which M is this? } };

it gets evaluated as if

struct A { typedef int M; template<class M> void f(); };

template<class M> void A::f() { M m; }

The name look-up rules as currently defined do not attach the template parameters to the lexical scope of the member template, so when in the body of A::f(), the scope stack looks like this:

namespace scope (global scope) template scope (contains class M) class A scope (contains typedef int M;) A::f()-body scope

Therefore, look-up for M finds the class-scope declaration "typedef int M;" -- which is the `"obvious" alternative close by' if I were to use your words

Not AFAICS. The template parameter is closer to the use of M and to my eye, much more obvious.

In order to choose the other `"obvious" alternative close by', the out-of-class definition for member template syntax should have been

A:: // <-- Note the enclosing A template<class M> void f() { M m; }

which would yield the scope stack

namespace scope (global scope) class A scope (contains typedef int M;) template scope (contains class M) A::f()-body scope

The point of the discussion we reached last week (with no vote to evaluate consensus) was that, we would change/clarify the lookup rules so that when the compiler sees the out-of-class definition of a member-template it does a resuffling of the scope stack to make it look like what one would get did we have the "right" syntax (as shown above).

The case posted by David Vandevoorde on the newsgroups is not revolved by the change I mentioned above. It should also be pointed out the decision of have a member name hides an enclosing template-parameter name was a conscious decision, made after debate.

Now, just to illustrate the point "name look up" is not an "obvious" thing; let's condider the following example

template<class T> struct S { typedef char C; template<class> int f(); };

Consider the following two "possible" out-of-class definitions

// alternative #1

template<class C> template<class U> int S<C>::f() { return sizeof (C); }

// alternative #2 template<class T> template<class C> int S<T>::f() { return sizeof (C); }

As interpreted by at least 3 or 4 major compilers, the "C" in the body consistently refers to S<>::C. If changed abruptly to the "obvious" close by, then the two definitions are not equivalent

Right.

and in the "C" in the body of #1 will not be the "obvious" close by (template-parameter).

Why not? It seems to me you could design the rules so the "C" in each case referred to a (different) template parameter.

During the discussion on the CWG reflector, we introduced the notion of "valid template-parameter renaming" (which is similar to what FP community calls valid "alpha conversion"). I proposed that if an out-of-class definition of a member template uses an invalid template-parameter renaming, then the program is ill-formed. The idea is to avoid misunderstanding about the `"obvious" alternative close by' between programmer and compiler.

That sounds like it might be a good idea.

| > Introducing a typedef in an | > enclosing namespace should not affect the meaning or well-formedness | > of a use of a template parameter, especially because this sort of | > thing is liable to happen due to changes in #includes.

Dave, I can't find anything like the one you're describing in the example you gave. Class-scope declarations are "preserved" against outside perturbation -- that is what the re-evaluation rules is about.

Yeah, I misread the example. -- Dave Abrahams Boost Consulting www.boost-consulting.com

Gabriel Dos Reis <gdr@integrable-solutions.net> writes:

If you follow the first principle that scopes nest, then the "C" in the altenrative #1 gets hidden by the member S<>::C in the local scope of S<>::f()-body (current situation). If you want something different, you'll end up with a notion similar to "parallel scopes" or "injection of enclosing scope into the enclosed scopes" (which is contrary to "scopes nest").

You could just use the rule that each template parameter list forms a scope that encloses whatever it adorns. Then the reading that's obvious to me follows "from first principles". -- Dave Abrahams Boost Consulting www.boost-consulting.com

On 14 Feb 2004 18:00:58 +0100, Gabriel Dos Reis <gdr@integrable-solutions.net> wrote:

Dave, I got the copy of the message you sent. While the rules as currently defined are not clear and not even what you would expect, I don't agree that "insane" (or such words) are qualifications that should be used if you wanted a technical discussion about your request.

Since I joined his comment with a warm "yeah" I feel the need to clarify that, to me, it didn't carry any depreciative judgement against the committee. I'm not a native speaker and, in context, the term looked to me as simply meaning "extremely dangerous". Only after your reaction I have seen on the dictionary that it is a little stronger than I imagined. Sorry for that, but really I don't have actual spoken practice in English. I also want to apologize with Brian, Yannis and Mat, since this is the formal review FC++, not a comp.std.c++ thread. I had a look at the documentation but it is really overwhelming so I should probably find another way to have you forgive me than making a review, at least for now :)

On Fri, 13 Feb 2004 17:54:41 -0500, David Abrahams <dave@boost-consulting.com> wrote:

"Giovanni Bajo" <giovannibajo@libero.it> writes:

...

struct A { typedef int M;

template <class M> void foo(void) { M m; // which M is this? } };

I know the C++ committe is discussing this issue at this moment. The argument would be that "M" names the typedef because it's "more stable" than the template parameter (which could get renamed in an out-of-class definition). See also http://gcc.gnu.org/PR13967 for a detailed discussion.

I'm sorry, but that's insane from a usability POV.

Yeah. These aren't certainly the kind of rules that make we sleep peacefully :( BTW, template parameters are among the things which get more easily hidden in C++ - see 14.6.1/5 and 14.6.1/7. Daveed Vandevoorde is collecting opinions on this (and replies are a plebiscite) http://google.com/groups?threadm=52f2f9cd.0402031816.5fb1e8a0%40posting.goog... Do you know why? Is he proposing some change in the CWG? Genny.

On Sat, 14 Feb 2004 02:09:03 +0100, "Giovanni Bajo" <giovannibajo@libero.it> wrote:

Gennaro Prota wrote:

...

Do you know why? Is he proposing some change in the CWG?

I guess it's just a consequence of the GCC bugreport, and Gaby raising the issue with the CWG. Let's hope we see a Defect Rerport about this soon.

Aha, thanks! Indeed, Daveed has just posted that now there is one :) Genny.

Gabriel Dos Reis wrote:

"Giovanni Bajo" <giovannibajo@libero.it> writes:

...

Gennaro Prota wrote:

...

Do you know why? Is he proposing some change in the CWG?

I guess it's just a consequence of the GCC bugreport, and Gaby raising the issue with the CWG.

Yes, that is the case. But the issues are slightly different. A long time ago, the committee decided, after debate, that member names hide enclosing template-parameters. The fact that names from base classes can hide template-parameters too comes as a -logical- consequence from that general rule and the principle that scopes nests.

This however runs contrary to the principle that I, as a user, should be able to explicitly disambiguate; since I can write a qualified name for a class member but not for a template parameter, it naturally follows that template parameters should hide class members and not vice versa.

{- We are still off-topic for boost but at least we don't bother Brian and all the people who are interested in the FC++ review -} On 16 Feb 2004 16:39:23 +0100, Gabriel Dos Reis <gdr@integrable-solutions.net> wrote:

"Peter Dimov" <pdimov@mmltd.net> writes:

| This however runs contrary to the principle that I, as a user, should be | able to explicitly disambiguate; since I can write a qualified name for a | class member but not for a template parameter, it naturally follows that | template parameters should hide class members and not vice versa.

That does not follow. Your raisoning is based on the assumption that there is an ambiguity, but there is none.

Come on, he didn't mean it's ambiguous. Just that with one rule the user can choose what the name should refer to, with the other one he/she cannot. Genny. Genny.

Gabriel Dos Reis <gdr@integrable-solutions.net> writes:

Gennaro Prota <gennaro_prota@yahoo.com> writes:

| Just that with one rule the user can choose what the name should | refer to, with the other one he/she cannot.

I understand the reasoning of letting the user says what the name should refer to. Which makes me inclined to make the construct ill-formed: That is not different from the rule that says you cannot reclare a template-parameter in its scope.

That would be horrible for generic code. What are the requirements on the T parameter of this class template? template <class T> struct Der : T { template <class U> void f(U x) {} }; -- Dave Abrahams Boost Consulting www.boost-consulting.com

On Mon, Feb 16, 2004 at 06:46:26PM +0100, Gabriel Dos Reis wrote:

David Abrahams <dave@boost-consulting.com> writes: | Gabriel Dos Reis <gdr@integrable-solutions.net> writes: | > I understand the reasoning of letting the user says what the name | > should refer to. Which makes me inclined to make the construct | > ill-formed: That is not different from the rule that says you cannot | > reclare a template-parameter in its scope. | | That would be horrible for generic code.

Care to explain why?

| What are the requirements | on the T parameter of this class template? | | template <class T> | struct Der : T | { | template <class U> | void f(U x) | {} | };

Should those requirements be different from those for

template<class S> struct Der : S { template<class U> void f(U x) { } };

Apologies ahead of time if I am putting words in anyone's mouth. I think Dave wants to imply that the class T in his example is not allowed to have a member named U according to Gaby's proposed rule. But this isn't how the rule would work; the way I see it, _these_: struct Base { typedef int U; }; template <class U> struct Derived : Base { }; // illegal struct Derived2 : Base { template <class U> void f(U u) {} // illegal }; would be illegal ("the name U already has another meaning; choose another name for your template parameter"), whereas _these_: struct Base { typedef int U; }; template <class B, class U> struct Derived : B { }; /* ... Derived<Base> ... */ template <class B> struct Derived2 : B { template <class U> void f(U u) {} }; /* ... Derived2<Base> ... */ are both fine, since Base's "U" is hidden by virtue of being a dependent name. There is still an issue of what happens if one of these last two classes issues a "using B::U" in its body; presumably then "U" ought to become outlawed as a valid template parameter name. Just my two cents here... -- -Brian McNamara (lorgon@cc.gatech.edu)

Brian McNamara <lorgon@cc.gatech.edu> writes:

Apologies ahead of time if I am putting words in anyone's mouth.

I think Dave wants to imply that the class T in his example is not allowed to have a member named U according to Gaby's proposed rule.

That's what I thought he meant.

But this isn't how the rule would work; the way I see it, _these_:

struct Base { typedef int U; };

template <class U> struct Derived : Base { }; // illegal

struct Derived2 : Base { template <class U> void f(U u) {} // illegal };

would be illegal ("the name U already has another meaning; choose another name for your template parameter"), whereas _these_:

struct Base { typedef int U; };

template <class B, class U> struct Derived : B { }; /* ... Derived<Base> ... */

template <class B> struct Derived2 : B { template <class U> void f(U u) {} }; /* ... Derived2<Base> ... */

are both fine, since Base's "U" is hidden by virtue of being a dependent name. There is still an issue of what happens if one of these last two classes issues a "using B::U" in its body; presumably then "U" ought to become outlawed as a valid template parameter name.

Just my two cents here...

OK, I think I understand now. I think it's a bizarre and inconsistent rule. Shall we outlaw: struct Base { static int x; }; struct Derived : Base { void f(int x) {} // Horrors! we're masking a base class member! }; Sorry for the sarcasm; it'll wear off by tomorrow. -- Dave Abrahams Boost Consulting www.boost-consulting.com

Gabriel Dos Reis <gdr@integrable-solutions.net> writes:

David Abrahams <dave@boost-consulting.com> writes:

| Brian McNamara <lorgon@cc.gatech.edu> writes: | | > Apologies ahead of time if I am putting words in anyone's mouth. | > | > I think Dave wants to imply that the class T in his example is not | > allowed to have a member named U according to Gaby's proposed rule. | | That's what I thought he meant.

But then, your remark about "would be horrible for generic program" is, ahem, puzzling. You still did not explain why.

Well, if my understanding of your proposed rule had been correct, it would've meant that the names of a template's parameters would leak out into requirements on its arguments (that they not have nested members with that name).

| > But this isn't how the rule would work; the way I see it, _these_: | > | > struct Base { typedef int U; }; | > | > template <class U> | > struct Derived : Base { }; // illegal | > | > struct Derived2 : Base { | > template <class U> | > void f(U u) {} // illegal | > }; | > | > would be illegal ("the name U already has another meaning; choose | > another name for your template parameter"), whereas _these_: | > | > struct Base { typedef int U; }; | > | > template <class B, class U> | > struct Derived : B { }; | > /* ... Derived<Base> ... */ | > | > template <class B> | > struct Derived2 : B { | > template <class U> | > void f(U u) {} | > }; | > /* ... Derived2<Base> ... */ | > | > are both fine, since Base's "U" is hidden by virtue of being a dependent | > name. There is still an issue of what happens if one of these last two | > classes issues a "using B::U" in its body; presumably then "U" ought to | > become outlawed as a valid template parameter name. | > | > Just my two cents here... | | OK, I think I understand now. I think it's a bizarre and inconsistent | rule.

What is bizarre and inconsistent about it and is not with the other alternatives you care to name?

As I expected my example below to demonstrate, I think it's inconsistent with the way names from non-dependent base classes are dealt with when masked by function parameter names. Up till now, the only way that changing the name of a private member could break a derived class (AFAIK) is if the member were virtual. IIUC your proposed rule would add another one.

This question is not rhetorical. When you register strong opinons about a technical issue or throw in the air fashionable words like "generic programming",

Generic Programming went out of style around here about 7 weeks ago. It's so, like, 2003.

the least that could be expected is that you provide technical explanation.

I just misunderstood your proposal, Gaby, and in that light I thought the meaning behind my remarks to be "obvious".

| Shall we outlaw: | | struct Base { static int x; }; | | struct Derived : Base { | void f(int x) {} // Horrors! we're masking a base class member!

If you happen to pause a second and have look at the issue at hand, I'm confident that you'll see the difference between the case we're discussing and your example.

Of course I see the difference; they use different language features. I also see the analogy. I honestly don't see why the other case would warrant a diagnostic, while this one doesn't. Why is the difference between a member template parameter and a function parameter significant?

| }; | | Sorry for the sarcasm;

And, I'm pretty sure you know sarcasm is not appropriate for tehcnical discussions.

Sorry, I'm not at my best. Now returning to proper decorum. -- Dave Abrahams Boost Consulting www.boost-consulting.com

"Gabriel Dos Reis" <gdr@integrable-solutions.net> wrote

David Abrahams <dave@boost-consulting.com> writes:

| > What is bizarre and inconsistent about it and is not with the other | > alternatives you care to name? | | As I expected my example below to demonstrate, I think it's | inconsistent with the way names from non-dependent base classes are | dealt with when masked by function parameter names.

But function parameters are not template parameters vice versa. And you should not expect them to behave the same without providing technical reasons why such expectations may take place.

Strikes me that one of C++ s problems is that the namespace system (which itself partly resulted from confusion over C struct space) does not really formally address the fact that there are many different kinds of 'space', as examples(simplifying) object-space, type-space and template-space. (there are of course many more and more complexities). In addition to this there are many different programming 'contexts'. The name resolution/ hiding problems are a result of the fact that the spaces and contexts are informal (under the 'instinctive' banner). ie there are no written rules to formalise it. It might be interesting to try to understand/ hence formalise in the language, the programmers concept of 'space' when coding.(I believe that C++ already does this ... but under the banner of 'what is instinctive') As an example when coding in a function ( a context) using expressions, objects should(and do) come to the fore exemplified by C++s template deduction mechanism. However when actually writing a template definition( a different context) the template parameters themselves should come to the fore. In a sense this is the secret of the success of C++. ie There are many different contexts involved in coding, which C++ informally acknowledges. C++ has evolved to work in a particular way in a particular context. perhaps the idea of 'context' should be the arbitrator in these kinds of matters. Accept there are different contexts and spaces etc and formalise it. Formally work out the contexts and spaces, and arguments of this type would more easily resolve themselves. regards Andy Little

"Gabriel Dos Reis" <gdr@integrable-solutions.net> wrote in message news:m31xougf9h.fsf@uniton.integrable-solutions.net...

"Andy Little" <andy@servocomm.freeserve.co.uk> writes:

| "Gabriel Dos Reis" <gdr@integrable-solutions.net> wrote | > David Abrahams <dave@boost-consulting.com> writes: | > | > | > What is bizarre and inconsistent about it and is not with the other | > | > alternatives you care to name? | > | | > | As I expected my example below to demonstrate, I think it's | > | inconsistent with the way names from non-dependent base classes are | > | dealt with when masked by function parameter names. | > | > But function parameters are not template parameters vice versa. And | > you should not expect them to behave the same without providing | > technical reasons why such expectations may take place. | | Strikes me that one of C++ s problems is that the namespace system (which | itself partly resulted from confusion over C struct space) does not really | formally address the fact that there are many different kinds of 'space', as | examples(simplifying) object-space, type-space and template-space.

I'm sorry, I cannot undersand what you mean here.

The reference is in D&E 3.12 2/3rd way in (example struct outer{struct inner ... ) C had(has?) a different take on nested scopes. My point is more general. As laid out in the standard scope rules are dependent on context, however there is no overall vision of the concept behind why certain rules apply in a given context. What you have is a large number of bits and pieces hinting at the greater picture. As it stands you just have to know All the rules... nobody really does... hence confusion. In order to provide concepts rather than intuition as a guide to what rules to apply in a given situation, there should be some hierarchy of entities. I would put the first branch off the root of the tree at template versus non template. What that means is that on the template branch Dont expect the same rules as on the non template branch. From there function/class giving 4 contexts each again with its own rules. There is no crossover of rules from one context to another. However travelling down from the root there should be consistency of rules. One rule at top of the template branch Might be that 'new' template parameters should hide names from 'older' scopes. The rationale being that the template parameters are the most prominent part of a template definition. Whatever... template 'space' just aint the same as non template 'space'... so there should be no crossover of rules between different contexts. e.g you cant apply rules from a non-template function to a function template etc. This sounds like a recipe for an explosion of rules, but in fact many rules that apply in one context would *look very similar* to rules that applied in other contexts. However there is no Requirement that that must be the case. The tree models language syntax Template branch.: template<..> T ---> {function template} template<..> class -->{ class template} Non-template branch T --->{function} class ---> {class}; (Next branch from root to be 'concept' ...) ;-) All these entities are entirely different animals. Hope the above makes some sense. It does to me anyways :-) regards Andy Little

"Gabriel Dos Reis" <gdr@integrable-solutions.net> wrote in message news:m33c99g3ft.fsf@uniton.integrable-solutions.net...

"Andy Little" <andy@servocomm.freeserve.co.uk> writes:

[snip]

I usually describe that as "a long list of special cases", but I would not go that far saying that there is no overall vision of the concept behind why certain rules apply in a given context. Each time I've asked why certain rules don't apply, I think I've got an answer as to why.

It would be nice to have the vision explained... close by the C++ standard. (D&E is out of date. ARM is out of date) The standard is merely a collection of bits and pieces. Re your posts on comp.lang. One needs some didactics to point at students. "This is the concept behind this rule in this context" etc. Or this is the special case we are stuck with because the concept here has not been officially stated/confused.

But, there are situations where general rules are prevalent. It looks to me that this issue of members hide enclosing template-parameters is one where a general rule applies.

As I understand it currently base class names hide template parameter names. The problem with this is that it sees names in terms of parsing a grammar. "nested scopes" sees all scopes equal... takes no account of the concept 'What is a template all about'. 'What is a function body scope all about', 'what is a function parameter scope all about'. Each scope has a different character.C++ is a much richer language than to be constrained by dead grammar. Bjarne Stroustrup has commented on this in D&E. [snip]

I see now what you meant. Thanks for taking the time to clarify. Your picture leave out whether you consider nesting of scopes or not.

I have attempted to answer this on comp lang. Some scopes nest .... some dont. Each has its own character. There is a reason for that. Which is that C++ has tried to provide an intuitive language, one that feels write to the programmer. C++ reflects real world complexity. C++ has alway done things the hard way. I hope that concepts are the next major development. They really do ' feel right' to me for C++ :-) regards Andy Little

Gabriel Dos Reis wrote:

David Abrahams <dave@boost-consulting.com> writes:

| > What is bizarre and inconsistent about it and is not with the other | > alternatives you care to name? | | As I expected my example below to demonstrate, I think it's | inconsistent with the way names from non-dependent base classes are | dealt with when masked by function parameter names.

But function parameters are not template parameters vice versa. And you should not expect them to behave the same without providing technical reasons why such expectations may take place.

The fundamental difference between them is the places where they are declared. The scope of template-parameters *encloses* the template -- that is why members of non-dependent bases hide them -- whereas member functions *are enclosed* by the scope (non-dependent) base classes -- which is why those function parameters can hide members in base classes.

An action that would be consistent is to leave the current rule as is -- but some people been saying that that consistent behaviour goes against their intuition.

Apologies in advance if I jump in too quickly without fully understanding your point, but to me it seems that the current rule is consistent only when viewed from a certain point of view. The POV is that class B { typename X }; template< typename X > class Y // 1 : public B // 2 { // 3 ... }; You think that template parameter X is declared in line 1, then hidden by the base classes name introduced by line 2, thus non-accessible in line 3, right? You think of the name scopes to nest in the same order, do you? This is a reasonable point of view, actually it's the current language's interpretation of the topic, but it's not what I call intuitive. While it's technically correct, it still feels wrong - at least to me. I see things more from a visibility-point-of-view. I can't see the base class' names at 2, I have to go back (that is, I have to go one "scope" up) to see the names that are contained. This makes me think of the lines 1 and 2 to be swapped wrt the names they introduce into the scope of 3. To put it another way: The names of B are not declared at 2, and that line 2 is also not like a using declaration, although it might seem similar. If we use Dave's or my POV (and we are not alone ;), the advantages of scopes can be used in this example, too. The advantage is a smaller scope to search for a name: If you need to know what a name refers to, you just look into the scopes from the inside (nearest scope), to the outside. And it feels more natural to me if the scope of the template parameters is closer than the scope of the class' base(s). Regards, Daniel [I CC'ed Daveed as he might be interested in this, too, and I'm not sure if he's reading the boost list] -- Daniel Frey aixigo AG - financial solutions & technology Schloß-Rahe-Straße 15, 52072 Aachen, Germany fon: +49 (0)241 936737-42, fax: +49 (0)241 936737-99 eMail: daniel.frey@aixigo.de, web: http://www.aixigo.de

On Tue, Feb 17, 2004 at 12:29:25PM +0100, Gabriel Dos Reis wrote:

Daniel Frey <daniel.frey@aixigo.de> writes: | Apologies in advance if I jump in too quickly without fully | understanding your point, but to me it seems that the current rule is | consistent only when viewed from a certain point of view.

Someone has characterized it as inconsistent without giving details as to why; all I'm saying is that, if one accepts the principle that "scopes nest", then the rule is a logical consequence, hence consistent with that principle. Now, it might be that that logical consequence is unacceptable for some non-rational reasons, but that does not mean the consequence itself is inconsistent.

| The POV is that | | class B { typename X }; | | template< typename X > class Y // 1 | : public B // 2 | { // 3 | ... | }; | | You think that template parameter X is declared in line 1, then hidden | by the base classes name introduced by line 2, thus non-accessible in | line 3, right? You think of the name scopes to nest in the same order, | do you?

What I think is: If we accept the principle that scopes should nest, then we should accept its logical consequences; if don't want to accept those logical consequences, then we should abandon the idea that scopes nest. Which one do you want to pick?

I do find this (Gaby's) argument quite compelling. Nevertheless, despite its logic, human psychology seems resistant to it. I have been thinking about it a lot, and I think I finally nailed down why. Here's my story. Most scopes nest. Most of the time, introducing a new scope and new names happens in a way that all of the "new" names are visible at the point of introduction: int x; double y; void f( Foo x, Bar z ) { // x and z introduced here, _explicitly_ // x is a Foo, y a double, z a Bar } As homo sapiens we seem good at coping with this kind of nesting/shadowing, owing to the explicitness. However there are two kinds of language constructs which introduce names in an "invisible" way: using namespace foo; // many new names come in from foo struct Derived : Base { // many new names come in from Base These constructs are the potentially problematic ones from a psychological point of view. Now, for whatever reason, C++ has a "good" rule with regards to namespaces bringing in hosts of new names all at one. Witness: -------------------------------------------------- namespace Foo { void f() { cout << "1" << endl; } } void f() { cout << "2" << endl; } using namespace Foo; int main() { f(); } -------------------------------------------------- The compiler warns of an ambiguity. Saying "using namespace foo" does not shadow outer names, but rather brings them into equal contention in the current scope. The ambiguity diagnostics are good from the human perspective; these non-visible-at-the-point-of-introduction names are the ones we are apt to forget about. However inheritance is error-prone: -------------------------------------------------- struct Base { void f() { cout << "1" << endl; } }; ... // maybe many many lines of code here void f() { cout << "2" << endl; } struct Derived : Base { void g() { f(); } }; int main() { Derived().g(); } -------------------------------------------------- He who reads the code for Derived is apt to be mistaken about the f() being called. Unlike namespaces, inheritance introduces "invisible" new names in a way in which they hide/shadow the old names, rather than merely coming into equal contention (and causing an ambiguity error requiring the programmer to explicitly specify what is desired). The "moral" I am trying to sell is that this issue has nothing to do with templates. It is "inheritance" which is the error-prone construct here. If the rules for inheritance worked like the rules for "using namespace", then I think this "intuitive naming issue" would be solved. That said, I have not given any thought/consideration to what the implications would be for changing the way "inherited names" work in the language. Offhand it seems like currently legal programs would either stay legal with the same semantics, or become non-well-formed (compiler diagnostic). But I don't want to put the cart ahead of the horse yet. Does anyone agree with my "moral"? Let's stick with that question first. -- -Brian McNamara (lorgon@cc.gatech.edu)

Gabriel Dos Reis wrote:

What I think is: If we accept the principle that scopes should nest, then we should accept its logical consequences; if don't want to accept those logical consequences, then we should abandon the idea that scopes nest. Which one do you want to pick?

I like the idea of nested scopes, still the question is which scopes should be nested in which order. You seem to imply a contradiction where I don't see any. It's not the literal order of characters typed that *must* define the order of scopes, we have the freedom to think about this from a user's perspective and differ from the strict literal order. Inconsistent? Maybe, but if it helps to write code in the *sense* of nested scopes, I'm fine with it. This - of course - opens the question what this sense is, and I am certainly not qualified to define it. Maybe Bjarne could clarify what was originally intended. :) All I can say - and I seem to be consistent with several other folks - is, that I would like to see the scope for template-parameters be nested inside the base-class' scope, not outside. A small change which makes the code more managable by increasing the locality of names from a human reader's perspective.

If you think it is wrong, while being consistent with first principle, then propose a a new first principle. The worst thing to do, I believe, is to introduce barnacled hacks that comes from nowhere.

We need general rules on which to decide, we don't need barnacled hacks that would appear "intuitive" at a moment when we don't have a working definition of "intuitive".

I feel accused of "introducing barnacled hacks that come from nowhere". I don't think you meant to be insulting, but would you mind to assume that I'm not just seeing things from such a narrow perspective? I could also assume you are just "blindly applying rules without looking at the consequences" - but I think you have better reasons for your POV. I am actually trying to see where the idea of nested scopes comes from and what purpose it fulfills. This purpose is IMHO to keep the code readable with the smallest amount of code involved and the least number of interactions between the names of different scopes. This keeps the code readable, maintainable, etc. and leads to fewer surprises. In that sense, the current order of nesting scopes is consistent with the pure logic of nested scopes applied, but it's IMHO against the spirit of nested scopes.

If the scope of the template-parameters should come after the scope of base classes -- that is the way I understand you "is closer" -- then one would not be able to use template-parameters in base classes.

I don't understand this. The scope of the base classes identifiers is not the scope where the base classes of a class are enumerated. Regards, Daniel -- Daniel Frey aixigo AG - financial solutions & technology Schloß-Rahe-Straße 15, 52072 Aachen, Germany fon: +49 (0)241 936737-42, fax: +49 (0)241 936737-99 eMail: daniel.frey@aixigo.de, web: http://www.aixigo.de

{I will not try to take this any further as it is - as Dave pointed out - totally OT for boost. Anyway, I am just forwarding this message from Daveed as requested (my fault, I CC'ed him :), I suggest we take any further discussion to clc++m where Daveed opened an appropriate thread some time ago and has a chance to participate directly. Daniel} From: David Vandevoorde <daveed@vandevoorde.com> Date: February 17, 2004 11:33:04 AM EST To: Gabriel Reis <gdr@integrable-solutions.net> Cc: Boost mailing list <boost@lists.boost.org> Subject: Re: [boost] Re: [OT] Hiding template parameters [was: FC++ Formal review is...] (I'm not on the Boost list. Gabriel: Apologies if you get this twice. I don't know if you're on the list. So to be sure, I left you on the recipients' list.) On Feb 17, 2004, at 6:29 AM, Gabriel Dos Reis wrote: [...]

Someone has characterized it as inconsistent without giving details as to why; all I'm saying is that, if one accepts the principle that "scopes nest", then the rule is a logical consequence, hence consistent with that principle.

You're making one more assumption: That template parameters really have their own full-fledged scope. I say that they shouldn't, just as function parameters don't.

Now, it might be that that logical consequence is unacceptable for some non-rational reasons, but that does not mean the consequence itself is inconsistent.

The current situation is IMO unacceptable for _rational_ reasons, no matter what the language- legalistic arguments for it may be.

| The POV is that | | class B { typename X }; | | template< typename X > class Y // 1 | : public B // 2 | { // 3 | ... | }; | | You think that template parameter X is declared in line 1, then hidden | by the base classes name introduced by line 2, thus non-accessible in | line 3, right? You think of the name scopes to nest in the same order, | do you?

What I think is: If we accept the principle that scopes should nest, then we should accept its logical consequences; if don't want to accept those logical consequences, then we should abandon the idea that scopes nest. Which one do you want to pick?

As mentioned above, I disagree that these are the only options. But even if you insist on thinking in those terms, then there it is still possible to come up with a mechanism that matches what most of us expect: Simply consider the base class scope to enclose the scope of the derived class parameters.

| This is a reasonable point of view, actually it's the current | language's interpretation of the topic, but it's not what I call | intuitive.

One major problem is what is defined "intuitive" and when "intuitive" is defeated by logic.

Oh please. I notice you posted to the thread "Opinion sought on name hiding issue" in comp.lang.c++.moderated. Except for you, the other dozen participants seem to have no problem on agreeing what is intuitive. See above for what I think is the flaw in your logic.

| While it's technically correct, it still feels wrong - at | least to me.

If you think it is wrong, while being consistent with first principle, then propose a a new first principle. The worst thing to do, I believe, is to introduce barnacled hacks that comes from nowhere.

| I see things more from a visibility-point-of-view. I

"scopes nest" is visitbility-point-of-view :-)

| can't see the base class' names at 2,

yet, you see the following

struct A { struct iterator { }; };

struct B : A { vodi f(iteraror); };

don't you? Or are you proposing that to be ill-formed?

I also see: struct A { typedef int I; }; struct B: A { typedef long I; }; struct C: B { I x; // B::I hides A::I }; Same with template parameters: They hide the homonyms in farther scopes.

We need general rules on which to decide, we don't need barnacled hacks that would appear "intuitive" at a moment when we don't have a working definition of "intuitive".

| I have to go back (that is, I | have to go one "scope" up) to see the names that are contained. This | makes me think of the lines 1 and 2 to be swapped wrt the names they | introduce into the scope of 3. To put it another way: The names of B | are not declared at 2, and that line 2 is also not like a using | declaration, although it might seem similar. | | If we use Dave's or my POV (and we are not alone ;), the advantages of | scopes can be used in this example, too. The advantage is a smaller | scope to search for a name: If you need to know what a name refers to, | you just look into the scopes from the inside (nearest scope), to the | outside. And it feels more natural to me if the scope of the template | parameters is closer than the scope of the class' base(s).

If the scope of the template-parameters should come after the scope of base classes -- that is the way I understand you "is closer" -- then one would not be able to use template-parameters in base classes.

And you cannot: You can only use them to _name_ base classes. In fact, now that I reflect on it, base classes are not really "enclosing scopes" of derived classes. You can simulate the lookup that way, but you could just as well say that inheritance is a sort of "base class using-directive" (which is how many compilers implement it).

If you don't agree with the "scopes nest" principle, then propose another principle, but please don't label it "intuitive" :-)

On the contrary: If that what Daniel's intuition says, he should label it as intuitive.

If I were to design a new programming language, I'm not sure I would just stick to "scopes nest", but here we're not designing a new programming language. IMO, one of the the worst things we can to the already complex C++ scope rules, is to come up with yet another special scope. At this point of complexity, following logical consequences is less evil than "intuitive" hacks where we don't have working definition of "intuitive".

You seem to imply that scope nesting is the only mechanism that directs C++ name lookup. That is not the case though (using directives, using declarations, and ADL don't work that way; inheritance doesn't really work that way either). Anyway, as many have pointed out, enabling sound C++ engineering requires the current situation to be fixed. Daveed

Gabriel Dos Reis wrote:

David Abrahams <dave@boost-consulting.com> writes:

...

Shall we outlaw:

struct Base { static int x; };

struct Derived : Base { void f(int x) {} // Horrors! we're masking a base class member!

If you happen to pause a second and have look at the issue at hand, I'm confident that you'll see the difference between the case we're discussing and your example.

...

};

I think that you win the consistency argument. struct X { typedef void Y; typedef void Z; }; struct Y { }; template<class Z> struct V: public Y, public X { Y y; // X::Y Z z; // X::Z }; Very counter-intuitive since C++ code usually respects locality of reference, but consistent.

At 21:53 13/02/2004, you wrote:

Mat Marcus wrote:

<snip>

(while compiling monad3.cpp) monad3.cpp:82: error: no type named `second_type' in `struct OutputMonad<std::string>' This is a template shadow parameter issue, related to the fact that OutputMonad<> defines an inner type called "M", but a template member functions uses M as template parameter as well:

struct A { typedef int M;

template <class M> //* void foo(void) { M m; // which M is this? } };

I know the C++ committe is discussing this issue at this moment. The argument would be that "M" names the typedef because it's "more stable" than the template parameter (which could get renamed in an out-of-class definition). See also http://gcc.gnu.org/PR13967 for a detailed discussion. It's possible that GCC will emit a warning about this borderline case of shadowing, one day. This is worked around easily, by renaming the innest template parameter, in line 107 and 110, and all its subsequent uses.

I can't believe that??? So for int i = 12; { int i = 123; //* std::cout << i; } should I expect an output of 12 because the other i is more stable? Maybe I'm wrong, but it seems stupid to have one rule for one case, and a different rule for the effectively the same case. The sensible opinion seems that the lines marked //* take precedence, albeit ideally with a warning. --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.573 / Virus Database: 363 - Release Date: 28/01/2004

On Sat, Feb 14, 2004 at 09:08:19AM +0900, Darren Cook wrote:

...

The documentation at http://www.cc.gatech.edu/~yannis/fc++/boostpaper/ provides a walkthrough and reference for most of the main features of the library. There are also some pointers out to other papers and documentation on the main FC++ web site: http://www.cc.gatech.edu/~yannis/fc++/

A minor point, but it confused me: the links in the bibliography to FC++ web site point to the boostpaper URL.

Fixed. (Wow--In the BoostBook XML, I accidentally had a few instances of <ulink linkend="httpyadda"> rather than <ulink url="httpyadda"> and apparently <ulink>s without "url" attributes will link back to the current document.) -- -Brian McNamara (lorgon@cc.gatech.edu)

On Mon, Feb 16, 2004 at 09:31:52PM -0500, Jim Apple wrote:

Here's a laundry list of complaints. After having read the documentation, I think FC++ is a great idea, but I don't think it's boost-compatible yet.

Some good food for thought here...

This library is difficult to review because it is so large. Perhaps it could be split into 1. Functoids, including currying and thunks 2. Lists 3. Lambda 4. Monads

I realize there is some overlap, but it could help with making everything more understandable. For instance,, presently, boost::lambda objects can be converted to boost::function objects. These could be one library, but I think they're quite nice as is.

It occurs to me only now that it would be really useful to have a "conceptual dependency hierarchy", since this will compartamentalize the library into more manageable sub-chunks. Let's see if I can generate one now: Direct Functoids Lists / / | \ \ | / / | \ \ | / / | \ \ | Lambda Full Misc Lib Indirect List Library | Functoids Functions Functoids Functions | and Datatypes | Monads In practice, there are a number of non-obvious implementation dependencies among these chunks (some of which are more easy to eliminate than others). Here is also the capsule summary of what each part contributes: Direct Functoids Result type computation ("typeof"). The std::result_of has now become the standard way to do this. Practically everything depends on this, as you can't do higher-order polymorphic functions without a mechanism for computing return types. Full Functoids All the syntax niceties, like currying, infix, lambda's []/%. Indirect Functoids Very much like boost::function. Lambda and Monads Just that. Not yet proven "useful". Lists List data structures. Useful in the functional sense (cons() and tail() are O(1)). Provides one easy way to do lazy evaluation. Misc Lib Functions/Datatypes Lots of useful operator-functoids and combinators. Also some datatypes, like maybe and by_need. List Lib Functions Lots of functoids that work on lists. The last three are the bulk of the "library", whereas the rest are more "framework". It might be more manageable to consider each of these categories individually. (Perhaps I could/should start seven separate email threads--one for each category. Good/bad idea, anyone?)

I'm concerned about documentation that says "Look at the Haskell Report."

Concerned from the "Brian is too lazy to provide ample documentation" standpoint, or something more? Regarding your comments about names of various entities: I am willing to change names if people find other names more attractive. -- -Brian McNamara (lorgon@cc.gatech.edu)

On Tue, Feb 17, 2004 at 05:08:21PM -0500, Jim Apple wrote:

...

(Perhaps I could/should start seven separate email threads--one for each category. Good/bad idea, anyone?)

Well, I think it's a great idea! :-)

I think I will do this later tonight. There are also a couple of other topics that aren't "library components" per se which deserve mention.

...

...

I'm concerned about documentation that says "Look at the Haskell Report." ... There are lots of people completely unfamiliar with haskell syntax.

This is a good point.

"Could the functoids with arity attached to their names have that removed, like boost::function? "

which is more a technical question. Is it possible for curryN to be just curry?

In the general case, yes. (Also, curryN doesn't really exist anymore; it's been replaced by thunkN in the Boost version of FC++.) -- -Brian McNamara (lorgon@cc.gatech.edu)

Full Functoids Full functoids are "wrappers" that add various capabilities to basic functoids. The capabilities added are - currying (partial application) - infix call syntax (x ^f^ y) - lambda awareness (ability to call with [] or % to delay evaluation) - smartness (knowing the arity of the underlying functoid) - result_of (converting FC++ sig information to result_of info) These capabilities are orthogonal to one another (they are all "wrapped up" in the fullN classes merely as a "convenient" way to add all the features at once). In principle, these wrappers could be applied to any function-object that supports return-type deduction--not just FC++ functoids. In practice, however, a number of issues arise: - currying: If the wrapped functoid takes non-const reference parameters, then currying won't work. I think the current implementation would try to create a "reference to a reference". Apart from this implementation issue, currying with reference parameters creates object-lifetime issues if the curried function object outlives the captured reference. Little messes like this are one of the reasons FC++ tries to stick with "pure" (effect-free) interfaces. - lambda awareness: While any function object could be wrapped to support our f[args] "delayed call" syntax, the result would be an FC++ lambda expression, which is only useful in the context of an FC++ lambda. - smartness: FC++ has a totally different view of "arity" compared to, say, boost::lambda. (See http://www.boost.org/libs/lambda/doc/ar01s05.html#sect:placeholders for some discussion about boost::lambda.) This isn't an inherent problem, but it could cause confusion owing to the different designs. - result_of: This feature only makes sense for FC++ functoids (as it converts our sigs into result_of information). Also, while none of the rest of the library depends on these features in principle, in practice the implementation of many functoids implicitly expect a number of these features to be there (especially currying). As a result, the rest of the library has a strong implementation dependency on full functoids. In my opinion, the syntax sugars provided by full functoids are very valuable; it's no fun to do functional programming when you have to use huge syntax to do simple things like currying. -- -Brian McNamara (lorgon@cc.gatech.edu)

On Thu, Feb 19, 2004 at 02:58:12AM -0500, Gennadiy Rozental wrote:

C++ standard introduces and STL uses notion of function object a generic abstraction for function. By design it is structure type with non-template operator(), where types of the arguments and result types are unambiguously defined by the type of function object (it could be hardcoded types or types dependent on structure template parameters). So let's call it monomorphic function object, opposed to the alternative design for function object, which uses member template operator() which we will call polymorphic function object. Polymorphic function objects in some ways better/more powerful than monomorphic ones. But there is an issue of result type deduction, which require special support. Solution is described in mention by you paper and your library implements it (as I understand it).

Now the question is why would we need separate notion of functoid, while we already have powerful abstraction for function objects presented by boost::function? Why boost function does not present this functionality (if it does not, I am not sure)? This looks like obvious place to implement this solution: you provide helper facilities to make it easier to write polymorphic function objects and you make boost::function polymorphic function object aware (even though it still be a model for monomorphic function object). This will cover indirect functoids also.

I am not sure exactly what you are asking/suggesting, but I think I get the gist of it enough that hopefully I can answer anyway. As a quick aside, let me define "adaptable" to mean a function object which exports a way to ask about its return type (in the same general sense as http://www.sgi.com/tech/stl/AdaptableBinaryFunction.html ). Also let me define "rebindable" to mean function objects which are variables that can be bound to different function objects during the variable's lifetime, as in boost::function<int (int,int)> f; f = _1+_2; // f is now "plus" f = _1-_2; // f is now "minus" Given those definitions, functon objects can be generally classified along 3 different axes: - they can be monomorphic or they can be polymoprhic - they can be adaptable or not - they can be rebindable or not "Direct functoid" is just the term we use in FC++ to describe adaptable, non-rebindable function objects. "Indirect functoid" is the term for "adaptable, rebindable, monomorphic" function object. boost::function objects are classified just like indirect functoids (adaptable, rebindable, monomorphic). My goal for the various "functoid" terms is not to introduce new vocabulary, but rather merely to simplify the exposition. Hopefully "standard" terms for these concepts will arise soon (I don't think they have yet). In any case, a "standard" implementation for result-type deduction has already arisen (result_of), and FC++ supports this standard. (Let me know if I still have not answered/addressed your question.)

(BTW why do you need virtual for it's implementation - I did not see single one in boost::function)

This amazes me, though http://www.boost.org/doc/html/function.misc.html#id2517594 maybe explains it. I still cannot quite fathom how "virtual" is avoided in boost::function, but I look forward to exploring the code or chatting with Douglas Gregor about it. :) In any case, you need some kind of way to do the indirection, and we chose virtual functions as our implementation.

Currying: I have a question: why boost::function does not provide currying internally? Why do we need boost::bind? IMO polymorphic function object currying should be incorporated in boost::bind.

I see that others are currently discussing the pros and cons of this in other messages on the list.

Smartness: I believe boost::function interface may be extended a little if we find your addition convenient. But as for now arity seems to be good enough.

Indeed; also, FC++ and boost have a different notions "arity", so this doesn't "blend" well into boost functions.

Infix syntax: I am not sure I like the idea in a first place. But it could be discussed and added

It is more useful when you have "named" operators as in FC++, simply because x ^plus^ y is often more readable/natural than plus( x, y ) It may also be attractive to OO people, who have gotten into the now-fashionable habit of avoiding member functions. Switching from the member notation Shape s; Point p; if( s.contains(p) ) ... to the non-member notation if( contains(s,p) ) ... loses the subject-verb-object order; using function objects that support infix, however, lets you say if( s ^contains^ p ) ... which some people may find attractive. Thanks for your comments! -- -Brian McNamara (lorgon@cc.gatech.edu)

On Thu, Feb 19, 2004 at 08:10:54PM +0200, Peter Dimov wrote:

Brian McNamara wrote:

...

Indeed; also, FC++ and boost have a different notions "arity", so this doesn't "blend" well into boost functions.

Can you elaborate please?

Yes; please see a message in the indirect functoids thread, where I give the example:

For example, when you have implicit currying, you expect that

f(x,y)(z) == f(x,y,z)

to hold. But look:

int i, j, k; _1(i, j, k) // returns i, discards j and k _1(i, j)(k) // Oops! Not the same!

-- -Brian McNamara (lorgon@cc.gatech.edu)

On Friday 20 February 2004 04:35 am, Gennadiy Rozental wrote:

I do not think we understand each other. Let me rephrase. My position is that polymorphic function object support does not belong to the library dedicated to "functional programming", even though I propose to update boost/functional.hpp header ;))

(BTW it would really help for the whole library review, if you could provide a little introduction what is a "functional programming" in a first place; how it differ from other programming styles and what is the place of your library in this - I mean what purpose does it serve, what solution does it provide).

I think this is way out of scope for the library. Functional programming has been around for ages, and it shouldn't be up to Brian to introduce and explain all of it in his introduction.

Also after previous letter I found that you actually provide a lot more "functoids" within the library, then counterparts to the STL functional.hpp function objects. These should go in FC++ specific headers (and I mean headers - one per name).

If I need to include boost/fcpp/plus.hpp separately from boost/fcpp/minus.hpp, I'm going to very, very unhappy. I agree with breaking up large components intro multiple headers, but when we get 20 headers with 10 lines of code in them each, all related, we've gone too far. Doug

On Sat, Feb 21, 2004 at 10:19:02AM -0000, Andy Little wrote:

Functional programming' basically separates 'Joe User' from 'Johnny Guru'. ... I am being too dismissive about 'Functional programming' . Basically it is ... [ much more of the "cynic's view of FP" elided ] ... Execution order not guaranteed... :-) No assignment ...No variables... (hmm.. because Execution order not guaranteed)

IOW its an entirely Different Programming Paradigm . FP doesnt have any OOP concept, its not 'close to the machine' I'm not against it outright(Some of the above ideas are very useful in certain areas...e.g parallel processing) but it seems daft to mix it with C++.

I agree with some of your comments here about FP in the C++ setting. Specifically, absolute purity (side-effect freedom) trades away assignment and mutable variables in order to buy freedom of evaluation order, which can be useful for compiler optimizations, especially in parallel settings. Since C++ clearly has effects and assignment (and the C++ compiler is not going to employ the same kinds of optimizing transforms common in pure languages), purity is not going to be a big win here. But there's more to FP than just purity. Higher-order functions, lambda, and closures let you write code in a more applicative style, which can raise the overall level of abstraction. Features like these do "mix well" with C++ or other OOP languages. Indeed, recent OOPLs have been including various functional features: for example, Python has lambda and list comprehensions, and Scala has lambda as well as generalized monadic comprehensions in its "for" statement. Functional programming is a wide umbrella, and though not all of it mixes well with OOP, a number of features do work synergistically between the two paradigms. As an aside, it occurs to me that I've been remiss in failing to mention "algebraic datatypes" and "pattern matching". These are other features commonly associated with FP, and these features also mix well with OOP (see e.g. Scala and Nemerle). FC++ doesn't include any support for these features, but I should have at least mentioned them in my other messages about FP. Thanks to everyone for your comments so far! Remember that the review for FC++ has been extended (until March 1); comments along the lines of http://www.boost.org/more/formal_review_process.htm#Comments are very much appreciated. -- -Brian McNamara (lorgon@cc.gatech.edu)

On Wed, Feb 25, 2004 at 10:07:12AM -0000, Andy Little wrote:

"Brian McNamara" <lorgon@cc.gatech.edu> wrote

...

I agree with some of your comments here about FP in the C++ setting. Specifically, absolute purity (side-effect freedom) trades away assignment and mutable variables in order to buy freedom of evaluation order, which can be useful for compiler optimizations, especially in parallel settings. Since C++ clearly has effects and assignment (and the C++ compiler is not going to employ the same kinds of optimizing transforms common in pure languages), purity is not going to be a big win here.

In that case is not FC++ rather a grand title for the library? There is a strong sense in the documentation of attempting to 'hijack' (your word) C++ for your own purposes.IOW you don't actually like C++ much. For example the

If the documentation suggests that the authors don't like C++, then this is a problem. (Do others of you have that sense?) I disagree that "not liking C++" logically follows from "hijacking". Many of the successful Boost libraries "hijack C++ for their own purposes" (e.g. boost::lambda, with expression templates, or boost::mpl, with compile-time computation). I don't think this implies that Jaakko, David A., etc. don't like C++.

lambda library uses the term function-objects rather than functoids. Its a small difference but significant. You appear to be fighting C++ at every level from the compiler upwards. You appear to have got things upside down. It would seem more appropriate to call C++ process to do the dirty work from within the pure FP, rather than embedding FP in C+. That would be interesting, and would allow each language to perform at its best.

It is not clear to me that one way is "better" than the other. (Both pure FP nested within an impure language and impurities nested within pure FP are interesting, IMO.)

...

But there's more to FP than just purity. Higher-order functions, lambda, and closures let you write code in a more applicative style, which can raise the overall level of abstraction.

Ok I am trying to find where or why I might find this library useful. Higher order functions are essentially functions that manipulate other functions? Say to build up an expression from some token, object sequence either(both) at compile time or(and) at run time according to some rule. Presumably I am allowed to manipulate objects as well as functions in FC++.?

Yes and yes.

As I understand lambda in Context of C++, it is basically an alternative syntax to writing a function in the traditional (C/C++) sense, and achieves the above?

Kinda, yes. Here's an analogy that's rooted in C++. Consider iterator adapters. They let us view a container (or sometimes multiple containers) through a "lens" that adapts our view of what's in the container. For example, we might have a vector of pointers (vector<Obj*>), but we are interested in the objects pointed at by the pointers. So we use an iterator adapter which automagically dereferences the pointers, so that the value_type of the iterators is "Obj" rather than "Obj*". When our iterators "aren't quite right" for what we want to do, we can use iterator adapters to "fix" them for our current purposes, rather than writing entirely new iterator classes or writing special-purpose code (like an Obj*-comparator) to deal with things being not-quite-right. Now, higher-order functions (HOFs) are like "function adapters". If you have a function (or sometimes multiple functions) which almost does what you want, you can simply use HOFs to adapt them (via composition, currying, binding, etc.), rather than writing entirely new functions or writing special-purpose code to deal with the situation.

Raising the level of abstraction seems to be an end in itself. And for simplicity it requires that all entities conform. The C++ abstraction mechanisms make allowances for entities that dont conform.That is it can reach in at a low level. How does FP see non conforming entities.. Are they an impurity which is banned?

I am not sure exactly what you are asking, but... FC++ cannot prevent you from creating functoids with side-effects. If you do so, it is up to you to make sure the effects "happen at the right time" (just as always in normal C++ code). FC++ currently doesn't allow reference parameters. However discussions in another thread appear to have found a solution which would loosen this constraint. If by "non conforming entities" you mean "functions with side-effects", for the most part, FC++ cannot see/detect these things at all.

I do not know what 'comprehensions are. . I like (think in terms of ) objects(C++ classes). They play a far more important role in my day to day programming than functions. but they appear to be anathema to FP. Hence it seems to be an impoverished language. Its good with the abstractions . but cant deal very well with 'meat'.

Objects are not anathema to FP. (But _mutable_ objects are anathema in _pure_ FP.) Both paradigms (OOP and FP) deal with both data and behavior. In OOP, the data (objects) are the focus. In FP, the behaviors (functions) are the focus. A lot of this discussion may be "too abstract" to be useful. To make things more concrete, in a little while I will post a short example illustrating some differences of how the paradigms deal with data and behaviors. -- -Brian McNamara (lorgon@cc.gatech.edu)

Brian wrote: [snip]

The FP programmer, being enamored with higher-order functions (HOFs), notices that all these tree functions have a common pattern. Rather than write tons of miscellaneous tree functions, he instead writes one function to do them all:

// using FC++ struct FoldTree { template <class L, class N, class SPT> struct sig : public fun_type<L> {}; template <class L, class N, class T> L operator()( const L& leaf, const N& node, shared_ptr<Tree<T> > t ) const { if( !t ) return leaf; else return node( t->data, (*this)(leaf,node,t->left), (*this)(leaf,node,t->right) ); } }; typedef full3<FoldTree> fold_tree_type; fold_tree_type fold_tree;

[snip]

In this sense, HOFs and iterators (or iterator adapters) are very similar. The FPers will create _functions_ (like fold_tree) to abstract away in order to create generalized algorithms, whereas the OOPers will create _objects_ (like iterators) for the same general ends. Both strategies have the same general flavor, just taken with a different tack.

I think this is a "problem." That most senior C++-ers already do use the Visitor pattern, or an Iterator model, using function objects as callbacks. And, with Boost.Lambda, they can even construct those visitors on-the-fly. I think one has to dive into the (abstract) beauty of catamorphisms in general to see what an FP mindset can bring to the this table of traversal. Perhaps traversal is not the sexiest introductory field to FP, without deeper appreciation of monads and other functorial constructs, since traversal is already handled, conceptually by a C++-programmer - Iterators; seen it, done it...? /David

On Thu, Feb 26, 2004 at 02:39:40PM +0200, Peter Dimov wrote:

Brian McNamara wrote:

...

The FP programmer, being enamored with higher-order functions (HOFs), notices that all these tree functions have a common pattern. Rather than write tons of miscellaneous tree functions, he instead writes one function to do them all:

// using FC++ struct FoldTree { template <class L, class N, class SPT> struct sig : public fun_type<L> {}; template <class L, class N, class T> L operator()( const L& leaf, const N& node, shared_ptr<Tree<T> > t ) const { if( !t ) return leaf; else return node( t->data, (*this)(leaf,node,t->left), (*this)(leaf,node,t->right) );

The order of evaluation is unspecified, so if the subexpressions have side effects, the code is not portable. But the programmer wouldn't know it if it happens to work on the specific platform/compiler/version/moon phase.

Pure FP doesn't suffer from these problems, but C++ does. As a simple example, try to print a tree using fold_tree.

Yes. I think I mentioned it in an earlier message, but it bears repeating. When you use FP, HOFs make it harder to specify and reason about side-effects, and currying and lazy evaluation make it harder to specify and reason about object lifetimes. These are among the reasons the FP people usually go for purity.

...

With fold_tree(), it is easy to express new tree functions on the fly. For example, here is a function to get an inorder travseral:

fold_tree( fcpp::list<T>(), lambda(D,L,R)[ cat[ L, cons[D,R] ] ] )

You got me confused for a while with the omission of the third argument. I then spent a while trying to understand how the lambda thing would get passed around. ;-)

Oops! My bad. I'm actually trying to make the examples accessible, rather than inpenetrable. :) When you get accustommed to automatic currying, you don't ever notice that you're using it.

Anyway, the obvious question #1 is, of course, why should I use a fcpp::list instead of std::list,

Here the answer is because I'm building the the list "pure"ly, so fcpp::list is probably much more efficient (fcpp::cons is constant time, whereas the same operation on a std::list is potentially O(N)).

and obvious question #2 is why I should copy the tree into a list at all, when I could've just flattened it into a sequence with iterators. To be fair, you do point that out yourself.

Good point; I was just keeping the example simple. (In fact, when you use fcpp::lists with lazy evaluation, lists effectively become input iterators. It would be interesting to implement inorder-iterators for this tree both with and without FC++. Hmm. How easy/hard would it be to build bidirectional iterators for a tree like this? I've never tried. If you have the time to cook some up (using Boost), I would enjoy seeing them, and comparing them to whatever I could cook up with FC++.)

However iterators do avoid the unspecified traversal order problem. Not that you couldn't have created inorder_fold_tree, of course, but iterators make it much harder to accidentally leave the order unspecified.

Time out. This: fold_tree( fcpp::list<T>(), lambda(D,L,R)[ cat[ L, cons[D,R] ] ] ) always yields an inorder traversal. And: // preorder fold_tree( fcpp::list<T>(), lambda(D,L,R)[ cons[ D, cat[L,R] ] ] ) // reverse-inorder fold_tree( fcpp::list<T>(), lambda(D,L,R)[ cat[ R, cons[D,L] ] ] ) // etc. The lambda uses no effects, and thus the order of evaluation inside the fold_tree() doesn't matter.

...

fold_tree( 0, lambda(D,L,R)[ plus[ 1, max[L,R] ] ] )

Okay, but this is easy to do even with boost::bind. We want unique selling points. ;-)

I do see the smiley, but just to be abundantly clear, I was selling HOFs ("Here's an example that shows how higher-order functions are useful..."), and boost::bind is indeed just such an entity. -- -Brian McNamara (lorgon@cc.gatech.edu)

On 02/25/2004 10:22 PM, Brian McNamara wrote:

On Wed, Feb 25, 2004 at 12:24:40PM -0500, Brian McNamara wrote: [snip]

...

A lot of this discussion may be "too abstract" to be useful. To make things more concrete, in a little while I will post a short example illustrating some differences of how the paradigms deal with data and behaviors.

Ok, as promised...

[snip] Maybe an advantage of FC++ over other libraries could be illustrated with a truly functional program written in lisp. For example, as I mentioned in: http://article.gmane.org/gmane.comp.lib.boost.devel/23704/match=mixin , it seemed to me that FC++ would be ideal for doing what was described in the article mentioned. This is because the dynamic inheritance was expressed using lisp. I've attempted to do this in c++ in code located at: <boost_files>/regexp_first.zip , but I was hoping FC++ could maybe do better. One problem (maybe not too big of a problem) with the regexp_first code is that there's a pointer from subtype to supertype and back (more memory) and the virtual functions in supertype must be delegated to in the subtype (more programmer time). I don't know if FC++ can improve on that, but I'd like to see if it can. Brian, do you have any ideas on how to do it better in FC++?

"Brian McNamara" <lorgon@cc.gatech.edu> wrote

On Wed, Feb 25, 2004 at 12:24:40PM -0500, Brian McNamara wrote:

...

Objects are not anathema to FP. (But _mutable_ objects are anathema in _pure_ FP.) Both paradigms (OOP and FP) deal with both data and behavior. In OOP, the data (objects) are the focus. In FP, the behaviors (functions) are the focus.

A lot of this discussion may be "too abstract" to be useful. To make things more concrete, in a little while I will post a short example illustrating some differences of how the paradigms deal with data and behaviors.

Ok, as promised...

Thanks for the Examples. Apologies for not giving more constructive feedback on them. I will have to spend a lot more time reading the docs to understand all of it.... but I am getting an inkling of what the thing is about. Maybe if I had to manipulate generic trees I would be keener yet. Meanwhile I have spent some time on it but gradually got carried away with an expression tree. Somewhere in the FC++ Docs is a call for applications. A calculator ala C++3rd Ed examples might be just what is needed as a, less abstract yet, working example. This is my guess at one way the FP expression might look(Guessing from a quick look at Haskell): Expr <-- TokenStream. //(Whatever :-) ) FWIW below is a very OOP attempt(not tested). OTOH maybe this is not FC++ arena? Thanks again for time on the examples.. Appreciate I may be straying... :-) regards Andy Little. ------------------------ template <typename TokenStream> struct ExprTree; //wrapper for the expression, maybe needs a Rule struct Expr; // data, ops etc template<typename TokenStream> struct ExprTree{ Expr& node; ExprTree (TokenStream& in); // build in ctor }; //generic expression tree node struct Expr { //ideally leaves us with one Prim node in ExprTree virtual Expr& optimise()=0; }; // null statement struct Empty : Expr{ Expr& optimise(){return *this;} }; // data template <typename Value_type> struct Prim : Expr{ typedef Value_type value_type; Value_type m_value; Prim(Value_type value_in):m_value(value_in){} Prim& optimise(){return *this;} }; //ops template <typename Op> //e.g plus,divide etc struct Binary : Expr{ typedef Op type; Expr& left,& right; Binary(Expr& left_in, Expr& right_in) : left(left_in.optimise()),right(right_in.optimise()){} Expr& optimise() { // Expr* Op::operator()(const Expr&,const Expr&); Expr* t = Op()(left,right); return t? delete this,*t : *this; } ~Binary(){ delete &left, delete &right;} }; template <typename Op> //e.g minus, inc , call etc struct Unary : Expr{ typedef Op type; Expr& expr; Unary( Expr& expr_in) : expr(expr_in.optimise()){} Expr& optimise() { Expr* t= Op()(expr); return t? delete this,*t : *this; } };

On Thu, Feb 26, 2004 at 12:04:19PM -0600, Larry Evans wrote:

...

<boost_files>/regexp_first.zip , but I was hoping FC++ could maybe do better. One problem (maybe not too big of a problem) with the regexp_first code is that there's a pointer from subtype to supertype and back (more memory) and the virtual functions in supertype must be delegated to in the subtype (more programmer time).

I don't know if FC++ can improve on that, but I'd like to see if it can. Brian, do you have any ideas on how to do it better in FC++?

Brian wrote: Sorry, no. Indeed, I cannot imagine _any_ solution in C++ which doesn't involve "manually" writing the delegation methods as well as keeping pointers in both directions.

I changed my mind, after talking to Yannis about it in front of a white board. :) Ok, so you can do something clever with "delegation" or "dynamic inheritance" or the "state pattern" or whatever you want to call it if we change the constraints of the problem some. I am still fooling with the solution, so I'm not sure of all the design constraints yet. I also want help smoothing it out. If class "C" wants to delegate method "f" to class B, then below is code to do it. I used boost::{lambda,function} rather than FC++ in the example. The basic idea is to get rid of member functions (a recurring motif) in favor of function objects. Annoyingly, you have to pass a pointer-to-the-current-object as an extra argument, which I can't seem to get rid of right now. (Ideas?) #include <iostream> using std::cout; using std::endl; #include "boost/lambda/lambda.hpp" #include "boost/lambda/bind.hpp" #include "boost/function.hpp" using boost::function; using boost::lambda::_1; using boost::lambda::_2; using boost::lambda::constant; struct B { int x; function<void(B*,int)> f; // instead of void f(int); B( int x_, function<void(B*,int)> f_ ) : x(x_), f(f_) {} }; struct C : B { int y; function<void(C*,int)> g; // instead of void g(int); C( const B& b, int y_, function<void(C*,int)> g_ ) : B(b), y(y_), g(g_) {} }; int main() { // make two different Bs with different behavior B b1( 1, cout << constant("x is ") << bind(&B::x,_1) << " and arg is " << _2 << "\n" ); B b2( 2, cout << constant("my x is ") << bind(&B::x,_1) << " and my arg is " << _2 << "\n" ); // make a C that delegates f() to b1 C c( b1, 3, cout << constant("y is ") << bind(&C::y,_1) << " and arg is " << _2 << "\n" ); // the &c extra argument is annoying, grr c.f(&c,5); c.g(&c,5); // now delegate to b2, instead dynamic_cast<B&>(c) = b2; c.f(&c,5); c.g(&c,5); } -- -Brian McNamara (lorgon@cc.gatech.edu)

On Fri, Feb 27, 2004 at 09:07:10PM -0600, Larry Evans wrote:

On 02/27/2004 04:52 PM, Brian McNamara wrote:

...

...

On Thu, Feb 26, 2004 at 12:04:19PM -0600, Larry Evans wrote: [snip]

...

If class "C" wants to delegate method "f" to class B, then below is code to do it. I used boost::{lambda,function} rather than FC++ in the example. The basic idea is to get rid of member functions (a recurring motif) in favor of function objects. Annoyingly, you have to pass a pointer-to-the-current-object as an extra argument, which I can't seem to get rid of right now. (Ideas?)

[code snipped]

I'm having a hard time understanding what the code does except what amounts to using function pointers (or boost::function objects) instead of methods. The code is dynamic because the function pointers can be updated at any time. Am I missing something?

I was wanting something more like section 12.7 of _Design&Evolution of C++_. At first, I guessed that maybe that's what you were doing, but then I wondered why C inherits from B instead of C just contains a B*, as in section 12.7.

I was indeed trying to model that example. For those of you without D&E at hand, here's a slightly-modified version: // A stab at delegation that Stroustrup eventually rejected class B { int x; void f(int); }; class C : *p { // delegation-inheritance B* p; int y; void g(int); }; void f( C* q ) { q->f(3); // automagically means q->p->f(3) } Since there is no such construct in C++, you have to manually write the call-forwarding routines: // actual C++ class B { int x; void f(int); }; class C { B* p; int y; void g(int); // as before void f(int i) { return p->f(i); } // manual delegation }; In both cases, "C" can dynamically "change behavior" by changing "p", and thus delegating the work of "f" to a different "B" object. Note that the delegate (B) has no way to "get back" to the original object (C). You mentioned a couple problems with the regexp code:

there's a pointer from subtype to supertype and back (more memory) and the virtual functions in supertype must be delegated to in the subtype (more programmer time).

So I tried to find a solution where you didn't have to write any call-forwarding routines, and where the delegate doesn't have to store a pointer back to the original object. I used inheritance to solve one of the problems (no more need to write a forwarding function: "C" inherits "f"), and used an explicit extra "self" pointer parameter to solve the other problem (way to "get back" to the original object) as well as to recover the information intrinsically lost when switching from member functions to function objects. Now "C" can change the "object delegated to" just by assigning a different delegate object to its own "B" subobject: dynamic_cast<B&>(c) = b1; // delegate to b1 ... dynamic_cast<B&>(c) = b2; // delegate to b2 That said, I admit I didn't look very carefully at the regexp code, so I may be missing some important aspect/intention of what you want to do with all of this. So let me know, both if my explanation above clarified what I was trying to do, and if it does or does not accomplish what you're seeking. -- -Brian McNamara (lorgon@cc.gatech.edu)

On 02/27/2004 11:01 PM, Brian McNamara wrote: [snip] One thing I forgot to emphasize is that this design pattern seeks to, in essence, replace a tree<A>, with a tree<B> where A and B represent inheritance heirarchies. In addition, the instance of tree<A> transformed should remain untouched. For example,

On 02/28/2004 08:08 AM, Larry Evans wrote: the tree<A> in the regexp_first example is the regexp_tree_top and tree<B> is regexp_first_top. The whole idea is to perform passes over tree<A>, adding information (via attributes attached to nodes of tree<A> by a pointer (e.g. regexp_tree_top::my_subtype ) used by subsequent passes (regexp_nilable_top::my_nilable is used to calculate regexp_first_top::my_first), and then to remove the information when it's no longer needed. Hope that clarifies things.

On 02/28/2004 08:24 AM, Larry Evans wrote:

...

On 02/28/2004 08:08 AM, Larry Evans wrote: [snip] Ok, maybe just one more thing: tree<B> has additional virtual [snip] OK, YAT (Yet Another Thing), the tree<B> virtual functions must be able to treat their children in the tree as tree<B> instances also. For example, the tree_nilable_seq::calc_nilable is defined in terms of the my_nilable's of it's children. However, in order to access it's children, it has to go through it's "superclass" regexp_tree_seq; hence, there's got to be a pointer to the superclass, and in order to attach the extra information for a particular pass,

On 02/28/2004 09:05 AM, Larry Evans wrote: there's got to be a pointer from the superclass to subclass. Therefore, I don't see a way to avoid the extra memory.

On 02/28/2004 06:29 PM, Brian McNamara wrote:

On Sat, Feb 28, 2004 at 09:38:26AM -0600, Larry Evans wrote: ...

...

OK, YAT (Yet Another Thing), the tree<B> virtual functions must be able to treat their children in the tree as tree<B> instances also.

...

Ok, now I think I understand what you want.

As a result, an expression like "x+y" ends up being represented by a data structure that looks something like this:

Expr tree TExpr tree + <----------------- . / \ / \ / \ / \ x_ y_ int int |\ |\_____________/_____/ \_________________/

where the tree on the right contains only the "added data" as well as pointers to the corresponding nodes of the original tree.

YES! This is exactly what I want, expressed "most directly" AFAICT, this looks like the homomorphism or isomorphism diagrams I vaguely remember from category theory. The mostly horizonal lines represent the data mappings and the mostly diagonaly lines represent the function mappings. This much more clearly represents what I want. Thanks. However, this implementation has a disadvantage. For example, The tree -> first isomorphism, implemented by the regexp_first_constructor :: new_from_nilable, would have to create, for the regexp_tree_alt -> regexp_first_alt mapping a list<regexp_first_top*> as superclass of regexp_first_alt corresponding to the list<regexp_tree_top*> superclass of regexp_tree_alt. This duplicates the list overhead. OTOH, the way it's currently coded may be represented as: Expr tree TExpr tree + <----------------> . / \ / \ x_ y_ _int _int |\ |\ /| /| \ \___________/ / \_____________________/ where <---> represents two pointers, one the inverse of the other. At first, it looks like the number of pointers added by the isomorphism in the 2nd diagram exceeds that of the 1st (6 vs.5); however, the list overhead, I think, would tip the balance in the other direction at the cost, of course, of less speed for accessing the TExpr children. It's a trade-off, which I'll have to think on some more. Thanks for helping me clarify my thinking :) .

On Fri, Feb 20, 2004 at 04:35:55AM -0500, Gennadiy Rozental wrote:

I do not think we understand each other. Let me rephrase. My position is that polymorphic function object support does not belong to the library dedicated to "functional programming", even though I propose to update boost/functional.hpp header

Oh, I see. Yes, I agree. This is a generally useful feature; one not limited just to functional programming. I dunno if/what the plans are for a Boost "result_of" implementation (Doug?), but that will be the keystone library component to this end, I think.

...

It may also be attractive to OO people, who have gotten into the now-fashionable habit of avoiding member functions. Switching from the member notation

Shape s; Point p; if( s.contains(p) ) ...

to the non-member notation

if( contains(s,p) ) ...

loses the subject-verb-object order; using function objects that support infix, however, lets you say

if( s ^contains^ p ) ...

which some people may find attractive.

And some really confusing. Moreover in a user code, above most probably would look like if( s ^ boost::fcpp::contains ^ p ).

No; contains() in this example is a function the author of Shape would be writing. That is, it's a function you write in your own namespace.

Also it only look pretty for binary functoids. In any case I believe that if you do find it widely useful and could "prove" that, this should be the feature also implemented by boost::function.

Agreed on both points. -- -Brian McNamara (lorgon@cc.gatech.edu)

On 2/20/04 2:35 PM, "Douglas Paul Gregor" <gregod@cs.rpi.edu> wrote:

On Fri, 20 Feb 2004, Brian McNamara wrote:

...

I dunno if/what the plans are for a Boost "result_of" implementation (Doug?), but that will be the keystone library component to this end, I think.

A couple more whacks of the boot and I'll finally bring this in. It's been languishing in the sandbox for way too long. The major sticking point (and it's a boring one) is that I need to put in workarounds for broken compilers. I *hate* putting in workarounds for broken compilers.

Then don't. You don't have to put in workarounds (unless _your_ compiler needs them). If someone complains, they can fix the code on their time. (They should let you know first, though.) -- Daryle Walker Mac, Internet, and Video Game Junkie darylew AT hotmail DOT com

On Tuesday 17 February 2004 07:25 pm, Brian McNamara wrote:

Indirect Functoids

This is perhaps the most Boost-redundant feature of the library. FC++ indirect functoids do almost the same thing as boost::function objects. The differences: [reordering] - indirect functoids don't work with non-const reference parameters, whereas boost::function object do - indirect functoids can't be "null", whereas boost::function objects can

My inclination is that these two really don't matter all that much. They are convenient, perhaps, but not a reason to keep funN and boost::function separate.

- indirect functoids are imbued with "full functoid" features

Hmmm, what would this involve? I know that we would need to implement an operator() that supports currying and an operator[] that supports the Lambda stuff. I suspect that a careful reading of the fullN class templates would answer all of my questions, but I haven't gotten there yet. I could see having a macro like "BOOST_FUNCTION_FCPP_SUPPORT" (possibly defined by the FC++ headers before including boost/function.hpp, for instance).

If FC++ were accepted into Boost, my gut tells me that in the long-term, indirect functoids would naturally disappear (by getting absorbed by boost::function).

This would make me happy. I like FC++, but am (and have been---I think we've discussed this before) a little concerned about the fact that we're introducing a lot of nearly-redundant bits into Boost. Doug

On Thursday 19 February 2004 08:17 am, Peter Dimov wrote:

Douglas Gregor wrote:

...

On Tuesday 17 February 2004 07:25 pm, Brian McNamara wrote:

...

- indirect functoids are imbued with "full functoid" features

Hmmm, what would this involve? I know that we would need to implement an operator() that supports currying and an operator[] that supports the Lambda stuff.

I don't agree that function<> should support currying. It can lead to silent and hard to find bugs.

It seems to me that the currying operator() only gets enabled when the placeholder '_' or something involving '_' shows up. Assuming we can detect those reliably, I don't see the problem. (I'm not supporting this on broken compilers, so I'd be perfectly happy to use enable_if to do it, if necessary). Could you give an example of a problem with this? Doug

On Thursday 19 February 2004 09:09 am, Peter Dimov wrote:

Douglas Gregor wrote:

...

Could you give an example of a problem with this?

I have "prefix" currying in mind (the usual theoretical meaning of "currying", but please correct me if I'm wrong).

boost::function<void(int, int)> f;

f(2)(3); // same as f(2, 3)

but note what happens when someone inadvertently omits a trailing argument:

f(2); // compile-time error now, silent no-op with prefix currying support

Ah, that'd be a killer. I wasn't thinking about prefix currying.

As for f(..., _[k], ...) as a shorthand for bind(f, ..., _k, ...), you can do this today if you like, this isn't fc++ dependant in any way. Fact of the matter is that you have not, ergo, there is no user demand, ergo, this places the feature in the "but wouldn't it be cool if ..." category as far as I'm concerned.

Some demand might come from FC++ users if it's accepted and we are able to switch over its indirect functions, but I agree. Doug

Lambda There's been a bit of discussion about FC++ lambda on the Boost list, so I think it is pretty well-understood at this point. The documentation (and some prior discussions) explain the differences from boost::lambda. There are occasionally cases where I think FC++ lambda expressions are better[*] than boost::lambda expressions, but in most cases, boost::lambda provides the most concise solution. [*] where "better" means shorter, easier to read/write, or other intuitive measures of "goodness" Monads depend on lambda; in my opinion, over the long term, I think monads will be the main "selling point" that makes it worth keeping fcpp::lambda around in addition to boost::lambda. -- -Brian McNamara (lorgon@cc.gatech.edu)

On Wed, Feb 18, 2004 at 01:58:58AM +0000, Matthew Vogt wrote:

Couple of questions:

What are the features of fcpp::lambda that monads depend on, that boost::lambda does not provide?

Well, technically you don't even need lambda to implement monads. But to do monads well/easily, you need syntax sugar (along the lines of Haskell's "do" notation or FC++'s comprehensions). This sugar requires explicit lambda variables (the same kind you see in FC++ lambda's let/letrec). It's unclear to me if/how to do that with (or extend) boost::lambda in a natural way to get this. You also need to nest lambdas more often with monads. This is easier to do with explicit lambda (like FC++) than with boost::lambda (which has "protect" to help with this). There are also nits, like FC++'s letrec and if1/if2, which have come in handy in the implementation of certain monad functions, for various obscure reasons. This is a pretty rambling and vague-sounding answer, I suppose, hrm.

Are these features not able to be implemented in boost::lambda (given the current design), if that were desirable?

I think they don't "mesh well" with boost::lambda's design. While "anything's possible", my impression is that it's hard to design one tool that is as good at doing what boost::lambda does well and is simultaneously also good at doing monads; in a single design incorporating both, one or the other has to "suffer" a bit. -- -Brian McNamara (lorgon@cc.gatech.edu)

On Wed, Feb 18, 2004 at 07:07:43AM +0000, Matthew Vogt wrote:

Brian McNamara <lorgon <at> cc.gatech.edu> writes:

...

Well, technically you don't even need lambda to implement monads. But to do monads well/easily, you need syntax sugar (along the lines of Haskell's "do" notation or FC++'s comprehensions). This sugar requires explicit lambda variables (the same kind you see in FC++ lambda's let/letrec). It's unclear to me if/how to do that with (or extend) boost::lambda in a natural way to get this.

So, the less-concise syntax of fcpp::lambda means that there's more often a term in an expression which can be used to cause an operator overload to be selected from the lambda library?

Sorry, I don't quite understand the question. Here's an example which may or may not help: In Haskell you can write a list comprehension like this: [ x+y | x <- [1,2,3], y <- [2,3], x<y ] which means "a list of all the values x+y where x is selected from the list [1,2,3] and y from the list [2,3], under the condition x<y". In FC++ you can write it as // where l_123 and l_23 are lists with those values compM<ListM>()[ X %plus% Y | X <= l_123, Y <= l_23, guard[ X %less% y ] ] This desugars (just as in Haskell) into code with calls to the basic monad functions (bind, unit, zero): l_123 ^bindM<ListM>()^ lambda(X)[ l_23 %bindM<ListM>()% lambda(Y)[ if1[ X %less% Y, unitM<ListM>()[ X %plus% Y ], zeroM<ListM>() ] ] ] You can see the nested lambdas here; in boost::lambda-speak, X would be _1, and Y would be protect(_1), I think. Most monadic computations desugar into nested lambdas. When you have a "named variable" scheme as your representation for lambda expressions, it is pretty straightforward to express code involving nested lambdas. When you have a "positional placeholder" scheme, as in boost::lambda, it is not as easy to express nested lambdas. (The trade-off, of course, is that the Boost scheme is more succinct in cases which don't have heavily nested lambdas--and this is the common case for most (non-monadic) C++ code.) There's also the "sugar" issue, that we overload operators like "<=" to mean "gets" (that is, bind this lambda variable to this monad computation), which is relatively incompatible with boost::lambda's scheme of overloading operators to mean what that are in C++. Both operator overloading schemes are valuable; in boost, stuff like this 40 <= _1 // a concise way to express this function is great; in FC++ (especially if you are already accustommed to reading Haskell), the "compM" version with the "<="s is much more readable than the version with all the bindMs. In the end, there are two very different design goals, and depending upon which one you want to "optimize" (make easiest to express), you naturally gravitate towards one kind of design or the other. -- -Brian McNamara (lorgon@cc.gatech.edu)

On Thu, Feb 19, 2004 at 01:14:57PM -0300, Fernando Cacciola wrote:

Here's my review of the lambda sublibrary:

Thanks for these comments! One comment/question in return: ...

OTOH, I don't agree with the arguments against overloaded operators. Since FC++ lambda expressions are explicit I can't see any overload resolution problems. Given that C++ allows you to overload operators is hard to understand why would I have to write [X %plus% 1] instead of [X+1]

The docs says that this is in order to keep the interface small, that a compact interface is easier to remember that a wide one. However, C++ operators are so ubiquitous that the above argument doesn't hold... no one needs to "additionally" remember that he can use operators.... rather, with FC++, we need to remember that when can't; and furthermore, we need to remember what the spelling for "==" is: %equal%, %equal_to%, or whatever.

Also, lambda comprenhesions and guards do use overloaded operators. IMO, this library feature will be much less frequently used than lambda expressions per see, so IMO those overloads should be moved into the expressions themselves and taken out from the comprenhensions/guards.

This of course doesn't mean that the %f% should be drop alltoghther, or that the operator-like functoids removed. Just allow lambda expression to use (the most common at least) C++ operators.

Anyway, I think comprenhensions, guards, "let"/"letrec" and the LEType subsystem are great library features, so I'd like to keep this part of the library even thought it overlaps partially BLL.

I am just trying to clarify your opinion. You like "let"; in FC++, "let" works something like this: // inside a lambda expression let[ X == 3 ].in[ someLambdaExpressionInvolvingX ] You also say you would like it if common operators were overloaded, so we don't have to write X %equal% 3 I think as it stands now, these two are incompatible goals. We are currently overloading == to mean let-binding; we could overload it to mean %equal%, but then I'd want new syntax for let-binding. (Similarly, F[X] means delayed function call, not array indexing, as things currently stand.) How you would choose to resolve this issue? -- -Brian McNamara (lorgon@cc.gatech.edu)

Fernando Cacciola wrote:

Here's my review of the lambda sublibrary:

FCPP Lamba is "explicit" as opposed to "implicit" as in BLL. That is, I don't create a lambda expression on the fly by merely putting a placeholder somewhere in the expression; rather, I have to use brackets (% for infix notation).

I really like this part of the design.... I was myself caught at the "early evaluation" problems of implicit lambda until trained to put BLL's "protect()" on the right place.

When you work with complex lambdas it's hard to get it right (that is, avoid all possible early evaluations) implicitly, yet explicitly (the way FC++ works), it's straight forward (even if it requires a more adorned syntax).

OTOH, I don't agree with the arguments against overloaded operators. Since FC++ lambda expressions are explicit I can't see any overload resolution problems. Given that C++ allows you to overload operators is hard to understand why would I have to write [X %plus% 1] instead of [X+1]

The problem with operator overloading, LL/Phoenix style, can be alleviated by simply by disallowing automatic conversions. This is what I am advocating now, given the experience with Phoenix, possibly through a #define DISALLOW_AUTO_CONVERSION which defaults to ON. Let me explain... The folling snippet is your typical complaint: for_each(f, l, cout << "hello, " << arg1); << arg1 in Phoenix is _1 in LL >> The problem is that the expression cout << "hello, " is immediate, hence, cout << "hello, " << arg1 is immediate + deferred. Yet, the compiler does not catch this because the library allows overloading of the LHS and RHS where the RHS and LHS are curryable epxressions such as arg1 (or _1). This may be hard to get right. It took me lots of explaining about overloading rules in phoenix to make this point clear. See: http://www.boost.org/libs/spirit/doc/phoenix.html A simple solution is to simply disallow automatic conversions. Thus, if you want deferred evaluation, state it explicitly using var and val: for_each(f, l, var(cout) << val("hello, ") << arg1); In my opinion, this syntax is still clear while being safe from incorrect usage. Expressions such as: for_each(f, l, cout << "hello, " << arg1); will simply not compile. That said, I should point out that even FC++'s scheme can be prone to improper use too. No-one can prevent users from writing incorrect code with mixed operators such as: x * y %plus% 1 where x + y is immediately evaluated. Worse, the % precedence might be another cause of confusion. Consider: x + y %plus% 1 where the precedence of + is lower than %. The first will compile, (x * y is immediately evaluated) but the second might not (you probably can't add (y %plus% 1) to x. I fear that there might be still more confusion if you add ^ to the mix: x ^f^ y // ok x & y ^f^ y // x & y is immediate x | y ^f^ y // probably compile error << Having said all these, I have no problems with either LL/Phoenix or FC++'s syntax. IMO, it's just a matter of properly explaning the issues. >> Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

Lists Originally FC++ only provided one list datatype ("list", a lazy list). Then it occurred to me that, duh, many people may want to use list functions like "map", but not incur the overheads that come with lazy evaluation. (Don't pay for what you don't need to use.) So now there is also a strict_list datatype. Both list types meet the FC++ "ListLike" concept, and the list functions in the library use this interface so that functions like "map" are generic with respect to the actual list datatype used. Of course, much of FP revolves around using lists as a data structure, which is why this is pretty fundamental to the library. The FC++ list datatypes provide an iterator interface and constructors, so that you can convert to/from STL containers with ease. -- -Brian McNamara (lorgon@cc.gatech.edu)

Monads Monads are a work in progress. I think they have great potential, but they aren't yet "release quality" like the rest of the library. In any case, they are "at the bottom" of the dependency graph, so they can easily be included or removed. Point being, I would prefer it if people don't spend too much time debating them; it may be easier to just say "drop this component for now; if the library gets accepted, you may try again later"--that would be fine with me, especially if it expedites the review of other portions of the library. -- -Brian McNamara (lorgon@cc.gatech.edu)

On Thu, Feb 19, 2004 at 03:01:43AM -0500, Gennadiy Rozental wrote:

...

Monads are a work in progress. I think they have great potential, but they aren't yet "release quality" like the rest of the library.

Why then do we review them? It is not acceptable to me if even author says it's not release quality.

Good point! -- -Brian McNamara (lorgon@cc.gatech.edu)

| -----Original Message----- | From: boost-bounces@lists.boost.org | [mailto:boost-bounces@lists.boost.org] On Behalf Of Giovanni Bajo | Sent: 18 February 2004 09:00 | To: boost@lists.boost.org | Subject: [boost] Re: FC++ Formal review | | Paul A Bristow wrote: | > It would be nice to reduce warning noise (provided it is not too much trouble). | | If and only if the warning is not bogus. Otherwise, it should | be shut off at compiler level. MSVC is known to produce a few | bogus warnings. | | For instance, GCC has -Wshadow. If you do that, you can't | call a local variable "int y1;" because it will emit a | warning as it shadows "double y1(double)" in math.h. So what, | this is what I meant. I am sure the warning can be useful in | many other contexts, but we can't ask people to fix the code for this. If it isn't too much trouble, IMHO we SHOULD encourage people to fix this. | | > Msvc 7.1 (warnings level 4, no language extensions aka strict) | > produces some warning 'noise' at: const T operator*() const { return l.head(); } | > "qualifier applied to function type has no meaning; ignored | | See DR295 (http://anubis.dkuug.dk/jtc1/sc22/wg21/docs/cwg_defects.html#295). | I think this warning is bogus. | > However, this is a level 1 warning, so Microsoft obviously believe it is wrong C++. | It is, strictly speaking, but there is a DR for it. | I'm not sure it's worth a code fix. pragma warning disable will do it. I am sure you are right - but meanwhile it is a significant convenience for users to have a fix of some sort, if only to document that it isn't a problem, for example: #ifdef BOOST_MSVC # pragma warning (disable : 4180)// qualifier applied to function type has no meaning; ignored #endif Without this small fix, almost all use of FC++ will produce plenty of warning messages, which will distract from other useful warnings. Paul Paul A Bristow Prizet Farmhouse, Kendal, Cumbria UK LA8 8AB +44 1539 561830 +44 7714 330204 mailto: pbristow@hetp.u-net.com

| -----Original Message----- | From: boost-bounces@lists.boost.org | [mailto:boost-bounces@lists.boost.org] On Behalf Of David Abrahams | Sent: 18 February 2004 22:03 | To: boost@lists.boost.org | | Paul, could you change your "From:" address so it doesn't | look like your name is "Boost?" Grovelling apologies - I had to reload everything on my machine after being bitten by Microsoft 'Activation' which decided not to permit restore of backup because I was being naughty by changing too much hardware (When one RAID drive failed I had to replace TWO drives to get them the same!). I trust 'From' is now correct. Paul Paul A Bristow Prizet Farmhouse, Kendal, Cumbria UK LA8 8AB +44 1539 561830 +44 7714 330204 mailto: pbristow@hetp.u-net.com

On Wed, Feb 18, 2004 at 08:04:05PM -0500, Brian McNamara wrote:

It occurs to me today that there is some middle ground. Specifically, when you aren't using partial application (that is, "currying" in FC++, or "bind" in boost::lambda), I think that most of the reference issues go away. Perhaps there's a way to implement things so that functoids can have reference parameters, but trying to curry reference parameters is a compile-time error. I'll have to think about that more deeply.

Oh yeah, the other "hard problem" dealing with reference parameters is the "forwarding problem" (as described in http://std.dkuug.dk/jtc1/sc22/wg21/docs/papers/2002/n1385.htm ). Somehow I always forget about this issue. It's the real killer. Dealing with references is relatively straightforward for first-order functoids, but when you deal with higher-order ones, it quickly becomes hard (impossible?) to do the right thing. For example: suppose we want to write a functoid "app", such that app( f, x ) == f(x) Should the second parameter be "by value" or "by reference"? The "right answer", IMO, is that it depends on whether "f" takes its argument by value or reference. Except that this dependency now excludes many uses of app(), such as app( _, 3 ) Even though the result of that may be later used in a context that should/would be legal, "right now" we don't know if we should be copying the 3 or trying to take a reference to it. Whichever choice we make, it might turn out to be the "wrong" choice later. boost::lambda's "solution" is to always take reference parameters; this is why we have (_1 + _2)( 5, 7 ) // no, not 12--it's illegal In general, without a context-sensitive type system (like Haskell), I think there is no good solution to the problem of multiple parameter modes (value and reference) in the context of higher-order functions. That is, you cannot write "app" so that both calls here: int f(int); int g(int&); int x; ... app(_,3)(f); app(_,x)(g); work properly. It's impossible. I need to advertise this as the main reason we don't do references in FC++; it's kinda a tough argument to follow, but once you understand it, you see that there's no way to make things work "right". -- -Brian McNamara (lorgon@cc.gatech.edu)

On Thu, Feb 19, 2004 at 03:46:01PM +0200, Peter Dimov wrote:

Brian McNamara wrote:

...

That is, you cannot write "app" so that both calls here:

int f(int); int g(int&); int x; ... app(_,3)(f); app(_,x)(g);

work properly. It's impossible.

Let's see if I get this right:

#include <boost/bind.hpp> #include <boost/bind/apply.hpp> #include <iostream>

int f(int x) { std::cout << "f(" << x << ")\n"; return x; }

int g(int & x) { std::cout << "g(" << x << ")\n"; return ++x; }

int main() { int x = 2;

// app(_, 3)(f);

boost::bind( boost::apply<int>(), _1, 3 )(f);

// app(_, x)(g);

boost::bind( boost::apply<int>(), _1, x )(g); boost::bind( boost::apply<int>(), _1, boost::ref(x) )(g); }

You are saying that in the general case we can't decide between the last two lines, so we shouldn't support references.

In a nutshell, yes.

But I'm not sure that this is true. It seems to me that the rule is that I should use ref(x) if my x will outlive the function object, as in the above, and a plain x otherwise, and it will "just work".

This is not the rule I desire (though it is one possible way to go). I don't like this rule because it is deciding "whether to reference or not" based on the argument, rather than on the function. That is, in the case of app(_, x)(f); I don't want to pass "x" by reference simply because it appears be be a non-transient object with sufficient lifetime. "f" doesn't need a reference, so I shouldn't be creating one. In any case, the question of "if my x will outlive the function object" is non-trivial. Consider // assume h is a functoid with signature int h(int&,int) // assume l is a list<int> { int x = 1; l = map( h(x), l ); // note that "h(x)" means "h(x,_)" } Does "x" outlive the function object? Is it reasonable to pass "x" by reference? The answer to both is no. Functional programming idioms often create less-than-obvious lifetime issues. In the case above, the function object outlives "x", because "l" is a lazy list and it stores the function object so that list elements can later be computed on-demand.

...

boost::lambda's "solution" is to always take reference parameters; this is why we have

(_1 + _2)( 5, 7 ) // no, not 12--it's illegal

No, not correct. In the bind equivalent to app(_, x)(f), x is always copied by default. It's f that is passed by reference.

Right, sorry; I was unclear. The function objects created via lambda expressions always take non-const reference parameters. In contrast, FC++ function objects always take const reference parameters. (And, in both libraries, partial application makes a copy of the bound arguments.) (Is this right?) -- -Brian McNamara (lorgon@cc.gatech.edu)

"Brian McNamara" <lorgon@cc.gatech.edu> escribió en el mensaje news:20040219023655.GA20333@lennon.cc.gatech.edu...

On Wed, Feb 18, 2004 at 08:04:05PM -0500, Brian McNamara wrote:

...

It occurs to me today that there is some middle ground. Specifically, when you aren't using partial application (that is, "currying" in FC++, or "bind" in boost::lambda), I think that most of the reference issues go away. Perhaps there's a way to implement things so that functoids can have reference parameters, but trying to curry reference parameters is a compile-time error. I'll have to think about that more deeply.

Oh yeah, the other "hard problem" dealing with reference parameters is the "forwarding problem" (as described in http://std.dkuug.dk/jtc1/sc22/wg21/docs/papers/2002/n1385.htm ). Somehow I always forget about this issue. It's the real killer.

[A good description of the problems snipped]

In general, without a context-sensitive type system (like Haskell), I think there is no good solution to the problem of multiple parameter modes (value and reference) in the context of higher-order functions. That is, you cannot write "app" so that both calls here:

int f(int); int g(int&); int x; ... app(_,3)(f); app(_,x)(g);

work properly. It's impossible.

Though the problems you're addressing are real I think the solution is not to ban reference paramaters altoghether along the entire functoid taxonomy but to prescribe the following rule: first-order functoids can take parameters by value or by reference. high-order functoids should take parameteres only by value (but ref() can be used by the caller to denote a reference) Assuming app follows the rule above: app(_,3)(f); app(_,x)(g); // compile-time error, cannot bound a reference to a temporary. app(_,ref(x))(g); // ok Fernando Cacciola SciSoft

Brian McNamara wrote:

On Wed, Feb 18, 2004 at 05:04:11PM -0300, Fernando Cacciola wrote:

...

Referencial Integrity and Side Effects: ========================= ... What I'm trying to say is that I don't like the current design were functoids are prescribed to take only reference to const parameters, for a variety of reasons: ... [ a number of good arguments, elided ]

A number of people have voiced similar complaints, and I am gradually being worn down. I think one of the reasons I resist it is that I don't like how boost::lambda has dealt with the issue. Notably, examples like

// From 5.3.2 of the lambda docs //

http://www.boost.org/libs/lambda/doc/ar01s05.html#sect:bind_expressions bind(&A::set_j, a, _1)(k); // a.j == 0, as a copy of a is modified bind(&A::set_j, _1, 1)(a); // a.j == 1

where reference-versus-copy depends upon the precise "timing" of binding a function argument to a value, ...

There is no "precise timing" issue here, the rule is simple. Every argument passed to bind() is copied, whether it is &A::set_j, a, _1, or 1. An example that may illustrate your point is std::cout << _1 where 'std::cout' is taken by reference, versus _1 << 1 where '1' is copied. But std::cout << _1 is just a convenience spelling for ref(std::cout) << _1, which is reference-consistent. It's too prevalent in motivating examples to not warrant special treatment. :-)

Brian McNamara wrote:

On Thu, Feb 19, 2004 at 03:26:12PM +0200, Peter Dimov wrote:

...

Brian McNamara wrote:

...

A number of people have voiced similar complaints, and I am gradually being worn down. I think one of the reasons I resist it is that I don't like how boost::lambda has dealt with the issue. Notably, examples like

// From 5.3.2 of the lambda docs bind(&A::set_j, a, _1)(k); // a.j == 0, as a copy of a is modified bind(&A::set_j, _1, 1)(a); // a.j == 1

where reference-versus-copy depends upon the precise "timing" of binding a function argument to a value, ...

There is no "precise timing" issue here, the rule is simple. Every argument passed to bind() is copied, whether it is &A::set_j, a, _1, or 1.

I do understand this.

I am looking from another perspective. bind() lets me pass some of the arguments to a function "now", and the rest "later". In the example above, if I pass "a" now and "k" (1) later, I get one behavior, but if I pass 1 now and "a" later, I get another.

This is counter-intuitive, from my perspective.

Perhaps this is a question of underlying philosophy. You are looking at bind() from fc++ perspective. Conceptually, bind() is not a two-stage argument pass device, but a function adaptor device. It takes a function object as input and produces a different function object as output. Consider for example bind(f, _2, _1), or bind(f, _4), where no phased argument passing occurs. If you look at bind() from the fc++ perspective, where currying and invocation have the same syntax, it is understandable why the two "argument passing modes" appear counter-intuitive. It is, however, self-consistent. The bind() model of separate adaptation and invocation can handle the question of object lifetime better (const reference vs non-const reference is actually a distraction). Consider these two examples: [1] struct X {}; void g(X const & x, int); function<void(int)> f() { X x; return bind(g, x, _1); } [2] struct X: private noncopyable { }; void g(X const & x, int); template<class F> void h(F f); void f() { X x; h( bind(g, ref(x), _1 ); } In both cases, everything is pure const, no side effects, but the by-value vs by-reference difference is crucial. While the unified syntax of fc++ is quite appealing, there is no bind/call separation. Of course you could enable_if it to behave differently when one of the arguments is _ but I'm not sure whether this would be a good idea.

On Thu, Feb 19, 2004 at 02:47:25PM -0500, Brian McNamara wrote:

I think I may see a nice way to handle reference-issues in FC++ using boost::ref.

I need to think about it a little more, though. Will post more later.

Ok, I think I've got it now. Bear with me. First off, there is one design constraint ("DC") in FC++ that cannot be changed. Functoids must be able to parameters by value. This: // From 4.3 of the lambda docs int i = 1; int j = 2; (_1 + _2)(i, j); // ok (_1 + _2)(1, 2); // error (!) is not acceptable for FC++; in functional programming, you typically write code as one giant expression, and having to stop to declare statements and named variables is anathema. When programming inside the FC++ framework, practically everything is a functoid; whereas the issue above is only an occasional minor annoyance with boost::lambda, it would be a total killer with FC++. So that's just one issue that always has to be kept in the back of your mind. One other quick note: throughout this message, I am abusing terminology, and use "by value" to mean either by-value or by-const-&, and use "by reference" to mean only by-non-const-&. Ok, moving on... Currently in FC++, "full functoids" are classes which wrap up "basic functoids" and imbue them with a number of extra features (currying, infix, lambda, etc.). It occurs to me that "a way to deal with references" can be considered another one of these "features" which full functoids can do the work of supporting. (Aside: full functoids in FC++ in some sense do the same kind of work as bind() in boost. The reason you can't say "f(x,_1)" but instead have to say "bind(f,x,_1)" in boost is analogous to the reason that "f(x)" doesn't work for binary basic functoids but does work for binary full functoids in FC++. The main difference is that in boost, the "extra smarts" are typically added at the call site, whereas in FC++, the "extra smarts" are typically added at the function definition point.) In FC++, a polymorphic functoid is typically defined something like this struct Foo { // sig information template <class T, class U> T operator()( const T& x, const U& y ) { ... } }; typedef full2<Foo> foo_type; foo_type foo; // foo is a full functoid and only "by value" parameters are supported. Now, if we add some more smarts to full functoids, we could imagine doing this: struct Foo { // sig information // (A) template <class T, class U> T operator()( T& x, const U& y ) { ... } // note T param }; typedef full2<Foo> foo_type; // (B) foo_type foo; // foo is a full functoid Now we have a reference parameter. Provided that somewhere at either point (A) or point (B) we communicate this information (that the first parameter is by-reference) so that the full2 wrapper class is aware of it, we can delegate the "smarts" of dealing with references to the full2 class. Good. The idea is that references can be passed using ref(), as they sometimes are in Boost. The refs (reference_wrapper objects) get passed around by value until at some point we actually reach some underlying basic functoid which is expecting a reference. Its fullN wrapper knows this, and unwraps the reference at the last moment for the basic functoid to consume. Passing a non-ref-wrapped argument to a functoid that expects that argument by-reference is a compile-time error. Here is an example of how things would work, based on an example from earlier. Suppose I want to write the higher-order functoid app: app( f, x ) == f(x) Suppose I want to be able to apply it to functoids with these signatures: // int f(int); // int g(int&); Here's what we end up with int x; app(_,3)(f); // fine (works now) app(_,x)(f); // fine (works now) app(_,3)(g); // compile-time error app(_,x)(g); // compile-time error // (1) app(_,ref(x))(g); // works f(3); // fine (works now) f(x); // fine (works now) g(3); // compile-time error g(x); // compile-time error // (2) g(ref(x)); // works I think this is a system which is self-consistent, allows for references, does not have different behavior depending upon whether first-order or higher-order functions are used, and works with the design constraint (DC) mentioned at the top of this message. There are two main ways this works different from boost::lambda. At (1) (and maybe also the line above), boost would create a copy and then modify the copy. In the system I envision, this would be an error. (This is the desirable behavior, IMO.) At (2), even though g is the very function which expects a reference, you must still use ref() to pass its argument. I think this is an inescapable consequence of the combination of DC and the issues involving higher-order functions described earlier in the thread. Overall though, I think it looks like it will work, and it seems to me to be self-consistent and not contain any "special cases". I would appreciate feedback along these lines (especially if people can see some important issue I am missing; I would not be surprised if I have overlooked some "fatal flaw"). (One case I do know needs more thought is: f(ref(x)); Note that f() is not "expecting" a reference.) Assuming it works, do people "like it"? At some level, it is actually not too different from the way things are done now in FC++ to get mutable parameters (use a pointer). The advantages are - the explicit pointers are gone (no more worrying about 'null' issues and confounding the "intent" of parameters in the interface) - the machinery to do the "dereferencing" is hidden (in current FC++, you had to do it explicitly yourself--a pain) The main disadvantage is that - you still need to be explicit when passing arguments by reference but this is sometimes even true in boost::lambda (e.g. in bind() expressions). But now the trust is back in the hands of the programmer; you can write functoids involving references and pass references around, and it is your responsibility to deal with object lifetimes and such. The syntax/mechanism is closer to what happens in boost::lambda now, so it should be familiar. I'd appreciate any comments you-all have; especially from those of you with a deep understanding of the technical details. -- -Brian McNamara (lorgon@cc.gatech.edu)

On Fri, Feb 20, 2004 at 01:25:59AM +0200, Peter Dimov wrote:

Brian McNamara wrote:

...

Passing a non-ref-wrapped argument to a functoid that expects that argument by-reference is a compile-time error. ... Here's what we end up with

int x; app(_,3)(f); // fine (works now) app(_,x)(f); // fine (works now) app(_,3)(g); // compile-time error app(_,x)(g); // compile-time error // (1) app(_,ref(x))(g); // works ... There are two main ways this works different from boost::lambda. At (1) (and maybe also the line above), boost would create a copy and then modify the copy. In the system I envision, this would be an error.

There are some interesting subtleties here. In Lambda (and an earlier version of Bind), (1) will create a _const_ copy and then fail to call g. The current Bind, however, behaves as you describe since it follows the TR1 proposal more closely.

The TR1 bind specification says that "if the function application is made through a cv-qualified reference to, or a copy of, the function object, the same cv qualifiers are applied to f and aN."

In our case, since (the equivalent of) app(_, x) is non-const, its state, which includes the copy of x, can be changed. But if you pass app(_, x) by const reference to some algorithm, app(_, x)(g) will fail.

...

(This is the desirable behavior, IMO.)

Maybe. I'd be interested in some rationale, though. :-)

Here's my rationale; tell me what you think: If you have a reference parameter, it's because you want to have an effect on an argument. Examples are "in-out" or "out" parameters, where the parameter value is changed/assigned by the function so that the new value can be used by the caller. As a result, passing a copy of the argument and having the function modify the copy (as in some cases in boost::lambda) means that the modification gets "lost". In the example app(_,x)(g); // copy x, pass copy to g "g" is oblivious to the fact that it's not being called as just g(x); Whatever side-effects "g" makes to its argument never make it back to the caller, since "app" is passing it a copy of "x". The "lost" information probably indicates an error in the program. As a result, I think it's preferable that app(_,x)(g); should be a compile-time error, rather than copy "x" and pass the copy to "g". The call app(_,ref(x))(g); lets the caller see the effect on "x". If the client wants to explicitly ignore the "result" of g, he can always do X dummy(x); app(_,ref(dummy))(g); // modifies the dummy or some such. (My hunch is that this is very rare; you don't typically "try to ignore" reference parameter modifications.) What do you think? -- -Brian McNamara (lorgon@cc.gatech.edu)

Brian McNamara <lorgon@cc.gatech.edu> writes:

One other quick note: throughout this message, I am abusing terminology, and use "by value" to mean either by-value or by-const-&

Unfortunately, at least until 8.5.3/5 is reworded, those are not equivalent for some types :( See std::auto_ptr. Yeah, I know, types like that are anathema in FP ;-) Still, they exist in the real world of C++. -- Dave Abrahams Boost Consulting www.boost-consulting.com

"Peter Dimov" <pdimov@mmltd.net> writes:

David Abrahams wrote:

...

Brian McNamara <lorgon@cc.gatech.edu> writes:

...

One other quick note: throughout this message, I am abusing terminology, and use "by value" to mean either by-value or by-const-&

Unfortunately, at least until 8.5.3/5 is reworded, those are not equivalent for some types :(

See std::auto_ptr.

Yeah, I know, types like that are anathema in FP ;-) Still, they exist in the real world of C++.

Pass by value and pass by const reference have different semantics for std::auto_ptr regardless of 8.5.3/5.

True. I guess my move semantics example shows a case where they don't have different semantics, yet 8.5.3/5 thwarts their syntactic equivalence. -- Dave Abrahams Boost Consulting www.boost-consulting.com

4) Referencial integrity is anyway tricky enough to break even into a library specially designed to protect it. For example, FC++ uses reference to const objects in order to avoid mutable arguments, all in the name of referencial integrity. However, a "const&" parameter is not totally inmune to such problems the way a _true_ by-value parameter is, because it is after all still a reference, and references alias. Consider the following aparently contrived example:

inline point transform ( point const& p ) { return point(2.0* p.x * p.y, 0.5 * p.x * p.y); } void foo() { point p (..whatever...); p = transform(p); }

On many C++ implementations, the code above will yield an incorrect result because the return object will actually be constructed in place, producing an actual code equivalent to:

p.x = 2.0 * p.x * p.y ; p.y = 0.5 * p.x * p.y ;

and you can see how the aliasing resulted in the _argument_ "p.x" being modified inside the function.

That example is not too uncommon because any non-linear transformation

"Fernando Cacciola" <fernando_cacciola@hotmail.com> wrote in message news:c10gfp$qi0$1@sea.gmane.org... [snip] that

use a linear combination of the coordinates can fail because of the aliasing if a const& parameter is used instead of a true by-value argument.

Please educate me, is that really legal for a C++ implementation to do what you're saying? br Thorsten