Lorenzo Caminiti <lorcaminiti <at> gmail.com> writes:

On Thu, Nov 10, 2011 at 6:43 PM, Gennadiy Rozental <rogeeff <at> gmail.com>> Hi,

...

Current implementation of subject macro have a very serious shortcoming - it does not work for empty __VA_ARGS__. After googling around a bit for some ideas I came up with the following alternative:

A while back there was a discussion about the possibility/impossibility to handle empty macro parameters using variadics:

http://lists.boost.org/Archives/boost/2011/02/177321.php

(Make sure to address the concerns expressed by Paul in the replies to the above thread.)

Frankly, I do not have time to read through the whole discussion. I am not a language lawyer, but for what it worth my version works on all compilers I have tried and looks to be perfectly legal (to me). I do know about one limitation. It's where __VA_ARGS__ ends with macro function which produces comma when evaluated. It's very rare case though and even if it's possible to implement solution that deals with it (look here: https://gustedt.wordpress.com/2010/06/08/detect-empty-macro-arguments/ ), I find the complication does not worth the trouble. If Edward or Paul care to comment it might be easier for them to point to any shortcomings in my solution (if any). Regards, Gennadiy

On 11/12/2011 6:52 AM, Gennadiy Rozental wrote:

Lorenzo Caminiti<lorcaminiti<at> gmail.com> writes:

...

On Thu, Nov 10, 2011 at 6:43 PM, Gennadiy Rozental<rogeeff<at> gmail.com>> Hi,

...

Current implementation of subject macro have a very serious shortcoming - it does not work for empty __VA_ARGS__. After googling around a bit for some ideas I came up with the following alternative:

A while back there was a discussion about the possibility/impossibility to handle empty macro parameters using variadics:

http://lists.boost.org/Archives/boost/2011/02/177321.php

(Make sure to address the concerns expressed by Paul in the replies to the above thread.)

Frankly, I do not have time to read through the whole discussion. I am not a language lawyer, but for what it worth my version works on all compilers I have tried and looks to be perfectly legal (to me). I do know about one limitation. It's where __VA_ARGS__ ends with macro function which produces comma when evaluated. It's very rare case though and even if it's possible to implement solution that deals with it (look here: https://gustedt.wordpress.com/2010/06/08/detect-empty-macro-arguments/ ),

No, this does not workaround the well-known problem when the variadic macro data is a function-like macro. See my comment at the link above.

I find the complication does not worth the trouble.

If Edward or Paul care to comment it might be easier for them to point to any shortcomings in my solution (if any).

On 11/13/2011 11:02 AM, Gennadiy Rozental wrote:

Edward Diener<eldiener<at> tropicsoft.com> writes:

...

...

https://gustedt.wordpress.com/2010/06/08/detect-empty-macro-arguments/ ),

No, this does not workaround the well-known problem when the variadic macro data is a function-like macro. See my comment at the link above.

I cannot find it. Do you mean on a Jens page? Can you summarize it here?

Look at my comment on the bottom of the web page above.

On 11/13/2011 12:51 PM, Gennadiy Rozental wrote:

Edward Diener<eldiener<at> tropicsoft.com> writes:

...

On 11/13/2011 11:02 AM, Gennadiy Rozental wrote:

...

Edward Diener<eldiener<at> tropicsoft.com> writes:

...

...

https://gustedt.wordpress.com/2010/06/08/detect-empty-macro-arguments/ ),

No, this does not workaround the well-known problem when the variadic macro data is a function-like macro. See my comment at the link above.

I cannot find it. Do you mean on a Jens page? Can you summarize it here?

Look at my comment on the bottom of the web page above.

I see nothing there. Are you sure you actually post it?

Anyone else able to see Edward's comments?

There is a comment number 7 which is mine at the bottom of https://gustedt.wordpress.com/2010/06/08/detect-empty-macro-arguments/ . Eddie

On 11/13/2011 3:49 PM, Peter Dimov wrote:

Edward Diener wrote:

...

There is a comment number 7 which is mine at the bottom of https://gustedt.wordpress.com/2010/06/08/detect-empty-macro-arguments/ .

I don't see it.

I do not know why that is the case. Is the last comment number 6 that you see ? I know I added a comment and it shows up as 7 when I bring up the page. My comment is: "#define AMACRO(x) () IS_EMPTY(AMACRO) is true, where of course it should be false. This is the flaw in your presentation and it is well-known. See Paul Mensonides post at http://boost.2283326.n4.nabble.com/problem-with-BOOST-PP-IS-EMPTY-macro-td26.... I give a better and less complicated version of IS_EMPTY in my variadic data macro library in the Boost sandbox which is almost wholly based on Paul Mensonides brilliant version ( I have to do some tweaks for VC8 )." Eddie

On 11/13/2011 9:48 PM, Gennadiy Rozental wrote:

Edward Diener<eldiener<at> tropicsoft.com> writes:

...

the page. My comment is:

"#define AMACRO(x) () IS_EMPTY(AMACRO) is true, where of course it should be false.

Are you sure? My gcc (cygwin) says it false.

I admit I do not understand how this can be. in the expansion to the code on that page it sure looks to me like 'HAS_COMMA(_TRIGGER_PARENTHESIS_ __VA_ARGS__ (~))' will equate to 1 when __VA_ARGS__ = AMACRO since 'AMACRO (~)' = '()', '_TRIGGER_PARENTHESIS_ ()' = ',', and HAS_COMMA(,) = 1. I will look at it in Wave, which I trust more than most anything as a preprocessor.

On the other hand if you define it like this:

#define AMACRO(x,y) anything here

if fails to build.

In any case, as I said previous, this is corner case we should just document, but size should return zero when we actually did not pass any arguments to the macro.

I did document this corner case in my VMD but clearly it makes an IS_EMPTY macro not 100% reliable, as it makes your solution not 100% reliable. It is no use saying that you have a foolproof solution to size returning 0, when it is clearly not foolproof. OTOH I personally am not against having a solution that is not 100% reliable else I would not have put out the latest version of VMD. But, although I can not speak for him, I have a strong impression that Paul does not want such code in Boost PP.

On Sun, Nov 13, 2011 at 9:03 PM, Gennadiy Rozental <rogeeff@gmail.com>wrote:

Edward Diener <eldiener <at> tropicsoft.com> writes:

...

I did document this corner case in my VMD but clearly it makes an IS_EMPTY macro not 100% reliable, as it makes your solution not 100% reliable. It is no use saying that you have a foolproof solution to size returning 0, when it is clearly not foolproof. OTOH I personally am not against having a solution that is not 100% reliable else I would not have put out the latest version of VMD. But, although I can not speak for him, I have a strong impression that Paul does not want such code in Boost PP.

Variadic data library which does not say that A() has zero arguments is useless IMO (all the corner cases aside, which we might never hit in practice). This is the most important use case.

I'm just curious, is -------- #define A(...) A(,) -------- legal? - Jeff

On Sun, Nov 13, 2011 at 11:53 PM, Paul Mensonides <pmenso57@comcast.net>wrote:

On Sun, 13 Nov 2011 21:10:11 -0800, Jeffrey Lee Hellrung, Jr. wrote:

...

I'm just curious, is

-------- #define A(...) A(,) --------

legal?

Yes. "Empty" is a valid argument in >= C99 and >= C++0x. An example of when an empty argument actually appears when passing around a cv- qualifier: nothing | const | volatile | const volatile. This is only one of many many scenarios where emptiness is a valid element and has meaning.

Well, in that case, since "clearly" A(,) has two (empty) arguments in its invocation, one must "clearly" conclude that A() has one (empty) argument in its invocation. That was more or less the point of my question, and your logic is sound to me, Paul. - Jeff

On Mon, Nov 14, 2011 at 6:14 AM, Gennadiy Rozental <rogeeff@gmail.com> wrote:

Paul Mensonides <pmenso57 <at> comcast.net> writes:

...

On Sun, 13 Nov 2011 21:10:11 -0800, Jeffrey Lee Hellrung, Jr. wrote:

...

I'm just curious, is

-------- #define A(...) A(,) --------

legal?

Yes.  "Empty" is a valid argument in >= C99 and >= C++0x.  An example of when an empty argument actually appears when passing around a cv- qualifier: nothing | const | volatile | const volatile.  This is only one of many many scenarios where emptiness is a valid element and has meaning.

Yes. This all is "legal" and "valid", but does it have meaning? I doubt it.

Show me sane example where one would develop macro FOO which takes two arguments and tell users to invoke it like this FOO(,) or FOO(a,) or FOO(a,b). Instead one can present a macro which takes variadic data and does different things depending on how many arguments passed (0, 1, or 2). So the invocations would look like FOO(), FOO(a), FOO(a,b) - much better IMO.

IMO, the point is that there is nothing wrong with you providing FOO(a, b) to your end users so as part of your public API and document that it should never be called as FOO(,). Then you internally convert the variadic `a, b` to the sequence (a)(b) and do your pp manipulations internally using pp-sequences. This way the Boost.Preprocessor library that you use to implement your internals supports a sound and scalable pp data structure (sequence) and your users benefit from using the variadic syntax that is more natural to them-- everyone wins :). My Boost.Local and Boost.Contract libraries follow this exact same pattern.

FOO(,) is a stone in a road to awk-like hell and should be strictly forbidden from the end user code. It might only be used in some corner cases deep in library code and never exposed.

So, all the theoretical corner cases aside FOO() should be invocation with zero arguments and FOO(a) with one.

I disagree, this is not theoretical. If you write a general purpose library like Boost.Preprocessor you need to make sure it is general (not theoretical) in that it handles all the uses cases that can arise within what the C++ standard specifies (including corner use cases). Otherwise, as the Boost.Preprocessor author you will be making assumptions like "my users will _never_ have to use a macro FOO(,)" which might be true in most but no all cases. HTH, --Lorenzo

Paul Mensonides <pmenso57 <at> comcast.net> writes:

...

So, all the theoretical corner cases aside FOO() should be invocation with zero arguments and FOO(a) with one.

Sorry, no. The language (preprocessor) forces the distinction between nullary and unary at the point of definition of the macro. This is not

We are going in circles. I am not telling you how to define BOOST_PP interfaces or how we should treat A() *in general* or that there aren't any use case where A's author wants to treat it as uniry invocation with empty arguments and yet IMO if I want to author macro A(...) and treat A() as invocation with zero arguments I expect boost PP to help me separate this use case (from A(a)). If it is not the case we might as well start a separate set of macros which does that, cause it is still going to be more important/popular than the other semantic. Regards, Gennadiy

Paul Mensonides <pmenso57 <at> comcast.net> writes:

I am not against having such a macro. I am against any form of treating an empty sequence of preprocessing tokens and whitespace separations (or one containing only whitespace separations) as a non-element in a data structure. "Empty" is a data element just as any other. E.g. () is not a nullary tuple, it is a unary tuple, and so on.

At some point you must make such a distinction simply because these concepts are treated differently between core language and preprocessor (for both C99 and C++) and even in the preprocessor itself. For the core language A() is a call with zero arguments. For the preprocessor A() is - a macro call with one argument, the empty one, if A has been defined to receive any argument - a macro call with zero arguments if the macro was defined to receive no argument I think any macros that are designated to be used by an "end user" must mediate between those different views and provide an end result that reads well for that end user, who I think expects syntax written as if it where core language. In P99 I use the IS_EMPTY macro for exactly that, try to accommodate that supposed expectation. Jens

On Mon, Nov 14, 2011 at 11:35 PM, Paul Mensonides <pmenso57@comcast.net> wrote:

On Mon, 14 Nov 2011 19:57:31 +0000, Jens Gustedt wrote:

...

I think any macros that are designated to be used by an "end user" must mediate between those different views and provide an end result that reads well for that end user, who I think expects syntax written as if it where core language.

In P99 I use the IS_EMPTY macro for exactly that, try to accommodate that supposed expectation.

I strongly disagree that catering to that expectation is a good thing in any context.  The sooner people realize (and expect) that the preprocessor behaves differently than the core language, the better.  Virtually all issues related to the misuse of the preprocessor stems from attempting to make object-like macros look like constant variables and function-like macro invocations look like underlying-language function calls.  At best, the correlation between function-like macro invocations and function calls should be incidental.  It should never be considered to be a goal.  That is a fundamentally broken mentality.

Preprocessing directives and macro invocations are part of an EL (DSEL without the DS) that overlays the underlying language; they are not part of the syntax or semantics of the underlying language.  The preprocessing phases of translation are best thought of as a executing a program overlaid on the source that translates that source into a token sequence presented to the C or C++ parser.  Anything else is just perpetuating the problem.

While it is not my business to force anyone to do anything, I am certainly not going to write code which may be viewed as best practice that intentionally perpetuates that mentality.  And this really is about that mentality.  It isn't about the slight syntactic inconvenience.  Sure, with a sequence (a)(b)(c), that's a lot of extra parentheses (and you get other benefits with sequences such as elements containing commas and data structures with unbounded length), but a tuple (a,b,c) does not have a lot of extra parentheses.  You'd get MACRO() vs. MACRO((a,b,c)) which has an incidental two extra parentheses and the two invocation cases are unambiguously, compiler-error-free detectable as different.  The only supposed "downside" is that it doesn't look like a regular function call which is a symptom of the above mentality and is not actually a downside (it might even be an upside given the status quo which has been around since the early days of C).

With an interface macro without a certain domain (where the necessary domain restrictions are reasonable), it really depends on the reuseability level of the interface.  Otherwise, there are serious repercussions related to recursion inherent in any forwarding interface (i.e. convert- and-forward) unless you do something with recursion like Chaos does which requires an extremely good preprocessor.

In the typical case, it is never a good idea, even at the interface level.  In some more advanced cases, it isn't that big a deal.  For example, in Lorenzo's (sp?) contract programming library, the syntax is so blatantly not that of the underlying language and the domain is specific enough that he can get away with it without it being particularly adverse.  In that case, the resulting syntax is blatantly a DSEL and therefore can have whatever syntax it wants since confusion between different "languages" is nearly impossible.  That scenario, however, is

For example: CONTRACT_FUNCTION_TPL( public void (push_back) ( (const T&) val ) precondition( size() < max_size() ) postcondition( auto old_size = CONTRACT_OLDOF size(), auto old_capacity = CONTRACT_OLDOF capacity(), back() == val, requires boost::has_equal_to<T>::value, size() == old_size + 1, capacity() >= old_capacity ) ) { vect_.push_back(val); }

quite a bit different than some macro just defined as A(a, b, ...) as opposed to A((a, b, ...)) or A((a)(b)(c)) just to make it look like a function call.

--Lorenzo

Gennadiy Rozental <rogeeff <at> gmail.com> writes:

Edward Diener <eldiener <at> tropicsoft.com> writes:

...

...

https://gustedt.wordpress.com/2010/06/08/detect-empty-macro-arguments/ ),

No, this does not workaround the well-known problem when the variadic macro data is a function-like macro. See my comment at the link above.

I cannot find it. Do you mean on a Jens page? Can you summarize it here?

Gennadiy

It had been stuck in the moderation queue, sorry for that. He basically says that the ISEMPTY macro there would not work with function like macros as an argument, something like #define AMACRO(x) () ISEMPTY(AMACRO) would give 1. I could not reproduce his assertion, for me this produces 0 on all compilers that I tested. This macro is heavily used in P99 and I don't see how this could return 1 for a preprocessor that is conforming to C99. (P99 is for C99 and not C++) Jens

Paul Mensonides <pmenso57 <at> comcast.net> writes:

Regardless, the scenario here is fundamentally wrong-headed. A macro argument may be empty, but that does not change the number of arguments. E.g.

#define A(...)

A() // one argument (*not* zero arguments) B(,) // two arguments C(,,) // three arguments

This is all very theoretical. In practice I want for A() to indicate that it has zero arguments and A(b) that it has one, even though former is just later with empty b. Any other size logic would be very confusing. Gennadiy

On Mon, 14 Nov 2011 19:13:17 +0000, Jens Gustedt wrote:

Paul Mensonides <pmenso57 <at> comcast.net> writes:

...

You cannot detect emptiness in the general case (even excluding pathological input such as unbalanced parentheses). This is simply a fact.

could you please give us some arguments for such a strong statement?

P99_IS_EMPTY does at least pretty well, and contrary to Edward Diener's false assertion somewhere in this thread, it does well with function like macros

(Excluding the pathological case of a unbalanced parenthesis.) You can make it work with input terminating in a function-like macro at the cost of making it not work for other things. As with any sort of predicate-like detection, you have to somehow interact with the token sequence of the argument in a way that either yields 1 or 0 (or whatever the equivalent of true and false are). The problem ultimately comes from the fact there is very little one can do to interact with the argument in a way that is always legal and that yields a detectable result (i.e. not stringizing). One can use token-pasting, but that rules out a *vast* amount of input because token-pasting must yield a valid single token (otherwise it is an error). Many compilers allow it as a non-standard extensions or oversight, but this is about standard C/C++. If that behavior is bad (which I personally believe it is) change the standards. The other way is by intentionally putting a (variadic) function-like macro name in front of the argument and a () after the argument which may expand against the argument and may expand against the trailing () that you added. However, if the argument ends in a function-like macro name, you have no idea what that macro is, what its arity is, what it expands to, etc.. The bottom line is that there is no general way to interact with the argument that doesn't cause compiler errors with some input. Even worse, the token-pasting approach scenario rules out a *vast* amount of input. There, at least, with a good preprocessor, you get a veritable detection or a compiler error. With the function-like macro approach, there's no telling what will happen. It depends heavily on whatever that function- like macro expands to. It may cause an error, it may not. When it doesn't it may yield an incorrect answer to the IS_EMPTY predicate.

...

Regardless, the scenario here is fundamentally wrong-headed. A macro argument may be empty, but that does not change the number of arguments. E.g.

#define A(...)

A() // one argument (*not* zero arguments) B(,) // two arguments C(,,) // three arguments

I completely agree with that part. In P99 I use that in particular to provide default arguments for functions, and there it is important to have A() detect that this is an empty argument and to produce the default.

Such detection is fine provided the input is suitably restricted. The problem is that that is highly domain-specific rather than general- purpose. One could have two separate IS_EMPTY-type macros that have different domain restrictions. For example, Chaos has an IS_EMPTY_NON_FUNCTION for the latter case. I don't remember off hand whether I made the other, though it is easy enough. Regardless, neither of these could be used in any sort of data structure implementation. How to interpret emptiness, to the degree that it is detectable, is domain- specific and under the purview of the user not the general purpose library.

(Default arguments with macros is probably not an issue for boost, since this is for C++ which has default arguments in the core language. P99 is for C99 which hasn't)

...

Moreover:

A( ) // one argument containing a space

The only to have a function-like macro that takes zero arguments is to define the macro as nullary.

I didn't capture what you try to say, here.

#define A() // nullary macro definition #define B(x) // unary macro definition A() // invocation with 0 arguments B() // invocation with 1 argument A( ) // invocation with 0 arguments thanks // to special rules for nullary macros B( ) // invocation with 1 argument which contains // a single space due to the whitespace // compression from an earlier phase of // translation #define E // defined as nothing #define C(x) B(x) A(E) // error C( E E ) // B ends up getting invoked with 3 spaces // as the compression doesn't happen this // late the process In this latter case, there is no semantic effect that can be determined from the whitespace. The only way whitespace can be significant from this point on is during stringizing and there only when is inside of an argument being stringized (which is specifically compressed) versus not inside the argument. E.g. #define S1(x) S2(x) #define S2(x) #x #define N() E E S1(+N()+) // "+ +" Moreso, even with something like P(a, b) The space before the 'b' is part of the argument. So #define P(x, y) S1(+y) P(1,2) // "+" P(1, 2) // "+ 2" I don't know of a single preprocessor (aside from maybe Hartmut's) that always deals with whitespace correctly. However, even if such preprocessors where common, you'd have to design every part of the library extremely carefully to have predicable behavior in this type of corner case. Regards, Paul Mensonides

Paul Mensonides <pmenso57 <at> comcast.net> writes:

On Mon, 14 Nov 2011 19:13:17 +0000, Jens Gustedt wrote: (Excluding the pathological case of a unbalanced parenthesis.)

Paul thanks a lot for your detailed explanation. yes, I am not interested in that case. I don't expect a compiler/preprocessor to don anything sensible when someone tries to pass in expressions that lead to unbalanced parenthesis.

You can make it work with input terminating in a function-like macro at the cost of making it not work for other things.

Still (even with your explanations below) I don't capture cases that my macro wouldn't handle.

The other way is by intentionally putting a (variadic) function-like macro name in front of the argument and a () after the argument which may expand against the argument and may expand against the trailing () that you added. However, if the argument ends in a function-like macro name, you have no idea what that macro is, what its arity is, what it expands to, etc..

my macro does a case analysis of four different scenarios and decides upon that. And I re-read the standard (C99) several times and implemented my own preprocessor to see if I captured all corner cases. Maybe I overlooked a case, then it should be possible to add that or at least it should be possible to document that behavior. So I would be very curious if you could point me on a concrete example of such a case.

The bottom line is that there is no general way to interact with the argument that doesn't cause compiler errors with some input.

Please, be more concrete and show me such an input. Jens

On Wed, Nov 16, 2011 at 3:13 AM, Jens Gustedt <jens.gustedt@loria.fr> wrote:

Jens Gustedt <jens.gustedt <at> loria.fr> writes:

...

Paul Mensonides <pmenso57 <at> comcast.net> writes:

...

The bottom line is that there is no general way to interact with the argument that doesn't cause compiler errors with some input.

Please, be more concrete and show me such an input.

since there is no reply so far to my questions I conclude that you don't have any concrete example, what a pitty

It is not uncommon for people on this ML to reply within 1-2 days or over ~1 week (that is because most people on this ML are also busy with work-- a project release, etc). Occasionally, a thread gets a back and forth of replies within hours but that is the norm only for library review threads. HTH, --Lorenzo

On 11/13/2011 10:50 AM, Gennadiy Rozental wrote:

Edward Diener<eldiener<at> tropicsoft.com> writes:

...

I will look at this but please look at my variadic macro data library in the sandbox which attempts to solve this same problem.

I do not see vmd_size.hpp. Where should I look?

Look at BOOST_VMD_IS_EMPTY(...). I did not attempt to use BOOST_VMD_IS_EMPTY(...) on BOOST_VMD_DATA_SIZE(...) to return 0 for empty variadic data because of the innate flaw in BOOST_VMD_IS_EMPTY, for which there is no workaround. You can obviously use it as you wish, for your own macro testing the size of variadic data. Also, for VC8, I could not implement a BOOST_VMD_IS_EMPTY(...) taking variadic data and had to settle for a BOOST_VMD_IS_EMPTY(param) instead. I did use BOOST_VMD_IS_EMPTY on my new BOOST_VMD_ASSERT macros while documenting the flaw in BOOST_VMD_IS_EMPTY.

...

When I announced the latest update to my library, nobody paid any attention to it.

Everything comes when time is right ;).

Eddie