On 8/21/2014 8:30 AM, Niall Douglas wrote:

On 20 Aug 2014 at 22:26, Steven Watanabe wrote:

...

The formal review of the VMD library by Edward Diener starts today, Aug 21st and is scheduled to continue until Aug 30th.

Is there not the documentation on a website instead of inside a code repo?

http://htmlpreview.github.io/?https://github.com/eldiener/variadic_macro_dat...

On 8/21/2014 7:53 PM, Niall Douglas wrote:

On 21 Aug 2014 at 9:33, Edward Diener wrote:

...

On 8/21/2014 8:30 AM, Niall Douglas wrote:

...

On 20 Aug 2014 at 22:26, Steven Watanabe wrote:

...

The formal review of the VMD library by Edward Diener starts today, Aug 21st and is scheduled to continue until Aug 30th.

Is there not the documentation on a website instead of inside a code repo?

http://htmlpreview.github.io/?https://github.com/eldiener/variadic_macro_dat...

Firstly that viewer kept failing for me, so I gave up quite quickly.

What failure do you get ? Maybe I can reproduce it. OTOH if you choose to clone the library in your modularboost/libs/vmd the top-level index.html link should work perfectly.

However from what I read I ask this: why is this library useful?

The introduction explains in general the functionality in the library that would make it useful. Did you read the introduction ?

What problems does it solve?

Does a library have to solve specific problems to be used by others ? The usefulness of a library is that you can do things with it that would be much harder to hand-craft without it. I may have not made too much of an issue in the documentation, although I did devote a section to it toward the end of the documentation ( see 'Input as dynamic types' ), but the general "problem" it solves is being able to write macros where, within the constraints of the types which the library can parse, you can take different paths as far as output generaation depending on the type of any particular macro input. You can therefore design macros to be more flexible than you might ordinarily have.

On 8/22/2014 8:20 AM, Niall Douglas wrote:

On 21 Aug 2014 at 21:24, Edward Diener wrote:

...

...

...

http://htmlpreview.github.io/?https://github.com/eldiener/variadic_macro_dat...

Firstly that viewer kept failing for me, so I gave up quite quickly.

What failure do you get ? Maybe I can reproduce it.

It doesn't work at all for me on Chrome.

With IE it somewhat works, but from time to time the github website says it is overloaded and please try again later. This is irritating. It could be that European users get different servers to US users.

I use Firefox and am in the US and it seems to work for me.

...

OTOH if you choose to clone the library in your modularboost/libs/vmd the top-level index.html link should work perfectly.

That involves more work than I wish to do right now. Right now I merely wish to find out if I could contribute a meaningful review.

The library is on Github and I am following the modular-boost format so I do not know what else I can do to make it easy for others to use and review the library. I do not think that asking reviewers to clone a library and follow instructions in a readme file in order to use it, is asking an awful lot of reviewers.

...

...

However from what I read I ask this: why is this library useful?

The introduction explains in general the functionality in the library that would make it useful. Did you read the introduction ?

Yes. I get what it does. Not why would anyone need this.

It is a library for writing C++ macros, like Boost PP. If you do not see any use for Boost PP then naturally you would not see any use for the VMD library.

...

...

What problems does it solve?

Does a library have to solve specific problems to be used by others ? The usefulness of a library is that you can do things with it that would be much harder to hand-craft without it.

The answer is no, but to enter Boost yes. It's very hard for me or I suspect anyone to vote on this if I have no idea what it's useful for. Otherwise someone could submit a randomly generated library and claim it should enter Boost, and we couldn't refuse.

I can't make you see any use for the library other than writing the documentation to explain how it is used and what it offers for macro creation.

...

I may have not made too much of an issue in the documentation, although I did devote a section to it toward the end of the documentation ( see 'Input as dynamic types' ), but the general "problem" it solves is being able to write macros where, within the constraints of the types which the library can parse, you can take different paths as far as output generaation depending on the type of any particular macro input. You can therefore design macros to be more flexible than you might ordinarily have.

I get this is all very clever. And if this library were being proposed for a suite of C libraries, I'd be all over this like no tomorrow.

I don't get how this is useful, or wise for C++ however. If you can show me something it can do which is a pain point for many which cannot be achieved using C++ 14, you get my vote. Otherwise I don't see the use case given the much superior alternatives.

I do not know what the superior alternatives of C++ 14 are which you see. I can only guess that you do not feel that the creation of macros as part of the interface to a library should ever be necessary. But for a number of Boost libraries they still are, and for future libraries they still will be until C++ has built-in support for things which the macro prerprocessor can still do.

On 8/22/2014 9:36 AM, Daniel James wrote:

On 22 August 2014 14:21, Edward Diener <eldiener@tropicsoft.com> wrote:

...

On 8/22/2014 8:20 AM, Niall Douglas wrote:

...

On 21 Aug 2014 at 21:24, Edward Diener wrote:

...

...

...

http://htmlpreview.github.io/?https://github.com/eldiener/variadic_macro_dat...

Firstly that viewer kept failing for me, so I gave up quite quickly.

What failure do you get ? Maybe I can reproduce it.

It doesn't work at all for me on Chrome.

With IE it somewhat works, but from time to time the github website says it is overloaded and please try again later. This is irritating. It could be that European users get different servers to US users.

I use Firefox and am in the US and it seems to work for me.

It might work better to create a 'gh-pages' branch for use with github pages. I did that here:

https://github.com/boostorg/quickbook/tree/gh-pages

Do you mean a separate branch just for the documentation ? In this case how does it help things since your link just below seems the same use of 'http://boostorg.github.io' etc, as I show in my original reply about reading the VMD docs online. Is it less bandwith do you mean ? In that case I will be glad to create a separate branch just for the documentation.

http://boostorg.github.io/quickbook/doc/html/index.html

On 22 August 2014 16:30, Daniel James <dnljms@gmail.com> wrote:

It's not the same system. Htmlpreview is implement in client side javascript and uses a yahoo service to get the files from the github api. It's not actually a github service, they're just running it on github's servers. Github pages creates some sort of static site on github's servers - so it requires less work for them than handling api requests. All you need to do is create a gh-pages branch containing whatever you wish to serve (in my case it's just branched from a feature branch with the generated html checked in). Github will then create a site.

Easier to just do it myself: http://danieljames.github.io/variadic_macro_data/doc/html/index.html

On 8/22/2014 3:25 PM, Rodrigo Madera wrote:

Hello Edward,

Nice library.

Thanks !

I couldn't get the docs going but Daniel James' link [1] did the trick. I gave it a quick read and I haven't yet tested out your code. But I have some thoughts already.

I have used BOOST_PP extensively for C++03 TMP, so I see a familiar pattern here.

To begin, I agree with people here that we need some sort of problem to your solution. Right now it seems that VMD is better off just being merged into BOOST PP enhancements.

The main reason I did not seek to merge VMD into Boost PP is that I view Boost PP as Paul Mensonides library and it is certainly his excellence that has created nearly all its functionality. The VMD library also has a much more liberal ( technically ) philosophy about what it presents. But that I mean that it has to accept the fact that non-constrained input can produce preprocessing errors, and that the basis for much of the functionality/ability of the library is the ability to detect "empty" input or "empty" input elements, an ability that has a very slight flaw which has no absolute solution.

Could you provide (as in write and develop) an in-depth comparison of VMD with BOOST PP?

What can VMD do that BOOST PP can't?

The main extensions of VMD over Boost PP, given the constraints mentioned in the library, are explained generally in the introduction. I am also aware that I need to make the introduction more encompassing along the lines of why a macro programmer will want to use VMD along with Boost PP to do macro metaprogramming. It is certainly a criticism of which I am fully aware and is justified to an extent I did not foresee when I created the documentation. The extensions in VMD that go beyond Boost PP are: 1) The ability to parse/identify a subset of preprocessor data types. These include all four of the data types invented by Paul for Boost PP, which are arrays, lists. seqs, and tuples, as well as individual identifiers and Boost PP numbers. 2) The ability to identify "empty" data. 3) The ability to parse a sequence of top-level data types, consisting of any of the data types which VMD can parse. 4) Some very useful variadic versions of macros are in VMD which have a less useless non-variadic version in Boost PP. The main advantage of parsing/identify preprocessor data types is like one of the main advantages Boost MPL gives you in doing template metaprogramming: you can change the logic of macro creation based on the particular type of preprocessing data and, in the case of identifiers and pp-numbers, the particular value of the proprocessing data, in order to create C++ output based on macro input. In other words it gives you even more flexibility in design a macro interface to create C++ constructs. This is never to say that the use of very cleverly designed macros should replace the use of C++ constructs themselves in doing C++ programming. I would be the first to say that if you can do something equally as well with C++ constructs, like variadic templates, than you can do with variadic macros, use the C++ constructs as is and leave the much normally much less capable preprocessor alone. But there are a number of cases where using well-designed macros can make the use of a library much easier for the end-user programmer.

Also, I use BOOST PP to overcome C++03 limitations. Now with C++11 much of these needs has been eliminated. We've got C++14 here and C++17 is in the making. With all that, what is VMD here to solve?

It is an extension to the functionality of Boost PP which gives the macro programmer the abilities mentioned above. It does not exist to solve any particular problem.

A good example of usage would be very nice. Maybe some macro code generation that could make it more interesting, if you will. I don't know, I'm just trying to make VMD necessary in some context.

I have never thought about any specific problem. Whatever I might posit may well give those who don't ever see the need for macros to generate C++ to criticize why a macro should ever be needed. But here is a possible example, although purely theoretical as to its use. Suppose you designed a macro where you were creating a C++ class. In the macro you might have some input parameter which, if the user specified the identifier 'public' the code you were creating from further macro input would go in a base class, if the user specified the identifier "protected' the code you were creating from further macro input would go in a derived class, and if the user specified 'private' the code you were creating from further macro input would go in a pimpl class connected to the base class by a pimpl sort of identifier. VMD has the ability to test for specific identifiers, such as 'public', 'protected', 'private', and based on the test you could use the Boost PP macro 'BOOST_PP_IIF' to create the output you needed. In general because VMD can identify a subset, although a heavily used one in preprocessor metaprogramming, of preprocessor data types and all the Boost PP data types, it adds to the ability to create macros based on differing types of input. In a way it is like any input in data processing in general. Programming often responds to input logically, not just mathematically. In a function some input may mean we want to logically perform data manipulation A while some different input may mean we want to perform data manipulation B. In template metaprogramming, some type may mean we want to pursue type manipulation A while another type may mean we want to pursue type manipulation B. In preprocessor metaprogramming some preprocessor data type/value may mean we want to generate C++ code A while some other preprocessor data type/value meay mean we want to generate C++ code B. Of course we can ignore the flexibility in any of these arrangements, whether at the functional level, the template level, or the macro level, and just dictate that the input data must always be some specific thing and we will just manipulate it as is. But in general programming based on logical flexibility is much easier. VMD intends to give the macro programmer this sort of logical flexibility, within the fairly narrow confines of what the macro preprocessor can do, as opposed to the much larger confines of the C++ language itself. Perhaps this whole explanation should have gone into the documentation as the "argument for the work" as 17th/18th century writers might put it <g>.

On 8/23/2014 8:30 AM, Niall Douglas wrote:

On 22 Aug 2014 at 17:00, Edward Diener wrote:

...

But here is a possible example, although purely theoretical as to its use. Suppose you designed a macro where you were creating a C++ class. In the macro you might have some input parameter which, if the user specified the identifier 'public' the code you were creating from further macro input would go in a base class, if the user specified the identifier "protected' the code you were creating from further macro input would go in a derived class, and if the user specified 'private' the code you were creating from further macro input would go in a pimpl class connected to the base class by a pimpl sort of identifier. VMD has the ability to test for specific identifiers, such as 'public', 'protected', 'private', and based on the test you could use the Boost PP macro 'BOOST_PP_IIF' to create the output you needed.

Are you saying here that one can use macros to do static reflection of the member access attributes of member functions, or that one can use macros to synthesise the member access attributes of member functions?

Neither. The TTI library uses macros to pass the name of the inner element to be introspected. It then creates metafunctions to do the introspecting using that name. It also uses macros in other ways. I strongly suggest, instead of taking a stance that macro metaprogramming should be obsoleted and is therefore not important to you, that you look at Boost PP in order to understand what it is about. Then you might understand better what I am doing in the VMD library. It is not an effort to solve some specific problem. It is an effort to add facilities to do macro metaprogramming which have a different focus than Boost PP.

If the former that is something we cannot do in C++ 14 without extra manually inserted metadata. If the latter then it is something we can easily do in C++ 98 via policy inheritance, let alone C++ 14.

On 22 Aug 2014 at 9:21, Edward Diener wrote:

...

...

OTOH if you choose to clone the library in your modularboost/libs/vmd the top-level index.html link should work perfectly.

That involves more work than I wish to do right now. Right now I merely wish to find out if I could contribute a meaningful review.

The library is on Github and I am following the modular-boost format so I do not know what else I can do to make it easy for others to use and review the library. I do not think that asking reviewers to clone a library and follow instructions in a readme file in order to use it, is asking an awful lot of reviewers.

Equally you want us to give you our time for free to look at your stuff. Some would say that putting the documentation onto its own website - like every other library in the review queue does by the way - isn't asking much of you. For you it's literally a git push to github pages and it saves all of us the hassle of cloning your repo and patching it into boost just to see if it's interesting at all to us.

...

...

...

However from what I read I ask this: why is this library useful?

The introduction explains in general the functionality in the library that would make it useful. Did you read the introduction ?

Yes. I get what it does. Not why would anyone need this.

It is a library for writing C++ macros, like Boost PP. If you do not see any use for Boost PP then naturally you would not see any use for the VMD library.

I see a use for Boost PP in interfaces, mainly to emulate variadic templates in 03. With C++ 11 I see little use for Boost PP in interfaces, but I can see it might be handy to have occasionally in internal source code away from public interfaces. I struggle to see how variadic macros add to Boost PP in such a limited use case. This is the source of my difficulty.

...

The answer is no, but to enter Boost yes. It's very hard for me or I suspect anyone to vote on this if I have no idea what it's useful for. Otherwise someone could submit a randomly generated library and claim it should enter Boost, and we couldn't refuse.

I can't make you see any use for the library other than writing the documentation to explain how it is used and what it offers for macro creation.

It is you who wants the library to enter Boost. It is therefore you and only you who must argue in favour. We default to a reject answer, it is you who must persuade beyond a reasonable doubt to not reject.

...

I get this is all very clever. And if this library were being proposed for a suite of C libraries, I'd be all over this like no tomorrow.

I don't get how this is useful, or wise for C++ however. If you can show me something it can do which is a pain point for many which cannot be achieved using C++ 14, you get my vote. Otherwise I don't see the use case given the much superior alternatives.

I do not know what the superior alternatives of C++ 14 are which you see.

Neither do I. I think it is up to you as the presenter of the library to tell me why your library is useful to me. I'm not going to decide if it's useful to me on my own. I don't have the time. I suspect neither do most people here.

I can only guess that you do not feel that the creation of macros as part of the interface to a library should ever be necessary. But for a number of Boost libraries they still are, and for future libraries they still will be until C++ has built-in support for things which the macro prerprocessor can still do.

I think this is evading the question. You are equivocating here Edward. For reference, I do believe that in C++ 14 code there has to be a high bar set on the use of macros in public interfaces, especially as C++ 17 Modules will deprecate that. For that reason, macros in interfaces is a bad idea. But I'm giving you out here Edward: give me a use case where your library solves a hard problem for others which C++ 14 cannot and you get my yes vote because I trust you to have made a good technical solution, my problem is in why it is useful. I don't think I am being unreasonable or unfair here, if your library is useful we need to see proof. I equally don't mind if anybody else chimes in here with a concrete example of where this variadic macro library solved a real problem for them. I assume that if this library got a review manager, it must have solved something for somebody somewhere. Equally, if no one chimes in, I must assume that this library has not met the popularity test and therefore has not been suffiiciently tested in real world use. And that would affect my vote for obvious reasons. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 22 Aug 2014 at 11:46, Edward Diener wrote:

I was not aware that every other library puts its documentation on its own website outside of GitHub. In that case it seems to me that it should have been a requirement of submitting a library to Boost or at least strongly suggested somewhere.

An inspection of the review queue makes it very clear.

...

I struggle to see how variadic macros add to Boost PP in such a limited use case. This is the source of my difficulty.

The Boost TTI library ( I am the developer ) uses variadic macros in order to pass, among other things the name of the element to be introspected. Many other libraries use macros, and some variadic macros, as part of its interface. In my tests using Boost PP, since I work on fixes to it, there are about 45 other libraries that use Boost PP. There are a number of Boost libraries that use variadic macros in advanced cases. There are probably many end-users out there using variadic macros in their own programming.

Are you saying that VMD has resulted from internal code in TTI that you decided to tidy up and make available as a standalone library? That would be a *very* different situation to writing greenfield code.

...

For reference, I do believe that in C++ 14 code there has to be a high bar set on the use of macros in public interfaces, especially as C++ 17 Modules will deprecate that. For that reason, macros in interfaces is a bad idea.

I do not understand the link between C++ 17 Modules and macro programming.

C++ Modules is expected to obsolete macro programming, in fact eliminating macros completely from interface files is one of its chief goals. It tells you all about it in its TS. Macro programming will remain supported in non-interface files. I'm okay with that, it can save a lot of repetitive copy and pasting. Again, if anyone can chime in with an example real world use of Edward's VMD library I would feel much happier. I need some example code to look at. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 23/08/2014 09:21 a.m., Niall Douglas wrote:

C++ Modules is expected to obsolete macro programming, in fact eliminating macros completely from interface files is one of its chief goals. It tells you all about it in its TS.

That's a bit hard to believe, I fail to see the relation. Could you point us to the proposal that states this? (as far as I know, there is no Modules TS, not even a draft) Regards, -- Agustín K-ballo Bergé.- http://talesofcpp.fusionfenix.com

On 23/08/2014 10:51 p.m., Niall Douglas wrote:

So, to give a full and proper answer, yes the WG21 papers say no to macros in interfaces, but the recent trend is not in that direction at all [1].

To sum things up: a bit of scoping for macros, less macros as flags, but hardly the obsolescence of macro metaprogramming on which modules have no incidence at all. If anything, I'd say that the improvements presented by any of the several different modules proposals in flux would be gladly welcomed by libraries like PP and VMD. Regards, -- Agustín K-ballo Bergé.- http://talesofcpp.fusionfenix.com

On 8/25/2014 5:19 AM, Zhenghui Zhou wrote:

As the only one who is watching on the github repository, I'm interesting in this library and had implemented some similiar pattern and used in my own projects based on ANSI C:

https://github.com/zhouzhenghui/c-sigslot/tree/master/src/preprocessor

Thanks for the link. I will take a look at the work you have done and compare it to the implementation in VMD to see if any of your techniques can improve VMD.

In fact, my implementation was inspired very much by Edward's library, as well as Paul Mensonides's chaos-pp library <http://sourceforge.net/projects/chaos-pp/> and Jens Gustedt's P99 library <http://gustedt.wordpress.com/>, the difference is that I just rewrote certain features in a simple and straightforward way.

As I know, the VMD library had ever been in Boost sandbox for a long time. I had seen some evolution there since Edward tried to enhance the Boost PP libaray.

We can learn that some features are missing by the community on end. For example, to deduce a empty macro, those who want to use such features have to rewrite their own implementation which could not be well refined.

The version in VMD is called BOOST_VMD_IS_EMPTY and is largely taken from Paul Mensonides' posting on the Internet with a few tweaks of my own, mainly for VC++. I believe it is the closest one can come to testing for emptiness, but no test for emptiness can be perfect. I will look at your version also. I am aware of Jens Gustedt version but it has a larger flaw than the one in VMD even if originally he thought it was flawless. I am convinced from Paul's discussions that no version of testing for emptiness can be flawless but I do explain in the VMD doc why the VMD version will work given macro constraints.

The Edward's library, as well as some others, extends the functionality of macro preprocess, give metaprogramming more capability and constraint characteristic. Although, due to the macro preprocessing system, some of such constraints are less conservative and misuse of some macro lead to preprocessing errors, I still think it is worth the effort.

I appreciate your saying this. I did put a great deal of effort into the VMD library, a very large part of it getting it to work with the VC++ preprocessor and its many preprocessing bugs.

For example, to enforce a macro parameter is enclosed by parentheses, I use the IS_BEGIN_PARENS macro, of which the name is changed to IS_TUPLE later in Edward's library. It isn't 100% safety, but give us some ability to manipulate the input stream just like the template trait to the type system.

I have not looked at your implementation of IS_BEGIN_PARENS. The macro in VMD which is the equivalent of testing whether a sequence of tokens begins with a set of parenthesis ( which may have other tokens within it ) is BOOST_VMD_IS_BEGIN_TUPLE. This macro is 100% safe. Again it is largely taken from a Paul Mensonides posting. In the 'develop' branch of Boost PP I have added the equivalent functionality in the form of the macro BOOST_PP_IS_BEGIN_PARENS. I plan to merge the 'develop' branch of Boost PP to the 'master' branch sometime shortly after the review of VMD is over, no matter what is the final outcome of this review.

The point is of course that it doesn't give much since the language itself hasn't changed. Macro metaprogramming is still the only one literal programming methodology in C/C++ language before any reflection facilities be included. The VMD library will be handy to use if it is accepted, particularly in enhancement of the language and the libraries, which will boost the language evolvement.

I also think that the VMD is better being merged into the Boost PP library, as a sub library.

The problem of merging VMD into Boost PP is twofold. First Boost PP is still Paul Mensonides library and his philosophy of safety, which I totally respect, is part of Boost PP. Secondly, as you have already said and realize, the philosophy of VMD is that preprocessing errors can occur if the functionality in VMD is misused by a programmer. In other words VMD depends on the constraints discussed in the library to work correctly. But within those constraints it does work and adds a great deal of flexibility to macro metaprogramming and the design of macro input.

For a header files only library, I don't think that any potential error in this preprocess library could cause big fault in the user end. To my limited knowledge, I suggest that this library is well refined, and I think it should be accepted.

Hope that my experience can give some help to the review procedure.

Thank you very much for your review.

Regards, Zhou Zhenghui

2014-08-24 10:43 GMT+08:00 Agustín K-ballo Bergé <kaballo86@hotmail.com>:

...

On 23/08/2014 10:51 p.m., Niall Douglas wrote:

...

So, to give a full and proper answer, yes the WG21 papers say no to macros in interfaces, but the recent trend is not in that direction at all [1].

To sum things up: a bit of scoping for macros, less macros as flags, but hardly the obsolescence of macro metaprogramming on which modules have no incidence at all.

If anything, I'd say that the improvements presented by any of the several different modules proposals in flux would be gladly welcomed by libraries like PP and VMD.

Regards, -- Agustín K-ballo Bergé.- http://talesofcpp.fusionfenix.com

On 8/25/2014 6:04 PM, Rodrigo Madera wrote:

On Mon, Aug 25, 2014 at 3:12 PM, Edward Diener <eldiener@tropicsoft.com> wrote:

...

The problem of merging VMD into Boost PP is twofold.

First Boost PP is still Paul Mensonides library and his philosophy of safety, which I totally respect, is part of Boost PP.

Secondly, as you have already said and realize, the philosophy of VMD is that preprocessing errors can occur if the functionality in VMD is misused by a programmer. In other words VMD depends on the constraints discussed in the library to work correctly. But within those constraints it does work and adds a great deal of flexibility to macro metaprogramming and the design of macro input.

Could you please provide a link to this mentioned safety philosophy of Boost PP?

There is no link I can give, other than to say it is my interpretation of the philosophy of Boost PP from looking at code and working/talking privately with Paul. I do not believe that passing incorrrect data to Boost PP functionality will ever produce a preprocessing error. It may produce an undefined result but that is then the programmer's problem misusing some functionality of Boost PP. Another example of the "safety philosophy" of Boost PP is that Paul was well aware that he had written a better variadic macro to test for emptiness, as well as that the Boost PP library has an undocumented non-variadic test for emptiness used internally. He did not want the current non-variadic version to be documented and he made no move to add the variadic version to Boost PP. In general he has always been very careful to not introduce any functionality in Boost PP that could produce incorrect results when used "correctly", as well as being very careful not to introduce functionality into Boost PP which could cause programmers to misuse the library by making anything more complicated than he felt it need be. This is what I mean by his "philosophy of safety".

As far as I can tell, Boost PP will give strange results if you don't obey its design constraints. I.e., try to use a malformed tuple.

But it does not create a preprocesing error AFAIK. It basically is telling you that passing the wrong sort of data gives you undefined results and that the undefined results are not any of the expected results.

Is Boost PP using some kind of protection that VMD isn't?

No, VMD is simply designed in a number of cases without any protection if you feed it something it cannot handle, simply because there is no way you can use the preprocessor to pre-detect the sort of thing which could give a preprocessor error. This is not always the case and VMD does the best job it can to detect invalid input and act accordingly. But VMD accepts, because of the extended functionality it offers, that some incorrect input can give a preprocessing error. This is part of what I talk about in the documentation when I mention input "constraints". The defacto example of this in VMD is parsing/identifying a number or identifier. In order to do this it uses preprocessor concatenation ( the '##' symbol ). For this to work in VMD the preprocessor token being tested must not begin with anything which is not an alphanumeric, underscore, or the left parenthesis of a tuple ( all of which in the VMD docs resolves to what is called "vtype" preprocessing data ). VMD cannot catch an error if you attempt to identify as a number or an identifier a preprocessor sequence of tokens starting with "&aname" let us say. It is just impossible to do. But the functionality is still there because VMD takes the philosophy that the macro creator using VMD has set as a constraint for the user of a macro that the input being tested is what VMD can handle correctly for a particular use of its functionality. If the end-user does not follow the "constraint" of a macro designer for some type of input, and therefore a compiler preprocessor error is unavoidably produced, it is the end-user's fault in the opinion of VMD. So if you have some input like "&aname" you can certainly use it as is in your macro programming but you can't pass it to anything in VMD expecting it is one of the v-types which VMD understands. That's what I mean by VMD being different than Boost PP. VMD pushes the bounds of what you can do in the preprocessor, using variadic macro techniques, with the knowledge that incorrect input to its functionality will sometimes just stop the preprocessor with an error. Boost PP never does this AFAIK, in the interests of safety. Both have their place but to me they are two different libraries.

On 8/25/2014 11:59 PM, pmenso57@comcast.net wrote:

----- Original Message -----

...

From: "Edward Diener" <eldiener@tropicsoft.com>

...

On 8/25/2014 6:04 PM, Rodrigo Madera wrote:

...

On Mon, Aug 25, 2014 at 3:12 PM, Edward Diener <eldiener@tropicsoft.com> wrote:

...

The problem of merging VMD into Boost PP is twofold.

First Boost PP is still Paul Mensonides library and his philosophy of safety, which I totally respect, is part of Boost PP.

Secondly, as you have already said and realize, the philosophy of VMD is that preprocessing errors can occur if the functionality in VMD is misused by a programmer. In other words VMD depends on the constraints discussed in the library to work correctly. But within those constraints it does work and adds a great deal of flexibility to macro metaprogramming and the design of macro input.

Could you please provide a link to this mentioned safety philosophy of Boost PP?

There is no link I can give, other than to say it is my interpretation of the philosophy of Boost PP from looking at code and working/talking privately with Paul.

I do not believe that passing incorrrect data to Boost PP functionality will ever produce a preprocessing error. It may produce an undefined result but that is then the programmer's problem misusing some functionality of Boost PP.

It can produce preprocessing errors. E.g. nothing stops the user from writing BOOST_PP_CAT(+,-) which technically should produce a preprocessing error.

Thanks for pointing this out.

...

Another example of the "safety philosophy" of Boost PP is that Paul was well aware that he had written a better variadic macro to test for emptiness, as well as that the Boost PP library has an undocumented non-variadic test for emptiness used internally. He did not want the current non-variadic version to be documented and he made no move to add the variadic version to Boost PP.

My opinion WRT the above is as follows.

Unconstrained arguments to macros are sequences of preprocessing tokens and whitespace separations (hereafter called "token sequences"). That token sequence may be empty. Which is no different than passing an empty vector, empty range, or empty string to a function at runtime. A preprocessor tuple, for example, can hold arbitrary unconstrained data (aside from those tokens which would interfere with actual parameter delineation such as commas and unbalanced parentheses--which I call "pathological input"). Therefore, the library must interpret (,,) as a ternary tuple, (,) as a binary tuple, and () as a unary tuple, and there is no machinery in place for a rogue value representing a nullary tuple. For example, if one is processing a tuple of cv-qualifiers and popping elements off the fr ont as it processes, that tuple might start as (const volatile, volatile, const,) which is a four-element tuple. The front is popped, and it becomes a ternary tuple (volatile, const,). The front is popped again, and it becomes a binary tuple (const,). After a final pop, it becomes a unary tuple () _not_ a nullary tuple. So, with something like MACRO(a, b,, d), the third argument is not elided, it is supplied as an empty tokens sequence.

One *could* supply a rogue value representing a nullary tuple (and, in fact, chaos-pp does this) and then design all of the tuple-related machinery around this, but this is not in place in the Boost pp-lib.

Even if it was logical to consider () to be a nullary tuple, there is no way to detect emptiness on unconstrained (but not pathological) input. It is flat-out impossible. There are a variety of ways you can *almost* do it. If the input is constrained such that those "almost" cases are removed, only then can you do it, but then you must add constraints to (e.g.) tuple elements which you cannot add to such a general-purpose container.

With all of the above said, you can have optional arguments to a macro provided there is at least one non-optional argument. For example, if you have MACRO(a, b, c) where you want c to be optional, you change the definition to MACRO(a, ...), you detect whether the __VA_ARGS__ is unary or binary (which can be detected), and branch in some way based on that result. Chaos does this too in a number of places.

Once you start constraining data, you can detect lots of things. The basic problems are that (1) you cannot reasonably constrain the data held in general-purpose containers and (2) you cannot change the way arguments to a macro are delineated by the preprocessor.

You can "constrain" data if you document that input "x" must be one of certain preprocessor token types ( including possible emptiness ), or Boost PP data types, in a particular macro input situation in order for the macro to work correctly. That is the sort of input "constraint" I deal with in VMD. In other words it is up to the macro designer to document what choices the input data could be and if the data does not follow that "constraint" the macro "will not work". In VMD the macro "will not work" sometimes means that a preprocessing error will occur and I accept that as part of what VMD does and also I document that it could happen. The analogy to this in C++ itself is that if you have an overloaded function name and the types of data passed to that function do not match any of the overloaded set for that name, the compiler will issue an error. So, I am 100% against ever treating () as a nullary tuple. Something else like an empty token sequence representing a nullary tuple? I am okay with that provided all of the tuple-related machinery is updated to deal with this rogue value and provided it works across all supported preprocessors--which is not an easy task in some cases. I agree with you about a nullary tuple. It does not exist. But in a "constrained" situation you could say that a unary tuple which has empty data means "something". What that "something" means is defined by the situation. I view this as valid macro programming. There are also other ways of passing empty preprocessor data in "constrained" situations other than a unary tuple with empty data. In the VMD documentation I discuss this pretty extensively.

I am also not particularly fond of fractured interfaces. By that I mean that I do not like interfaces that only work on some preprocessors in a library that is targeting preprocessors rather than the standards. I especially don't like scenarios like interface A works, interface B works, but interface A + B does not work--i.e. combinatorial nightmare (which is what VC++ produces, BTW)--which is what happens when you start exposing the low-level primitives as interfaces. Right now that means that, in particular, variadic macros and placemarkers are not supported well because preprocessors don't support them very well--especially "important" ones like VC++. If this was not the case and if we had good preprocessors across a broad range of compilers and in particular the important compilers , the Boost pp-lib would simply get a breaking ground-up reimplementation built to support variadic macros and placeholders from the ground up. Which, in essence, is what chaos-pp already i s. So, from my point of view, Boost.Preprocessor is a relic that serves as a lowest common denominator. Otherwise, get a real preprocessor (i.e. not VC++) and use better libraries like Chaos.

I tore my hair out in many different situations trying to get VC++ to work reasonably in VMD. As you certainly know this is not a fun way to program. Even with all the work I did the VMD documentation mentions a number of situations where one needs to do something slightly different when using VC++. Reagarding your Chaos library I still think that having Chaos as part of Boost is worthwhile even if it works only for C++ standard conforming preprocessors, which obviously means that it will not work with VC++. But I understand that might mean alot of work for you, submitting Chaos as a Boost library, with what you may perceive as little reward.

On Sat, Aug 23, 2014 at 5:21 AM, Niall Douglas <s_sourceforge@nedprod.com> wrote:

Again, if anyone can chime in with an example real world use of Edward's VMD library I would feel much happier. I need some example code to look at.

My [not a boost lib] Boost.Generic library that I presented at BoostCon a few years ago depends on VMD, though I haven't updated for a while so I don't know how VMD has changed (I'll try to review current VMD tonight). VMD is very useful particularly for any DSELs that are implemented with the preprocessor rather than through expression templates. This includes libraries like mine as well as Lorenzo's Boost.Contract library. Prior to VMD, I was hand-rolling a lot of the same macros that it has, but once VMD was about I was able to swap out the macros, which freed me from having to worry about maintaining the macros and dealing with non-compliant preprocessors. Specifically, VMD is a library for library developers in much the same way that MPL is a library primarily for library developers. This still has an important place in Boost in my opion and neither C++14 nor likely C++1z will change that. Anyway, I will say that my previous experience with VMD has been very positive. -- -Matt Calabrese

On 25 Aug 2014 at 13:54, Matt Calabrese wrote:

Anyway, I will say that my previous experience with VMD has been very positive.

Matt this is very useful information indeed. I now feel happy that VMD has enough popularity and a real world tested API. Thank you. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 8/22/2014 10:39 AM, Bjorn Reese wrote:

On 08/22/2014 03:21 PM, Edward Diener wrote:

...

It is a library for writing C++ macros, like Boost PP. If you do not see any use for Boost PP then naturally you would not see any use for the VMD library.

Why is it proposed as a separate library rather than extensions to Boost PP?

While I have worked enhancing Boost PP using variadic macros the Boost PP library is really Paul Mensonides library and I have done as little as possible to change its basic functionality other than working with Paul on it. I know how important Boost PP is to other Boost libraries as well as end-users so I know how careful one must be in order not to break any existing code. The philosophy of VMD is different than Boost PP. Boost PP is more conservative, looking for stability and safety foremost. The VMD documentation clearly states that misuse of some of the macros can lead to preprocessing errors from the compiler, and there is a detail discussion about macro input constraints as well as fairly repeated explanations of why end-user input outside of the constraints imposed by a number of macros in the library will cause preprocessor errors. In other words the library is less conservative and places more of an onus on the macro programmer to enforce constraints in his documentation so that end-users who break those constraints are aware that preprocessing errors could occur. I view this as a worthwhile tradeoff for the extra functionality provided by the library ( the ability to parse specific types of preprocessor data and the ability to work with empty preprocessor data ).