[Preprocessor] Adding variadic macros support
Variadic macros would allow to support tuples better in Boost.PP, as it would not be necessary to specify the size of the tuple, and it could be possible to provide other nice features around tuples. Other possible applications could be to accept the sequence (a)(b, c, d)(e) as being the same as (a)((b, c, d))(e). If you think other things could be improved in PP with variadic macros, feel free to point out where. The question is: how should it be done? Should the new macros be named, say, BOOST_PPV_*? Should the macros in BOOST_PP_* be "overloaded" on arity (a possibility with variadic macros) so as to exhibit both the old and new interface?
On 11/24/2010 3:22 AM, Mathias Gaunard wrote:
Variadic macros would allow to support tuples better in Boost.PP, as it would not be necessary to specify the size of the tuple, and it could be possible to provide other nice features around tuples.
Other possible applications could be to accept the sequence (a)(b, c, d)(e) as being the same as (a)((b, c, d))(e).
If you think other things could be improved in PP with variadic macros, feel free to point out where.
The question is: how should it be done? Should the new macros be named, say, BOOST_PPV_*? Should the macros in BOOST_PP_* be "overloaded" on arity (a possibility with variadic macros) so as to exhibit both the old and new interface?
Please see my variadic_macro_data library in the sandbox. I have already addressed the problem of uniting variadic macros with Boost PP there as well as providing the support for tuples which you suggest in your opening paragraph above.
On 24/11/2010 13:55, Edward Diener wrote:
On 11/24/2010 3:22 AM, Mathias Gaunard wrote:
Variadic macros would allow to support tuples better in Boost.PP, as it would not be necessary to specify the size of the tuple, and it could be possible to provide other nice features around tuples.
Other possible applications could be to accept the sequence (a)(b, c, d)(e) as being the same as (a)((b, c, d))(e).
If you think other things could be improved in PP with variadic macros, feel free to point out where.
The question is: how should it be done? Should the new macros be named, say, BOOST_PPV_*? Should the macros in BOOST_PP_* be "overloaded" on arity (a possibility with variadic macros) so as to exhibit both the old and new interface?
Please see my variadic_macro_data library in the sandbox. I have already addressed the problem of uniting variadic macros with Boost PP there as well as providing the support for tuples which you suggest in your opening paragraph above.
So why hasn't there even been talk to get this reviewed or integrated into Boost.PP?
On 11/24/2010 8:42 AM, Mathias Gaunard wrote:
On 24/11/2010 13:55, Edward Diener wrote:
On 11/24/2010 3:22 AM, Mathias Gaunard wrote:
Variadic macros would allow to support tuples better in Boost.PP, as it would not be necessary to specify the size of the tuple, and it could be possible to provide other nice features around tuples.
Other possible applications could be to accept the sequence (a)(b, c, d)(e) as being the same as (a)((b, c, d))(e).
If you think other things could be improved in PP with variadic macros, feel free to point out where.
The question is: how should it be done? Should the new macros be named, say, BOOST_PPV_*? Should the macros in BOOST_PP_* be "overloaded" on arity (a possibility with variadic macros) so as to exhibit both the old and new interface?
Please see my variadic_macro_data library in the sandbox. I have already addressed the problem of uniting variadic macros with Boost PP there as well as providing the support for tuples which you suggest in your opening paragraph above.
So why hasn't there even been talk to get this reviewed or integrated into Boost.PP?
I have no idea. I do not think it is up to a library developer to constantly endeavor to promote his work, other than to do as excellent a development as he can. Dave Abrahams noticed it but no one else reacted to my original announcement. I too would love to see this library added to Boost but I do not think it can be added directly into Boost PP but rather exist as a separate Boost library. Paul Mensonides, the creator of Boost PP, in e-mails to me expressed the fact that he really did not want to add anything else directly into Boost PP since it was a fragile enough implementation given that it has to workaround bugs in many C++ implementations of the preprocessor, especially VC++. He also is working now on his own preprocessor library which needs a far more compliant C++ preprocessor than most compilers currently have. So I believe it would be best to add my library as a separate library to Boost. I can of course change the macro names in my library, as my documentation suggests, to be closer to Boost, perhaps simply by adding BOOST_ to the beginning of the macro names. Please feel free to "talk up" my library so that others notice it and want to add it to Boost, and have it reviewed. I am willing to do whatever is necessary on my end to get this done. I will soon be adding another library to the sandbox which can use variadic macros and uses my variadic_macro_data library. Questions, comments, suggestions etc. are all welcome regarding my library.
On 11/24/2010 6:14 AM, Edward Diener wrote:
On 11/24/2010 8:42 AM, Mathias Gaunard wrote:
On 24/11/2010 13:55, Edward Diener wrote: [...]
Please see my variadic_macro_data library in the sandbox. I have already addressed the problem of uniting variadic macros with Boost PP there as well as providing the support for tuples which you suggest in your opening paragraph above.
So why hasn't there even been talk to get this reviewed or integrated into Boost.PP?
I have no idea. I do not think it is up to a library developer to constantly endeavor to promote his work, other than to do as excellent a development as he can. Dave Abrahams noticed it but no one else reacted to my original announcement. I too would love to see this library added to Boost but I do not think it can be added directly into Boost PP but rather exist as a separate Boost library. Paul Mensonides, the creator of Boost PP, in e-mails to me expressed the fact that he really did not want to add anything else directly into Boost PP since it was a fragile enough implementation given that it has to workaround bugs in many C++ implementations of the preprocessor, especially VC++. He also is working now on his own preprocessor library which needs a far more compliant C++ preprocessor than most compilers currently have. So I believe it would be best to add my library as a separate library to Boost.
Are you referring to Chaos/Order [1]? [...]
Questions, comments, suggestions etc. are all welcome regarding my library.
I think macro composition and argument binding like that supported in Chaos/Order (forgive me, I'm not really sure what the difference is) would be great. I remember another individual posting a link to a preprocessor library in the past on the developer's mailing list that did something similar [2]. [1] http://sourceforge.net/projects/chaos-pp/ [2] http://lists.boost.org/Archives/boost/2010/01/160619.php - Jeff
On 11/24/2010 2:31 PM, Jeffrey Lee Hellrung, Jr. wrote:
On 11/24/2010 6:14 AM, Edward Diener wrote:
On 11/24/2010 8:42 AM, Mathias Gaunard wrote:
On 24/11/2010 13:55, Edward Diener wrote: [...]
Please see my variadic_macro_data library in the sandbox. I have already addressed the problem of uniting variadic macros with Boost PP there as well as providing the support for tuples which you suggest in your opening paragraph above.
So why hasn't there even been talk to get this reviewed or integrated into Boost.PP?
I have no idea. I do not think it is up to a library developer to constantly endeavor to promote his work, other than to do as excellent a development as he can. Dave Abrahams noticed it but no one else reacted to my original announcement. I too would love to see this library added to Boost but I do not think it can be added directly into Boost PP but rather exist as a separate Boost library. Paul Mensonides, the creator of Boost PP, in e-mails to me expressed the fact that he really did not want to add anything else directly into Boost PP since it was a fragile enough implementation given that it has to workaround bugs in many C++ implementations of the preprocessor, especially VC++. He also is working now on his own preprocessor library which needs a far more compliant C++ preprocessor than most compilers currently have. So I believe it would be best to add my library as a separate library to Boost.
Are you referring to Chaos/Order [1]?
Yes, although I can never find any files for it. But as far as I understood from Paul you could not use it with somewhat broken preprocessors, like the one in VC++, so its usage must be limited to highly compliant preprocessors like Boost's own Wave and perhaps a few others. I also ran into VC++ preprocessor problems with my variadic_macro_data library and had to find workarounds for them, largely by using some of Boost PP constructs as intermediary operations.
[...]
Questions, comments, suggestions etc. are all welcome regarding my library.
I think macro composition and argument binding like that supported in Chaos/Order (forgive me, I'm not really sure what the difference is) would be great.
Please be specific. Remember that the focus of my library was strictly to be able to use variadic macros easily along with Boost PP. I was not trying to do much more than that because I wanted to merely support what already exists in Boost PP with the addition of using variadic macros easily with it. I do not mind trying for more but I want to stay within the confines of what can be done with variadic macros to work with the current Boost PP. Also if Chaos/Order can do these things you may want it might be a good bet that they can not be done with the many of the compilers which Boost supports.
I remember another individual posting a link to a preprocessor library in the past on the developer's mailing list that did something similar [2].
On this site I see no files which can be accessed. Perhaps there is a way to access these files via cvs but I do not see instructions for doing so.
[2] http://lists.boost.org/Archives/boost/2010/01/160619.php
I do not know who this is and surely, whoever it is, he needs to put his work somewhere ( sandbox, vault ) where it can be found by others. Thanks for your interest in my library.
On 11/24/2010 1:09 PM, Edward Diener wrote:
On 11/24/2010 2:31 PM, Jeffrey Lee Hellrung, Jr. wrote: [...]
Questions, comments, suggestions etc. are all welcome regarding my library.
I think macro composition and argument binding like that supported in Chaos/Order (forgive me, I'm not really sure what the difference is) would be great.
Please be specific. Remember that the focus of my library was strictly to be able to use variadic macros easily along with Boost PP. I was not trying to do much more than that because I wanted to merely support what already exists in Boost PP with the addition of using variadic macros easily with it. I do not mind trying for more but I want to stay within the confines of what can be done with variadic macros to work with the current Boost PP.
To be specific: If one proposes to add an additional preprocessor library to boost, or an extension of Boost.PP, I only think one should also consider adding macro composition and argument binding, such as that in Chaos/Order or Avalanche (the [2] reference below). I brought it up under the context of limitations of the current Boost.PP interface.
Also if Chaos/Order can do these things you may want it might be a good bet that they can not be done with the many of the compilers which Boost supports.
True. How many compilers support your variadic macro library? It sounds like you've worked around many of MSVC's shortcomings, so I'm assuming most modern compilers. Perhaps I should be browsing through the docs/code...
I remember another individual posting a link to a preprocessor library in the past on the developer's mailing list that did something similar [2].
On this site I see no files which can be accessed. Perhaps there is a way to access these files via cvs but I do not see instructions for doing so.
Same question (and answer) is in one of the messages in [2] below.
[2] http://lists.boost.org/Archives/boost/2010/01/160619.php
I do not know who this is and surely, whoever it is, he needs to put his work somewhere ( sandbox, vault ) where it can be found by others.
Agreed; I had thought I had the code stashed away somewhere but I can't seem to find it.
Thanks for your interest in my library.
Boost.PP is already really awesome, but I welcome any facilities that make preprocessor metaprogramming a little easier. - Jeff
On 11/24/2010 4:29 PM, Jeffrey Lee Hellrung, Jr. wrote:
On 11/24/2010 1:09 PM, Edward Diener wrote:
On 11/24/2010 2:31 PM, Jeffrey Lee Hellrung, Jr. wrote: [...]
Questions, comments, suggestions etc. are all welcome regarding my library.
I think macro composition and argument binding like that supported in Chaos/Order (forgive me, I'm not really sure what the difference is) would be great.
Please be specific. Remember that the focus of my library was strictly to be able to use variadic macros easily along with Boost PP. I was not trying to do much more than that because I wanted to merely support what already exists in Boost PP with the addition of using variadic macros easily with it. I do not mind trying for more but I want to stay within the confines of what can be done with variadic macros to work with the current Boost PP.
To be specific: If one proposes to add an additional preprocessor library to boost, or an extension of Boost.PP, I only think one should also consider adding macro composition and argument binding, such as that in Chaos/Order or Avalanche (the [2] reference below). I brought it up under the context of limitations of the current Boost.PP interface.
I am still not sure what you mean by "macro composition" and "argument binding" in a preprocessor library. Boost PP has much support for macro composition within the context of what it offers in the form of repeatable constructs, data types, etc. but I have a feeling you mean something else. As far as "argument binding", in terms of preprocessor macros all arguments are initially in the form of tokens, so I am not sure what you mean by the phrase. My own focus in my library is strictly what can be done with variadic macro data. I admit an attempt to do other things to integrate in with Boost PP is daunting, since Paul Mensonides was quite honest in admitting that to work further on Boost PP in the face of broken preprocessor support from many compilers, especially VC++, is not something he wanted to do. If Paul could not, or would not do it, I have a feeling it would be quite a bit more imposing for anybody else, given the state of preprocessors in C++ compilers. Please also realize that Microsoft, in particular, is fully cognizant of the bugs in their preprocessor but has adamantly refused to fix them time and time again, no doubt valuing compatibility with user's code over adherence to the C++ standard in this regard. But who can truly know their reasons.
Also if Chaos/Order can do these things you may want it might be a good bet that they can not be done with the many of the compilers which Boost supports.
True. How many compilers support your variadic macro library? It sounds like you've worked around many of MSVC's shortcomings, so I'm assuming most modern compilers. Perhaps I should be browsing through the docs/code...
Please do. My testing shows that compilers which support variadic macros will support the functionality in my library. But support for variadic macros, even when I can workaround some of VC++'s broken support for variadic macros, hardly means the sort of preprocessor support needed for what you may want.
I remember another individual posting a link to a preprocessor library in the past on the developer's mailing list that did something similar [2].
On this site I see no files which can be accessed. Perhaps there is a way to access these files via cvs but I do not see instructions for doing so.
Same question (and answer) is in one of the messages in [2] below.
[2] http://lists.boost.org/Archives/boost/2010/01/160619.php
The only cvs address I see there is: cvs -d:pserver:anonymous_at_[hidden]:/cvsroot/chaos-pp and I have no idea what anonymous_at_[hidden] is supposed to be but I will try simply: cvs -d:pserver:anonymous@sourceforge.net:/cvsroot/chaos-pp
I do not know who this is and surely, whoever it is, he needs to put his work somewhere ( sandbox, vault ) where it can be found by others.
Agreed; I had thought I had the code stashed away somewhere but I can't seem to find it.
Neither can I.
Thanks for your interest in my library.
Boost.PP is already really awesome, but I welcome any facilities that make preprocessor metaprogramming a little easier.
I believe my library makes using variadic macros a bit easier and integrating its usage with whatever already exists in Boost PP cleaner. As I pointed out in my documentation, the various data types of Boost PP offer in general a much richer preprocessor programming interface than variadic data, but variadic macros offer a more user-friendly syntax than any of the data types of Boost PP. So to me the solution is to use the best of both worlds when doing preprocessor metaprogramming. This often involves using variadic macros as an interface for the end-user and then using the functionality in my library to convert that data, or pieces of that data, into Boost PP data types for internal manipulation and intergration with the rich set of Boost PP constructs. It is possible I could have done more with the variadic data in finding ways to convert it to and from Boost PP data constructs. I am willing to listen to suggestions on this account and add some functionality on this account if others find it useful.
On 11/24/2010 2:14 PM, Edward Diener wrote: [...]
I am still not sure what you mean by "macro composition" and "argument binding" in a preprocessor library. Boost PP has much support for macro composition within the context of what it offers in the form of repeatable constructs, data types, etc. but I have a feeling you mean something else. As far as "argument binding", in terms of preprocessor macros all arguments are initially in the form of tokens, so I am not sure what you mean by the phrase.
Imagine Boost.MPL without placeholders. As a simple example, suppose you already have a macro handy: #define macro( n ) ...insert your favorite expression involving n... and want to use it in an invocation of BOOST_PP_ENUM. This requires defining a forwarding macro: #define macro2( z, n, data ) macro( n ) BOOST_PP_ENUM( N, macro2, ~ ) Argument binding refers to the ability to bind the arguments of function-style macros to tokens or placeholders for tokens to be substituted later. So one would be able to do something like (making up the syntax from what I can remember) BOOST_PP_ENUM( N, BIND( macro, 2 ), ~ ) where BIND( macro, 2 )( z, n, data ) expands (intermediately) to macro( n ). The tokens 1, 2, 3, ... are placeholders to be substituted later, and to bind literal tokens to macro arguments, you wrap them in parentheses, so that BIND( macro, ( foo ) )( z, n, data ) expands to macro( foo ). Using the fictitious BIND macro, you can also compose macros inline, similar to how Boost.Bind works. This, at least, is what I remember from browsing through either the Avalanche code or Paul's Chaos/Order code, I don't remember which. But I think both basically provide this functionality. Hopefully that poor and hastily typed explanation helps, - Jeff
On 11/24/2010 5:45 PM, Jeffrey Lee Hellrung, Jr. wrote:
On 11/24/2010 2:14 PM, Edward Diener wrote: [...]
I am still not sure what you mean by "macro composition" and "argument binding" in a preprocessor library. Boost PP has much support for macro composition within the context of what it offers in the form of repeatable constructs, data types, etc. but I have a feeling you mean something else. As far as "argument binding", in terms of preprocessor macros all arguments are initially in the form of tokens, so I am not sure what you mean by the phrase.
Imagine Boost.MPL without placeholders.
As a simple example, suppose you already have a macro handy:
#define macro( n ) ...insert your favorite expression involving n...
and want to use it in an invocation of BOOST_PP_ENUM. This requires defining a forwarding macro:
#define macro2( z, n, data ) macro( n ) BOOST_PP_ENUM( N, macro2, ~ )
Argument binding refers to the ability to bind the arguments of function-style macros to tokens or placeholders for tokens to be substituted later. So one would be able to do something like (making up the syntax from what I can remember)
BOOST_PP_ENUM( N, BIND( macro, 2 ), ~ )
where BIND( macro, 2 )( z, n, data ) expands (intermediately) to macro( n ). The tokens 1, 2, 3, ... are placeholders to be substituted later, and to bind literal tokens to macro arguments, you wrap them in parentheses, so that BIND( macro, ( foo ) )( z, n, data ) expands to macro( foo ).
Using the fictitious BIND macro, you can also compose macros inline, similar to how Boost.Bind works.
This, at least, is what I remember from browsing through either the Avalanche code or Paul's Chaos/Order code, I don't remember which. But I think both basically provide this functionality.
Hopefully that poor and hastily typed explanation helps,
I understand the explanation. It does not have anything to do with variadic data as I see it so I do not see it being part of what I have done. Perhaps someone else can come up with an implementation of your bind idea for macros.
On Wed, 24 Nov 2010 14:45:02 -0800, Jeffrey Lee Hellrung, Jr. wrote:
On 11/24/2010 2:14 PM, Edward Diener wrote: [...]
I am still not sure what you mean by "macro composition" and "argument binding" in a preprocessor library. Boost PP has much support for macro composition within the context of what it offers in the form of repeatable constructs, data types, etc. but I have a feeling you mean something else. As far as "argument binding", in terms of preprocessor macros all arguments are initially in the form of tokens, so I am not sure what you mean by the phrase.
Imagine Boost.MPL without placeholders.
As a simple example, suppose you already have a macro handy:
#define macro( n ) ...insert your favorite expression involving n...
and want to use it in an invocation of BOOST_PP_ENUM. This requires defining a forwarding macro:
#define macro2( z, n, data ) macro( n ) BOOST_PP_ENUM( N, macro2, ~ )
Argument binding refers to the ability to bind the arguments of function-style macros to tokens or placeholders for tokens to be substituted later. So one would be able to do something like (making up the syntax from what I can remember)
BOOST_PP_ENUM( N, BIND( macro, 2 ), ~ )
For what it's worth, with the way that Chaos invokes user-supplied macros, it isn't difficult to provide general-purpose argument mappers. The general form would be something like (for only three arguments)... #define MAP(...) CHAOS_PP_VARIADIC_CAT(MAP_, __VA_ARGS__) #define UNARY_MAP(n, ...) \ ( \ CHAOS_PP_VARIADIC_ELEM(n, __VA_ARGS__) \ ) \ /**/ #define MAP_0(...) UNARY_MAP(0, __VA_ARGS__) #define MAP_1(...) UNARY_MAP(1, __VA_ARGS__) #define MAP_2(...) UNARY_MAP(2, __VA_ARGS__) #define BINARY_MAP(n, m, ...) \ ( \ CHAOS_PP_VARIADIC_ELEM(n, __VA_ARGS__), \ CHAOS_PP_VARIADIC_ELEM(m, __VA_ARGS__) \ ) \ /**/ #define MAP_01(...) BINARY_MAP(0, 1, __VA_ARGS__) #define MAP_02(...) BINARY_MAP(0, 2, __VA_ARGS__) #define MAP_10(...) BINARY_MAP(1, 0, __VA_ARGS__) #define MAP_12(...) BINARY_MAP(1, 2, __VA_ARGS__) #define MAP_20(...) BINARY_MAP(2, 0, __VA_ARGS__) #define MAP_21(...) BINARY_MAP(2, 1, __VA_ARGS__) #define TERNARY_MAP(n, m, o, ...) \ ( \ CHAOS_PP_VARIADIC_ELEM(n, __VA_ARGS__), \ CHAOS_PP_VARIADIC_ELEM(m, __VA_ARGS__), \ CHAOS_PP_VARIADIC_ELEM(o, __VA_ARGS__) \ ) \ /**/ #define MAP_012(...) TERNARY_MAP(0, 1, 2, __VA_ARGS__) #define MAP_021(...) TERNARY_MAP(0, 2, 1, __VA_ARGS__) #define MAP_102(...) TERNARY_MAP(1, 0, 2, __VA_ARGS__) #define MAP_120(...) TERNARY_MAP(1, 2, 0, __VA_ARGS__) #define MAP_201(...) TERNARY_MAP(2, 0, 1, __VA_ARGS__) #define MAP_210(...) TERNARY_MAP(2, 1, 0, __VA_ARGS__) #define A(n) n CHAOS_PP_EXPR(CHAOS_PP_ENUM( 10, A MAP(1) )) #define B(c, n) n / c CHAOS_PP_EXPR(CHAOS_PP_ENUM( 10, B MAP(2, 1), 2 )) This, of course, is a combinatorial implementation, but for relatively low numbers of arguments, it isn't too bad. Regards, Paul Mensonides
On 24/11/2010 22:09, Edward Diener wrote:
On 11/24/2010 2:31 PM, Jeffrey Lee Hellrung, Jr. wrote:
Are you referring to Chaos/Order [1]?
Yes, although I can never find any files for it.
There is some stuff on the CVS server associated with the sourceforge project, just not any release.
On 11/24/2010 5:06 PM, Mathias Gaunard wrote:
On 24/11/2010 22:09, Edward Diener wrote:
On 11/24/2010 2:31 PM, Jeffrey Lee Hellrung, Jr. wrote:
Are you referring to Chaos/Order [1]?
Yes, although I can never find any files for it.
There is some stuff on the CVS server associated with the sourceforge project, just not any release.
I do not see how one sees any project files in that sourceforge project. When I click on 'View all Files" nothing shows in the file list. No matter how I attempt to look at the files, nothing ever shows. I have no time or patience for such mysteries.
There is some stuff on the CVS server associated with the sourceforge project, just not any release.
I do not see how one sees any project files in that sourceforge project. When I click on 'View all Files" nothing shows in the file list. No matter how I attempt to look at the files, nothing ever shows. I have no time or patience for such mysteries.
What about: http://chaos-pp.cvs.sourceforge.net/viewvc/chaos-pp/ ? Regards Hartmut --------------- http://boost-spirit.com
On 11/24/2010 7:10 PM, Hartmut Kaiser wrote:
There is some stuff on the CVS server associated with the sourceforge project, just not any release.
I do not see how one sees any project files in that sourceforge project. When I click on 'View all Files" nothing shows in the file list. No matter how I attempt to look at the files, nothing ever shows. I have no time or patience for such mysteries.
What about:
http://chaos-pp.cvs.sourceforge.net/viewvc/chaos-pp/
?
Now I see some files. But I have no idea how one is supposed to get from the main project page to this page.
Edward Diener-3 wrote:
On 11/24/2010 7:10 PM, Hartmut Kaiser wrote:
There is some stuff on the CVS server associated with the sourceforge project, just not any release.
I do not see how one sees any project files in that sourceforge project. When I click on 'View all Files" nothing shows in the file list. No matter how I attempt to look at the files, nothing ever shows. I have no time or patience for such mysteries.
What about:
http://chaos-pp.cvs.sourceforge.net/viewvc/chaos-pp/
?
Now I see some files. But I have no idea how one is supposed to get from the main project page to this page.
Follows the path Develop/SVN browse code. Vicente -- View this message in context: http://boost.2283326.n4.nabble.com/Preprocessor-Adding-variadic-macros-suppo... Sent from the Boost - Dev mailing list archive at Nabble.com.
On 11/24/2010 8:26 PM, Vicente Botet wrote:
Edward Diener-3 wrote: [...]
Now I see some files. But I have no idea how one is supposed to get from the main project page to this page. Follows the path Develop/SVN browse code.
Actually, I think it's Develop/CVS browse code; going to "SVN browse code" looks pretty empty. - Jeff
On 11/25/2010 12:20 AM, Jeffrey Lee Hellrung, Jr. wrote:
On 11/24/2010 8:26 PM, Vicente Botet wrote:
Edward Diener-3 wrote: [...]
Now I see some files. But I have no idea how one is supposed to get from the main project page to this page. Follows the path Develop/SVN browse code.
Actually, I think it's Develop/CVS browse code; going to "SVN browse code" looks pretty empty.
You are right. Evidently the code is under CVS, but not under SVN ( yet ? ). I will look at Chaos and see what is done.
On Thu, 25 Nov 2010 08:42:37 -0500, Edward Diener wrote:
You are right. Evidently the code is under CVS, but not under SVN ( yet ? ).
I will look at Chaos and see what is done.
You'll have to pull it out of the CVS repository. The Subversion respository is not being used yet. The library is complete, as of right now there are more than five hundred primary interface macros (i.e. not counting implementation macros). There are more than two thousand interface macros if derivative macros are included (still not including implementation macros). If you checkout from the CVS, there is a file $CHAOS_ROOT/built-docs/ headers.html that lists all of the headers in the library. The first link (i.e. <chaos/preprocessor.h>) might be more useful for navigation. The reference documentation is complete (and extensive) except for a couple of macros that I added recently. There are lots of examples (mostly contrived). The topical documention is not complete, but feel free to ask me whatever you want. Chaos' use of variadics is controlled by CHAOS_PP_VARIADICS. This is normally set automatically based on whether the language supports variadics. However, there is currently no usable definition of __cplusplus for it to work with. Thus, by default, variadics are disabled in C++. However, the macro CHAOS_PP_VARIADICS can be defined on the command line to override the automatic settings. E.g. with GCC... g++ -E -P -std=c++0x -I $CHAOS_ROOT -D CHAOS_PP_VARIADICS test.cpp ===== Requirement for a Compliant Preprocessor ===== The techniques used by Chaos' internals are *extremely* advanced. Most of the time, they require the preprocessor to do *exactly* what it should do according to the standard(s). Because of that, Chaos cannot be ported to broken preprocessors like MS (and putting workarounds in the source would defeat the purpose of Chaos even if such workarounds were possible). I believe that GCC, EDG-based compilers, and Wave are 100% usable. There are a few others that are mostly usable as well as a few tools that have internal (custom) preprocessor implementations that are 100% usable also. In particular, VC++ is *not* usable. I am not sure about Sun at the moment. ===== Notable Differences from the Boost pp-lib ===== The most obvious (considering the context) is support for variadics/ placemarkers from the ground up. Probably more importantly, Chaos generalizes recursion and all higher-order constructs are reentrant without replication of their implementations. As an example, take a look at the implementation of BOOST_PP_REPEAT compared to the following: #define REPEAT(n, macro, ...) \ REPEAT_S(CHAOS_PP_STATE(), n, macro, __VA_ARGS__) \ /**/ #define REPEAT_S(s, n, macro, ...) \ CHAOS_PP_IF(n)( \ REPEAT_I, CHAOS_PP_EAT \ )( \ CHAOS_PP_OBSTRUCT(), CHAOS_PP_NEXT(s), \ CHAOS_PP_DEC(n), macro, __VA_ARGS__ \ ) \ /**/ #define REPEAT_INDIRECT() REPEAT_S #define REPEAT_I(_, s, n, macro, ...) \ CHAOS_PP_EXPR_S _(s)(REPEAT_INDIRECT _()( \ s, n, macro, __VA_ARGS__ \ )) \ CHAOS_PP_CALL(macro)()(s, macro, n, __VA_ARGS__) \ /**/ #define MACRO(s, n, ...) \ CHAOS_PP_COMMA_IF(n) class T ## n \ /**/ CHAOS_PP_EXPR(REPEAT( 5, MACRO, ? )) ===== Variadics as Data Structures ===== Over the course of development of Chaos, I rapidly discovered that the direct use of variadics as element sequences is only marginally useful (at best). Variadics are better used for other things. Let me clarify. Variadics as raw data structures are virulent. They take over the interface of everything else by forcing particular argument orders and "stealing" the single variadic "argument". As an obvious example, you can't have an algorithm that operates on two such data structures. OTOH, variadics used indirectly in data structures are very useful (e.g. variadics sequences). Even with tuples such that the size is not required to be known for element access. Long story short, (...) is better than just ... for all kinds of reasons. ===== Lambdas/Binding ===== Chaos has a pretty heavyweight lambda engine. For example (contrived), #include <chaos/preprocessor.h> CHAOS_PP_EXPR(CHAOS_PP_ENUM( 5, CHAOS_PP_PRIMITIVE_CAT_(class T, CHAOS_PP_ARG(1)) CHAOS_PP_WHEN_(CHAOS_PP_ARG(1))( CHAOS_PP_CAT_(= T, CHAOS_PP_DEC_(CHAOS_PP_ARG(1))) ) )) ...results in: class T0, class T1 = T0, class T2 = T1, class T3 = T2, class T4 = T3 However, the problem with lambdas (with the preprocessor) is that they are considerably more expensive. No matter how you do it, it requires some sort of interpretation--which is slower--and simply having a macro is usually clearer. I've actually been considering their removal from Chaos for several years. #define _(s, n) \ class T ## n \ CHAOS_PP_WHEN(n)(= CHAOS_PP_CAT(T, CHAOS_PP_DEC(n))) \ /**/ CHAOS_PP_EXPR(CHAOS_PP_ENUM(5, _)) #undef _
On 11/27/2010 9:36 AM, Paul Mensonides wrote:
On Thu, 25 Nov 2010 08:42:37 -0500, Edward Diener wrote:
You are right. Evidently the code is under CVS, but not under SVN ( yet ? ).
I will look at Chaos and see what is done.
You'll have to pull it out of the CVS repository. The Subversion respository is not being used yet. The library is complete, as of right now there are more than five hundred primary interface macros (i.e. not counting implementation macros). There are more than two thousand interface macros if derivative macros are included (still not including implementation macros).
If you checkout from the CVS, there is a file $CHAOS_ROOT/built-docs/ headers.html that lists all of the headers in the library. The first link (i.e.<chaos/preprocessor.h>) might be more useful for navigation. The reference documentation is complete (and extensive) except for a couple of macros that I added recently. There are lots of examples (mostly contrived). The topical documention is not complete, but feel free to ask me whatever you want.
I actually just downloaded the tar.gz file. While I could get Tortoise CVS and use it to keep up to date, I have gotten lazy enough where since its not in SVN ( or Git ) I tend not to want to have to deal with CVS anymore. But that's my problem.
Chaos' use of variadics is controlled by CHAOS_PP_VARIADICS. This is normally set automatically based on whether the language supports variadics. However, there is currently no usable definition of __cplusplus for it to work with. Thus, by default, variadics are disabled in C++. However, the macro CHAOS_PP_VARIADICS can be defined on the command line to override the automatic settings. E.g. with GCC...
g++ -E -P -std=c++0x -I $CHAOS_ROOT -D CHAOS_PP_VARIADICS test.cpp
Perfectly reasonable.
===== Requirement for a Compliant Preprocessor =====
The techniques used by Chaos' internals are *extremely* advanced. Most of the time, they require the preprocessor to do *exactly* what it should do according to the standard(s). Because of that, Chaos cannot be ported to broken preprocessors like MS (and putting workarounds in the source would defeat the purpose of Chaos even if such workarounds were possible).
I believe that GCC, EDG-based compilers, and Wave are 100% usable. There are a few others that are mostly usable as well as a few tools that have internal (custom) preprocessor implementations that are 100% usable also. In particular, VC++ is *not* usable. I am not sure about Sun at the moment.
What are the chances that you would be willing to propose Chaos for Boost even though it would work with only a subset of compilers, and specifically not with VC++ ? I personally think this would still be valuable for the authors of Boost libraries and for end-users, but I could understand your unwillingness to do so or other Boost developers and/or end-users unwillingness to accept Chaos into Boost because it does not work for a number of C++ compilers. But IMO that Chaos would provide easier preprocessing programming for even a subset of C++ implementations would make it worth using.
===== Notable Differences from the Boost pp-lib =====
The most obvious (considering the context) is support for variadics/ placemarkers from the ground up. Probably more importantly, Chaos generalizes recursion and all higher-order constructs are reentrant without replication of their implementations.
That is an excellent advantage in simplification over current Boost PP despite the fact that you have made Boost PP quite usable even with the recursion difficulties.
As an example, take a look at the implementation of BOOST_PP_REPEAT compared to the following:
#define REPEAT(n, macro, ...) \ REPEAT_S(CHAOS_PP_STATE(), n, macro, __VA_ARGS__) \ /**/ #define REPEAT_S(s, n, macro, ...) \ CHAOS_PP_IF(n)( \ REPEAT_I, CHAOS_PP_EAT \ )( \ CHAOS_PP_OBSTRUCT(), CHAOS_PP_NEXT(s), \ CHAOS_PP_DEC(n), macro, __VA_ARGS__ \ ) \ /**/ #define REPEAT_INDIRECT() REPEAT_S #define REPEAT_I(_, s, n, macro, ...) \ CHAOS_PP_EXPR_S _(s)(REPEAT_INDIRECT _()( \ s, n, macro, __VA_ARGS__ \ )) \ CHAOS_PP_CALL(macro)()(s, macro, n, __VA_ARGS__) \ /**/
#define MACRO(s, n, ...) \ CHAOS_PP_COMMA_IF(n) class T ## n \ /**/
CHAOS_PP_EXPR(REPEAT( 5, MACRO, ? ))
===== Variadics as Data Structures =====
Over the course of development of Chaos, I rapidly discovered that the direct use of variadics as element sequences is only marginally useful (at best). Variadics are better used for other things. Let me clarify. Variadics as raw data structures are virulent. They take over the interface of everything else by forcing particular argument orders and "stealing" the single variadic "argument". As an obvious example, you can't have an algorithm that operates on two such data structures.
OTOH, variadics used indirectly in data structures are very useful (e.g. variadics sequences). Even with tuples such that the size is not required to be known for element access. Long story short, (...) is better than just ... for all kinds of reasons.
I totally agree and made a point of saying in my variadic_macro_data library that variadic macros real advantage over Boost PP sequences is largely in familiarity/ease of syntax for end-users. I am glad you have also found their usage in data structures valuable.
===== Lambdas/Binding =====
Chaos has a pretty heavyweight lambda engine. For example (contrived),
#include<chaos/preprocessor.h>
CHAOS_PP_EXPR(CHAOS_PP_ENUM( 5, CHAOS_PP_PRIMITIVE_CAT_(class T, CHAOS_PP_ARG(1)) CHAOS_PP_WHEN_(CHAOS_PP_ARG(1))( CHAOS_PP_CAT_(= T, CHAOS_PP_DEC_(CHAOS_PP_ARG(1))) ) ))
...results in:
class T0, class T1 = T0, class T2 = T1, class T3 = T2, class T4 = T3
However, the problem with lambdas (with the preprocessor) is that they are considerably more expensive. No matter how you do it, it requires some sort of interpretation--which is slower--and simply having a macro is usually clearer. I've actually been considering their removal from Chaos for several years.
#define _(s, n) \ class T ## n \ CHAOS_PP_WHEN(n)(= CHAOS_PP_CAT(T, CHAOS_PP_DEC(n))) \ /**/
CHAOS_PP_EXPR(CHAOS_PP_ENUM(5, _))
#undef _
If lambdas are syntactically easier to use I would still encourage you to keep them in Chaos. While I am never against techniques which increase compile-time speed I strongly feel that ease of use in programming is far more important than waiting longer for a compilation to finish. Thanks very much for explaining more about Chaos.
On Sat, Nov 27, 2010 at 11:57 AM, Edward Diener <eldiener@tropicsoft.com>wrote:
What are the chances that you would be willing to propose Chaos for Boost even though it would work with only a subset of compilers, and specifically not with VC++ ? I personally think this would still be valuable for the authors of Boost libraries and for end-users, but I could understand your unwillingness to do so or other Boost developers and/or end-users unwillingness to accept Chaos into Boost because it does not work for a number of C++ compilers. But IMO that Chaos would provide easier preprocessing programming for even a subset of C++ implementations would make it worth using.
Seconding this. I know that not supporting VC++ is a no-go, but I'd hope that we could make an exception here. Boost is intended to target standards-compliant compilers, and if VC++ is horribly broken in one area then it's their problem. If Chaos were accepted, Boost.Preprocessor should likely still exist for backwards compatibility and for non-compliant preprocessors. Chaos would just be pushed as the main preprocessor metaprogramming library whereas the Boost.Preprocessor library as a whole would then be considered one big workaround. Just as Boost.Function has the interface workaround for compilers that can't handle the function type form, Chaos would have Boost.Preprocessor. Also, while Microsoft has said in the past that they have no plans to fix their preprocessor I wouldn't consider that to be the final word, no matter how many times they say it. Perhaps if a library like Chaos were in Boost their sentiment would change -- it wouldn't be the first time Boost has had such an effect. I understand that they don't want to break code that used to work, but even that can be dealt with if they really wanted to. For instance, I'd imagine that it should be possible for them to make pragmas that affect how macro definitions would be treated by the preprocessor. By default everything would work in the old, broken way, but if a library like Chaos were to need more compliant preprocessing then it could "push" a flag specifying more compliant behavior before its macro definitions and then "pop" at the end of the header. Any macro definitions that were written with the compliant preprocessor activated would work as expected when invoked (regardless of what flags are set at the call-site) and any macros that were written without it would exhibit the old, broken behavior upon invocation, meaning that everyone's old code would still work right out of the box. This also lets people use their non-compliant code in the same translation unit as a library like Chaos without the user ever having to deal with specifying different compiler options or pragmas himself. Whether or not Microsoft would actually take the time to do something such as that is questionable, but until a library like Chaos is in Boost I'd imagine the chances are much less. Out of curiosity, just how much of Chaos can VC++ handle at this point? Is it really entirely unusable as in almost nothing is supported? Is it all of the looping constructs? If lambdas are syntactically easier to use I would still encourage you to
keep them in Chaos. While I am never against techniques which increase compile-time speed I strongly feel that ease of use in programming is far more important than waiting longer for a compilation to finish.
Again, seconding this. Unless we are talking about an order of magnitude difference I'd imagine that the use of lambdas outweigh the cost. Particularly when prototyping, it is a huge advantage to be able to quickly write concise code that works. I've been doing a relatively large amount of preprocessor metaprogramming lately (the libraries I've been working on, AutoFunction and Generic, are now about 17,000 lines combined, the vast majority of which are internal macros used in the implementation of just a handful of top-level macros) and I'd likely be able to go much faster with a library such as Chaos. Once all my tests were passing I could go back and replace lambda expressions with hand-written macros if compile times proved to be a problem. At the moment I find myself constantly rewriting small portions of code during development that ends up being fairly tedious due to the amount of boiler-plate that is necessary when working with Boost.Preprocessor. So, please, submit Chaos for review. I don't know how many other people feel about the problem of supporting non-compliant compilers, but I think the benefits are too numerous for a library like Chaos, especially given that preprocessor metaprogramming is sort of niche anyway. -- -Matt Calabrese
On Sat, 2010-11-27 at 16:55 -0500, Matt Calabrese wrote:
On Sat, Nov 27, 2010 at 11:57 AM, Edward Diener <eldiener@tropicsoft.com>wrote:
What are the chances that you would be willing to propose Chaos for Boost even though it would work with only a subset of compilers, and specifically not with VC++ ? I personally think this would still be valuable for the authors of Boost libraries and for end-users, but I could understand your unwillingness to do so or other Boost developers and/or end-users unwillingness to accept Chaos into Boost because it does not work for a number of C++ compilers. But IMO that Chaos would provide easier preprocessing programming for even a subset of C++ implementations would make it worth using.
Seconding this. I know that not supporting VC++ is a no-go, but I'd hope that we could make an exception here. Boost is intended to target standards-compliant compilers, and if VC++ is horribly broken in one area then it's their problem. If Chaos were accepted, Boost.Preprocessor should likely still exist for backwards compatibility and for non-compliant preprocessors. Chaos would just be pushed as the main preprocessor metaprogramming library whereas the Boost.Preprocessor library as a whole would then be considered one big workaround. Just as Boost.Function has the interface workaround for compilers that can't handle the function type form, Chaos would have Boost.Preprocessor.
Also, while Microsoft has said in the past that they have no plans to fix their preprocessor I wouldn't consider that to be the final word, no matter how many times they say it. Perhaps if a library like Chaos were in Boost their sentiment would change -- it wouldn't be the first time Boost has had such an effect. I understand that they don't want to break code that used to work, but even that can be dealt with if they really wanted to. For instance, I'd imagine that it should be possible for them to make pragmas that affect how macro definitions would be treated by the preprocessor. By default everything would work in the old, broken way, but if a library like Chaos were to need more compliant preprocessing then it could "push" a flag specifying more compliant behavior before its macro definitions and then "pop" at the end of the header. Any macro definitions that were written with the compliant preprocessor activated would work as expected when invoked (regardless of what flags are set at the call-site) and any macros that were written without it would exhibit the old, broken behavior upon invocation, meaning that everyone's old code would still work right out of the box. This also lets people use their non-compliant code in the same translation unit as a library like Chaos without the user ever having to deal with specifying different compiler options or pragmas himself. Whether or not Microsoft would actually take the time to do something such as that is questionable, but until a library like Chaos is in Boost I'd imagine the chances are much less.
Out of curiosity, just how much of Chaos can VC++ handle at this point? Is it really entirely unusable as in almost nothing is supported? Is it all of the looping constructs?
If lambdas are syntactically easier to use I would still encourage you to
keep them in Chaos. While I am never against techniques which increase compile-time speed I strongly feel that ease of use in programming is far more important than waiting longer for a compilation to finish.
Again, seconding this. Unless we are talking about an order of magnitude difference I'd imagine that the use of lambdas outweigh the cost. Particularly when prototyping, it is a huge advantage to be able to quickly write concise code that works. I've been doing a relatively large amount of preprocessor metaprogramming lately (the libraries I've been working on, AutoFunction and Generic, are now about 17,000 lines combined, the vast majority of which are internal macros used in the implementation of just a handful of top-level macros) and I'd likely be able to go much faster with a library such as Chaos. Once all my tests were passing I could go back and replace lambda expressions with hand-written macros if compile times proved to be a problem. At the moment I find myself constantly rewriting small portions of code during development that ends up being fairly tedious due to the amount of boiler-plate that is necessary when working with Boost.Preprocessor.
So, please, submit Chaos for review. I don't know how many other people feel about the problem of supporting non-compliant compilers, but I think the benefits are too numerous for a library like Chaos, especially given that preprocessor metaprogramming is sort of niche anyway.
I agree, especially since one can generally use an external (standards-compliant) preprocessor with your compiler of choice/necessity, you just need to change your build procedure. Was this not one of the motivating factors behind Wave in the first place? Furthermore, libraries like Boost.MPL which provide pre-preprocessed headers essentially use an external preprocessor anyway. -Hal
On Sat, 27 Nov 2010 22:10:49 -0500, Hal Finkel wrote:
I agree, especially since one can generally use an external (standards-compliant) preprocessor with your compiler of choice/necessity, you just need to change your build procedure. Was this not one of the motivating factors behind Wave in the first place? Furthermore, libraries like Boost.MPL which provide pre-preprocessed headers essentially use an external preprocessor anyway.
If this is an option, I'd far prefer statically generated headers during a Boost install/build for broken compilers rather than altering the build process for client programs or libraries. Two things are important. First, compilers need to fix their preprocessors and there has to be a motivating reason for them to do so. Second, using Boost must not turn into something like having to use Qt's obnoxious moc. -Paul
On Sat, 27 Nov 2010 16:55:08 -0500, Matt Calabrese wrote:
Also, while Microsoft has said in the past that they have no plans to fix their preprocessor I wouldn't consider that to be the final word, no matter how many times they say it. Perhaps if a library like Chaos were in Boost their sentiment would change -- it wouldn't be the first time Boost has had such an effect. I understand that they don't want to break code that used to work, but even that can be dealt with if they really wanted to. For instance, I'd imagine that it should be possible for them to make pragmas that affect how macro definitions would be treated by the preprocessor. By default everything would work in the old, broken way, but if a library like Chaos were to need more compliant preprocessing then it could "push" a flag specifying more compliant behavior before its macro definitions and then "pop" at the end of the header. Any macro definitions that were written with the compliant preprocessor activated would work as expected when invoked (regardless of what flags are set at the call-site) and any macros that were written without it would exhibit the old, broken behavior upon invocation, meaning that everyone's old code would still work right out of the box. This also lets people use their non-compliant code in the same translation unit as a library like Chaos without the user ever having to deal with specifying different compiler options or pragmas himself. Whether or not Microsoft would actually take the time to do something such as that is questionable, but until a library like Chaos is in Boost I'd imagine the chances are much less.
It most likely wouldn't be quite that simple. (Discounting the work necessary to implement correct semantics in the first place.) The symbol table structure would most likely have to change, for example. Ultimately, I think that the means of definition would be virulent WRT to everything except entirely self-contained stuff in a library's implementation. I.e. marking macro definitions themselves as working one way or another depending on their context of definition would be a disaster.
Out of curiosity, just how much of Chaos can VC++ handle at this point? Is it really entirely unusable as in almost nothing is supported? Is it all of the looping constructs?
It is certainly all of the looping constructs, but generally-speaking, almost everything. Maybe a few concatenation macros and the like work properly (in the right context).
If lambdas are syntactically easier to use I would still encourage you to
keep them in Chaos. While I am never against techniques which increase compile-time speed I strongly feel that ease of use in programming is far more important than waiting longer for a compilation to finish.
Again, seconding this. Unless we are talking about an order of magnitude difference I'd imagine that the use of lambdas outweigh the cost.
In general, they are more expensive per invocation. They are not asymptotically more expensive. There are things that can be done to mitigate that, but I'm not doing it ATM. The way that the lambda mechanism in Chaos works is that it translates the expression into an IL form, performs argument substitution, and then "executes" the result. Right now, the initial translation is being done every time, but it could be done once at the beginning of an algorithm. Take a look at the result of the following: // test.cpp #include <chaos/preprocessor.h> #include CHAOS_PP_PLACEHOLDERS(1) #define e \ CHAOS_PP_CAT_(_0, CHAOS_PP_CAT_(_1, _2)) \ /**/ // unparsed expression... e // parsed (and substituted) expression... CHAOS_PP_PARSE( e, A, B, C ) // "executed" expression... CHAOS_PP_EXECUTE( CHAOS_PP_PARSE( e, A, B, C ) ) // final result... CHAOS_PP_EXPR( CHAOS_PP_EXECUTE( CHAOS_PP_PARSE( e, A, B, C ) ) ) g++ -E -std=c++0x -I $CHAOS_ROOT -D CHAOS_PP_VARIADICS test.cpp The parse effectively flattens all of the nesting out of the expression. In this implementation, however, it is also doing argument substitution. However, it doesn't _have_ to do that.
Particularly when prototyping, it is a huge advantage to be able to quickly write concise code that works. I've been doing a relatively large amount of preprocessor metaprogramming lately (the libraries I've been working on, AutoFunction and Generic, are now about 17,000 lines combined, the vast majority of which are internal macros used in the implementation of just a handful of top-level macros) and I'd likely be able to go much faster with a library such as Chaos. Once all my tests were passing I could go back and replace lambda expressions with hand-written macros if compile times proved to be a problem.
I pretty much guarantee that compile times would be significantly higher. Furthermore, I don't think that a library author could ever justify increasing compile times that signficantly for a little sugar. I can see it in application code, but not in libraries--and libraries tend to be the place where code generation is more applicable.
At the moment I find myself constantly rewriting small portions of code during development that ends up being fairly tedious due to the amount of boiler-plate that is necessary when working with Boost.Preprocessor.
Can you give me an example of the boilerplate which you are referring to?
So, please, submit Chaos for review. I don't know how many other people feel about the problem of supporting non-compliant compilers, but I think the benefits are too numerous for a library like Chaos, especially given that preprocessor metaprogramming is sort of niche anyway.
I'd rather have a stronger feeling for the answer that question before bothering with a review process. Here's the basic questions: 1) What happens when a library cannot be implemented on a broken compiler--particularly a major (and otherwise fairly decent) compiler like VC++? 2) A fundamental constraint of Chaos is that it is supposed to be a _clean_ reference implementation--i.e. no workarounds at all, not just no major workarounds. The idea is to push the boundaries of what can be done, not "fix" vendor's compilers for them by way of hacks. Workarounds of any type compromise this goal. Once a workaround is in place, the interface gets used and has to be supported. This restricts the freedom of the implementation, provides no incentive for compilers to be fixed, and ultimately ends the lifetime of the library. 3) If Boost were to internally use Chaos, the build process would have to include a bunch of header generation (using a Wave driver, for example, that emulates a compiler).
On Sat, 27 Nov 2010 11:57:41 -0500, Edward Diener wrote:
I actually just downloaded the tar.gz file. While I could get Tortoise CVS and use it to keep up to date, I have gotten lazy enough where since its not in SVN ( or Git ) I tend not to want to have to deal with CVS anymore. But that's my problem.
I didn't know that there was a tar.gz file. AFAIR, I've never put one up. The project has existed since before Sourceforge had Subversion services. At that time, Boost was hosted there in a CVS repository.
What are the chances that you would be willing to propose Chaos for Boost even though it would work with only a subset of compilers, and specifically not with VC++ ? I personally think this would still be valuable for the authors of Boost libraries and for end-users, but I could understand your unwillingness to do so or other Boost developers and/or end-users unwillingness to accept Chaos into Boost because it does not work for a number of C++ compilers. But IMO that Chaos would provide easier preprocessing programming for even a subset of C++ implementations would make it worth using.
I'm not against it except that I will not compromise Chaos' fundamental principle--reference implementation with no workarounds. It is past time that compilers improve and/or get fixed rather than constant hacks permeating everything. Implementing a correct preprocessor is no where near as hard as implementing the rest of a modern C++ compiler.
The most obvious (considering the context) is support for variadics/ placemarkers from the ground up. Probably more importantly, Chaos generalizes recursion and all higher-order constructs are reentrant without replication of their implementations.
That is an excellent advantage in simplification over current Boost PP despite the fact that you have made Boost PP quite usable even with the recursion difficulties.
To a degree. However, in the pp-lib, there are a few reentrant constructs, each having their own "state". Macros built on top of those are generally _not_ reentrant. I hear about difficulties caused by that comparatively frequently. The problem is fundamental and discourages reuse (both in the library and outside of it).
I totally agree and made a point of saying in my variadic_macro_data library that variadic macros real advantage over Boost PP sequences is largely in familiarity/ease of syntax for end-users. I am glad you have also found their usage in data structures valuable.
I would say that "sequences" ala (a)(b)(c) have become the most effective preprocessor data structure used in Boost. Chaos supports non-unary sequences natively and variadic sequences "indirectly". I.e. the higher- order sequence algorithms in Chaos directly call user-supplied operations, predicates, etc., with the contents of the sequence element. E.g. #define macro(s, x, y, c) + c * (x - y) CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH( macro, (1, 2)(3, 4)(5, 6), 7 )) => + 7 * (1 - 2) + 7 * (3 - 4) + 7 * (5 - 6) More care has to be taken when the sequence itself is truly variadic (meaning the "arity" of elements in the sequence is different from one element to another) such as (1)(2, 3)(4, 5, 6). Chaos has alternative higher-order macros for those types of sequences (which it calls, unsurprisingly, "variadic sequences"). E.g. #define macro(s, e, c) + c * e CHAOS_PP_EXPR(CHAOS_PP_VARIADIC_SEQ_FOR_EACH( macro, (1)(2, 3)(4, 5, 6), 7 )) => + 7 * (1) + 7 * (2, 3) + 7 * (4, 5, 6) The basic difference is that the elements of the sequence are passed to user-defined operations (etc.) as "tuples" whose contents are the element. E.g. CHAOS_PP_EXPR( CHAOS_PP_SEQ_FOR_EACH( a, (1, 2)(3, 4)(5, 6) ) CHAOS_PP_VARIADIC_SEQ_FOR_EACH( b, (1)(2, 3)(4, 5, 6) ) ) => a(2, 1, 2) a(2, 3, 4) a(2, 5, 6) b(2, (1)) b(2, (2, 3)) b(2, (4, 5, 6)) More generally: CHAOS_PP_EXPR( CHAOS_PP_FOR_EACH( a, (CHAOS_PP_SEQ) (1)(2)(3) ) CHAOS_PP_FOR_EACH( b, (CHAOS_PP_TUPLE) (1, 2, 3) ) CHAOS_PP_FOR_EACH( c, (CHAOS_PP_LIST) (1, (2, (3, ...))) ) CHAOS_PP_FOR_EACH( d, (CHAOS_PP_VARIADIC_SEQ) (1)(2)(3) ) ) I.e. a sequence is the only data structure defined by Chaos that can carry non-unary (or variadic) content. ----- // with lambda...
CHAOS_PP_EXPR(CHAOS_PP_ENUM( 5, CHAOS_PP_PRIMITIVE_CAT_(class T, CHAOS_PP_ARG(1)) CHAOS_PP_WHEN_(CHAOS_PP_ARG(1))( CHAOS_PP_CAT_(= T, CHAOS_PP_DEC_(CHAOS_PP_ARG(1))) ) ))
// without lambda...
#define _(s, n) \ class T ## n \ CHAOS_PP_WHEN(n)(= CHAOS_PP_CAT(T, CHAOS_PP_DEC(n))) \ /**/
CHAOS_PP_EXPR(CHAOS_PP_ENUM(5, _))
#undef _
If lambdas are syntactically easier to use I would still encourage you to keep them in Chaos. While I am never against techniques which increase compile-time speed I strongly feel that ease of use in programming is far more important than waiting longer for a compilation to finish.
I'm not sure that they _are_ syntactically easier. I cannot define placeholders like _1, _2, and _3 (at least, not permanently). I find CHAOS_PP_ARG(n) makes things more wordy, not less. I have a facility intended to "temporarily" define placeholders like _1, _2, and _3... #include CHAOS_PP_PLACEHOLDERS(1) CHAOS_PP_EXPR(CHAOS_PP_ENUM( 5, CHAOS_PP_PRIMITIVE_CAT_(class T, _1) CHAOS_PP_WHEN_(_1)( CHAOS_PP_CAT_(= T, CHAOS_PP_DEC_(_1)) ) )) #include CHAOS_PP_PLACEHOLDERS(0) ...but that simply trades one type of line (defining and undefining a macro) for another (pair of includes). Also, because the preprocessor doesn't provide tools to examine arbitrary tokens, the mechanism requires a lot of stuff to be "escaped". I.e. you can't have template<class T = _1, class U = _2> class XYZ { }; because the mechanism cannot find the _1 and _2. Instead, you have to have something (minimally) like #include CHAOS_PP_PLACEHOLDERS(1) CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH( CHAOS_PP_ESCAPE(template<class T =) _1 CHAOS_PP_ESCAPE(, class U =) _2> class XYZ { };, (A, B)(X, Y)(P, Q) )) #include CHAOS_PP_PLACEHOLDERS(0) I do not find that better than the alternative of just: #define _(s, a, b) \ template<class T = a, class U = b> class XYZ { }; \ /**/ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH( _, (A, B)(X, Y)(P, Q) )) #undef _ So, they might not be too bad in terms of expressions in the language (i.e. macro invocations, etc.), but they aren't that great at representing output forms (i.e. the underlying language, C++-proper in this case) which is, at the end of the day, mostly the point.
On 11/27/2010 10:26 PM, Paul Mensonides wrote:
On Sat, 27 Nov 2010 11:57:41 -0500, Edward Diener wrote:
I actually just downloaded the tar.gz file. While I could get Tortoise CVS and use it to keep up to date, I have gotten lazy enough where since its not in SVN ( or Git ) I tend not to want to have to deal with CVS anymore. But that's my problem.
I didn't know that there was a tar.gz file. AFAIR, I've never put one up. The project has existed since before Sourceforge had Subversion services. At that time, Boost was hosted there in a CVS repository.
If you could get the Chaos files in a subversion repository I think it would be helpful. I have nothing in particular against CVS other than the fact that like most programmers I find subversion much more flexible to use.
What are the chances that you would be willing to propose Chaos for Boost even though it would work with only a subset of compilers, and specifically not with VC++ ? I personally think this would still be valuable for the authors of Boost libraries and for end-users, but I could understand your unwillingness to do so or other Boost developers and/or end-users unwillingness to accept Chaos into Boost because it does not work for a number of C++ compilers. But IMO that Chaos would provide easier preprocessing programming for even a subset of C++ implementations would make it worth using.
I'm not against it except that I will not compromise Chaos' fundamental principle--reference implementation with no workarounds. It is past time that compilers improve and/or get fixed rather than constant hacks permeating everything. Implementing a correct preprocessor is no where near as hard as implementing the rest of a modern C++ compiler.
I would not think anyone would want you to manipulate Chaos for compilers with broken preprocessing. The whole idea of Chaos in Boost should be that it strictly requires a compiler with a correct C++ standard preprocessor. As others have commented in this thread, it might even be an impetus for compilers which have preprocessor bugs to fix them. As you mention, implementing a correct preprocessor is easier than implementing correct C++ in general.
The most obvious (considering the context) is support for variadics/ placemarkers from the ground up. Probably more importantly, Chaos generalizes recursion and all higher-order constructs are reentrant without replication of their implementations.
That is an excellent advantage in simplification over current Boost PP despite the fact that you have made Boost PP quite usable even with the recursion difficulties.
To a degree. However, in the pp-lib, there are a few reentrant constructs, each having their own "state". Macros built on top of those are generally _not_ reentrant. I hear about difficulties caused by that comparatively frequently. The problem is fundamental and discourages reuse (both in the library and outside of it).
Understood.
I totally agree and made a point of saying in my variadic_macro_data library that variadic macros real advantage over Boost PP sequences is largely in familiarity/ease of syntax for end-users. I am glad you have also found their usage in data structures valuable.
I would say that "sequences" ala (a)(b)(c) have become the most effective preprocessor data structure used in Boost. Chaos supports non-unary sequences natively and variadic sequences "indirectly". I.e. the higher- order sequence algorithms in Chaos directly call user-supplied operations, predicates, etc., with the contents of the sequence element.
I have always found Boost PP sequences to be more than adequate enough to be used internally as opposed to variadic macro syntax. My main impetus for the variadic_macro_data library was to allow macros with variadic data to be specified for end-users for whom AMACRO(x,y,z) feels and looks more natural than AMACRO((x),(y),(z)). Other than that Boost PP sequences ( and arrays, lists, and tuples ) have much richer functionality than variadic data. But I think it was a very good thing that the C++ standard committee recognized that variadic macros were necessary from an ease of use standpoint, and of course Boost PP is not part of the C++ standard.
E.g.
#define macro(s, x, y, c) + c * (x - y)
CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH( macro, (1, 2)(3, 4)(5, 6), 7 ))
=> + 7 * (1 - 2) + 7 * (3 - 4) + 7 * (5 - 6)
Very cool !
More care has to be taken when the sequence itself is truly variadic (meaning the "arity" of elements in the sequence is different from one element to another) such as (1)(2, 3)(4, 5, 6). Chaos has alternative higher-order macros for those types of sequences (which it calls, unsurprisingly, "variadic sequences"). E.g.
#define macro(s, e, c) + c * e
CHAOS_PP_EXPR(CHAOS_PP_VARIADIC_SEQ_FOR_EACH( macro, (1)(2, 3)(4, 5, 6), 7 ))
=> + 7 * (1) + 7 * (2, 3) + 7 * (4, 5, 6)
The basic difference is that the elements of the sequence are passed to user-defined operations (etc.) as "tuples" whose contents are the element. E.g.
CHAOS_PP_EXPR( CHAOS_PP_SEQ_FOR_EACH( a, (1, 2)(3, 4)(5, 6) ) CHAOS_PP_VARIADIC_SEQ_FOR_EACH( b, (1)(2, 3)(4, 5, 6) ) )
=> a(2, 1, 2) a(2, 3, 4) a(2, 5, 6) b(2, (1)) b(2, (2, 3)) b(2, (4, 5, 6))
More generally:
CHAOS_PP_EXPR( CHAOS_PP_FOR_EACH( a, (CHAOS_PP_SEQ) (1)(2)(3) ) CHAOS_PP_FOR_EACH( b, (CHAOS_PP_TUPLE) (1, 2, 3) ) CHAOS_PP_FOR_EACH( c, (CHAOS_PP_LIST) (1, (2, (3, ...))) ) CHAOS_PP_FOR_EACH( d, (CHAOS_PP_VARIADIC_SEQ) (1)(2)(3) ) )
I.e. a sequence is the only data structure defined by Chaos that can carry non-unary (or variadic) content.
The choice of data structures in Boost PP is a big plus so I would expect the same from Chaos.
-----
// with lambda...
CHAOS_PP_EXPR(CHAOS_PP_ENUM( 5, CHAOS_PP_PRIMITIVE_CAT_(class T, CHAOS_PP_ARG(1)) CHAOS_PP_WHEN_(CHAOS_PP_ARG(1))( CHAOS_PP_CAT_(= T, CHAOS_PP_DEC_(CHAOS_PP_ARG(1))) ) ))
// without lambda...
#define _(s, n) \ class T ## n \ CHAOS_PP_WHEN(n)(= CHAOS_PP_CAT(T, CHAOS_PP_DEC(n))) \ /**/
CHAOS_PP_EXPR(CHAOS_PP_ENUM(5, _))
#undef _
In a way the lambda use above is syntactically easier because one does not have to break up the usage into two pieces. Often syntactically easier does not mean shorter but more understandable in the way something is constructed.
If lambdas are syntactically easier to use I would still encourage you to keep them in Chaos. While I am never against techniques which increase compile-time speed I strongly feel that ease of use in programming is far more important than waiting longer for a compilation to finish.
I'm not sure that they _are_ syntactically easier. I cannot define placeholders like _1, _2, and _3 (at least, not permanently). I find CHAOS_PP_ARG(n) makes things more wordy, not less. I have a facility intended to "temporarily" define placeholders like _1, _2, and _3...
#include CHAOS_PP_PLACEHOLDERS(1)
CHAOS_PP_EXPR(CHAOS_PP_ENUM( 5, CHAOS_PP_PRIMITIVE_CAT_(class T, _1) CHAOS_PP_WHEN_(_1)( CHAOS_PP_CAT_(= T, CHAOS_PP_DEC_(_1)) ) ))
#include CHAOS_PP_PLACEHOLDERS(0)
...but that simply trades one type of line (defining and undefining a macro) for another (pair of includes). Also, because the preprocessor doesn't provide tools to examine arbitrary tokens, the mechanism requires a lot of stuff to be "escaped". I.e. you can't have
template<class T = _1, class U = _2> class XYZ { };
because the mechanism cannot find the _1 and _2. Instead, you have to have something (minimally) like
#include CHAOS_PP_PLACEHOLDERS(1)
CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH( CHAOS_PP_ESCAPE(template<class T =) _1 CHAOS_PP_ESCAPE(, class U =) _2> class XYZ { };, (A, B)(X, Y)(P, Q) ))
#include CHAOS_PP_PLACEHOLDERS(0)
I do not find that better than the alternative of just:
#define _(s, a, b) \ template<class T = a, class U = b> class XYZ { }; \ /**/
CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH( _, (A, B)(X, Y)(P, Q) ))
#undef _
So, they might not be too bad in terms of expressions in the language (i.e. macro invocations, etc.), but they aren't that great at representing output forms (i.e. the underlying language, C++-proper in this case) which is, at the end of the day, mostly the point.
It is up to you whether or not you find lambda syntax within Chaos easier to use and/or flexible enough to justify their inclusion in Chaos. My general point is that unless compiler time increases to a very large extent when creating C++ constructs, I find that programmers complaining about excess compiler time when using C++ ( or any other language ) are off base. In today's programming world valuable programming work can almost always be done while a programmer waits for a compile/build to finish.
On Sun, 28 Nov 2010 08:08:15 -0500, Edward Diener wrote:
I didn't know that there was a tar.gz file. AFAIR, I've never put one up. The project has existed since before Sourceforge had Subversion services. At that time, Boost was hosted there in a CVS repository.
If you could get the Chaos files in a subversion repository I think it would be helpful. I have nothing in particular against CVS other than the fact that like most programmers I find subversion much more flexible to use.
I was planning on doing version 2 in the Subversion repository there. I also prefer SVN over CVS, but I also don't see any immediate gain by a switch over. This is particularly true as Chaos is a one developer project.
In a way the lambda use above is syntactically easier because one does not have to break up the usage into two pieces. Often syntactically easier does not mean shorter but more understandable in the way something is constructed.
In some ways, yes.
It is up to you whether or not you find lambda syntax within Chaos easier to use and/or flexible enough to justify their inclusion in Chaos. My general point is that unless compiler time increases to a very large extent when creating C++ constructs, I find that programmers complaining about excess compiler time when using C++ ( or any other language ) are off base. In today's programming world valuable programming work can almost always be done while a programmer waits for a compile/build to finish.
Well, here are some concrete timings. The following are two source files that do the same thing. They generate a binary sequence of pairs of the following form (1, n)(2, n - 1) ... (n - 1, 2)(n, 1) where n equals 100. (This is not interesting, it merely gives opportunity use multiple lambdas.) The first source file does everything with lambdas. The second source file does everything without lambdas. // 1.cpp #include <chaos/preprocessor.h> ??=include CHAOS_PP_PLACEHOLDERS(1) #define ZIP(a, b) \ CHAOS_PP_VARIADIC_ELEM( \ 0, \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ _2 (_1, CHAOS_PP_SEQ_HEAD_(_3)) CHAOS_PP_COMMA_() \ CHAOS_PP_SEQ_TAIL_(_3), \ a,, b \ )) \ ) \ /**/ #define NATURALS(n) \ CHAOS_PP_EXPR(CHAOS_PP_REPEAT( \ n, \ (CHAOS_PP_INC_(_1)) \ )) \ /**/ #define GAUSS(n) \ GAUSS_I(NATURALS(n)) \ /**/ #define GAUSS_I(seq) \ ZIP( \ seq, \ CHAOS_PP_SEQ_REVERSE(seq) \ ) \ /**/ GAUSS(100) ??=include CHAOS_PP_PLACEHOLDERS(0) // 2.cpp #include <chaos/preprocessor.h> #define ZIP(a, b) \ CHAOS_PP_VARIADIC_ELEM( \ 0, \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ ZIP_O, a,, b \ )) \ ) \ /**/ #define ZIP_O(s, e, r, b) \ r (e, CHAOS_PP_SEQ_HEAD(b)), CHAOS_PP_SEQ_TAIL(b) \ /**/ #define NATURALS(n) \ CHAOS_PP_EXPR(CHAOS_PP_REPEAT( \ n, \ NATURALS_M \ )) \ /**/ #define NATURALS_M(s, n) (CHAOS_PP_INC(n)) #define GAUSS(n) \ GAUSS_I(NATURALS(n)) \ /**/ #define GAUSS_I(seq) \ ZIP( \ seq, \ CHAOS_PP_SEQ_REVERSE(seq) \ ) \ /**/ GAUSS(100) The first file (w/lambda) takes about 49.4 seconds to preprocess with g++ on my Linux VM. The second file (w/o lambda) takes about 1.9 seconds to preprocess. That is about 26 times faster w/o lambda. In this particular case, the two operations that use lambdas are both O(n), so use your imagination for how bad it would get if an algorithm was non- linear. To me, that kind of performance drain is simply unacceptable for use in library code unless that code is self-contained (i.e. the library isn't providing macros that do this as a user-interface to clients) and the n is relatively small. That isn't an argument for total exclusion of the mechanism (or a similar mechanism), as non-library code could still use it if desired. However, the main context for code generation is in libraries or in application code using interfaces provided by libraries (i.e. macros published as interfaces by libraries). I'm simply not sure that 1) the implementation cost is worth the benefit, and 2) whether the tacit endorsement, by way of the existence of the mechanism, is a good idea. In reference to prototyping, there is a cost there as well. In the course of development, one of the things that I often do is strategically stop certain macros from expanding so that I can see what the partial results are. For example, in the following #define SUM(seq) \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ SUM_O, seq, 0 \ )) \ /**/ #define SUM_O(x, y) CHAOS_PP_ADD(x, y SUM( (1)(2)(3) ) ...I've made two errors. The first is that I'm missing the 's' parameter (which is the recursion state of the library) in the SUM_O folding operation. In this case, the problem is somewhat easy to find, but I can see what's going on just by putting a period after the SUM_O in the definition of SUM. #define SUM(seq) \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ SUM_O., seq, 0 \ )) \ /**/ #define SUM_O(x, y) CHAOS_PP_ADD(x, y SUM( (1)(2)(3) ) This yields => SUM_O.(3, 3, SUM_O.(3, 2, SUM_O.(3, 1, 0))) Given that I'm familiar with Chaos' recursion mechanisms, I immediately spot my error and fix it. #define SUM(seq) \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ SUM_O, seq, 0 \ )) \ /**/ #define SUM_O(s, x, y) CHAOS_PP_ADD(x, y SUM( (1)(2)(3) ) This time, however, I run afoul of my second error, the missing closing parenthesis in the invocation of ADD. Here, GCC's error messages are even more unhelpful. I get => unterminated argument list invoking macro "CHAOS_IP_EXPR_3" However, I know that the fold is generated correctly because of the debugging process for bug #1. Therefore, I strategically stop the expansion of ADD. #define SUM(seq) \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ SUM_O, seq, 0 \ )) \ /**/ #define SUM_O(s, x, y) CHAOS_PP_ADD.(x, y SUM( (1)(2)(3) ) However, I get the exact same error. Therefore, I know that the error is not occurring because ADD is getting nonsensical input. Eventually, I figure out my obvious problem and fix it. The issue here is not really this particular example so much as it is debugging techniques with preprocessor stuff. This example is trivial and obvious. Other times, the result is a mountain of garbage for output that makes template error messages look readable. (I'm sure anybody that has done any significant preprocessor generation has seen this.) Some of the algorithms in Chaos make exponential use of the recursion backend in such a way that if machine memory was unlimited, it would literally take several millenia to finish printing the output to the screen. It is techniques like those above that can be used to isolate bug locations, condense and structure output, or make the output finite (in a practical sense) thus producing something that has a hope of being deciphered. When lambda is involved, you simply cannot do things like that that easily. Because of that, whenever I'm illustrating the lambda mechanism, I virtually always prototype whatever I'm doing w/o lambda and then port it. Regards, Paul Mensonides
On 11/29/2010 7:58 AM, Paul Mensonides wrote:
On Sun, 28 Nov 2010 08:08:15 -0500, Edward Diener wrote:
I didn't know that there was a tar.gz file. AFAIR, I've never put one up. The project has existed since before Sourceforge had Subversion services. At that time, Boost was hosted there in a CVS repository.
If you could get the Chaos files in a subversion repository I think it would be helpful. I have nothing in particular against CVS other than the fact that like most programmers I find subversion much more flexible to use.
I was planning on doing version 2 in the Subversion repository there. I also prefer SVN over CVS, but I also don't see any immediate gain by a switch over. This is particularly true as Chaos is a one developer project.
The gain is only that people interested in Chaos will find it easier to obtain the latest code if, like me, they largely forego the use of CVS. I do admit that there are still plenty of other projects on the Net that still use CVS, so maybe I should just give in and install a CVS front end.
In a way the lambda use above is syntactically easier because one does not have to break up the usage into two pieces. Often syntactically easier does not mean shorter but more understandable in the way something is constructed.
In some ways, yes.
It is up to you whether or not you find lambda syntax within Chaos easier to use and/or flexible enough to justify their inclusion in Chaos. My general point is that unless compiler time increases to a very large extent when creating C++ constructs, I find that programmers complaining about excess compiler time when using C++ ( or any other language ) are off base. In today's programming world valuable programming work can almost always be done while a programmer waits for a compile/build to finish.
Well, here are some concrete timings. The following are two source files that do the same thing. They generate a binary sequence of pairs of the following form (1, n)(2, n - 1) ... (n - 1, 2)(n, 1) where n equals 100. (This is not interesting, it merely gives opportunity use multiple lambdas.) The first source file does everything with lambdas. The second source file does everything without lambdas.
// 1.cpp
#include<chaos/preprocessor.h>
??=include CHAOS_PP_PLACEHOLDERS(1)
#define ZIP(a, b) \ CHAOS_PP_VARIADIC_ELEM( \ 0, \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ _2 (_1, CHAOS_PP_SEQ_HEAD_(_3)) CHAOS_PP_COMMA_() \ CHAOS_PP_SEQ_TAIL_(_3), \ a,, b \ )) \ ) \ /**/
#define NATURALS(n) \ CHAOS_PP_EXPR(CHAOS_PP_REPEAT( \ n, \ (CHAOS_PP_INC_(_1)) \ )) \ /**/
#define GAUSS(n) \ GAUSS_I(NATURALS(n)) \ /**/ #define GAUSS_I(seq) \ ZIP( \ seq, \ CHAOS_PP_SEQ_REVERSE(seq) \ ) \ /**/
GAUSS(100)
??=include CHAOS_PP_PLACEHOLDERS(0)
// 2.cpp
#include<chaos/preprocessor.h>
#define ZIP(a, b) \ CHAOS_PP_VARIADIC_ELEM( \ 0, \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ ZIP_O, a,, b \ )) \ ) \ /**/ #define ZIP_O(s, e, r, b) \ r (e, CHAOS_PP_SEQ_HEAD(b)), CHAOS_PP_SEQ_TAIL(b) \ /**/
#define NATURALS(n) \ CHAOS_PP_EXPR(CHAOS_PP_REPEAT( \ n, \ NATURALS_M \ )) \ /**/ #define NATURALS_M(s, n) (CHAOS_PP_INC(n))
#define GAUSS(n) \ GAUSS_I(NATURALS(n)) \ /**/ #define GAUSS_I(seq) \ ZIP( \ seq, \ CHAOS_PP_SEQ_REVERSE(seq) \ ) \ /**/
GAUSS(100)
The first file (w/lambda) takes about 49.4 seconds to preprocess with g++ on my Linux VM. The second file (w/o lambda) takes about 1.9 seconds to preprocess.
The 49.4 seconds really is slow. Despite my protestations about programmers complaining about compile times, if it took nearly a minute to compile a single source file I would find this very unusual. I am not sure what you mean by a Linux VM however. Are you running under VMWare, or something similar ? Maybe that explains the slow time as opposed to running natively. I will try it myself and see the difference in timing, but I admit I am a Windows person largely, although I have a number of Linux distros into which I occasionally boot, so I will try it under Windows with mingw/gcc.
That is about 26 times faster w/o lambda. In this particular case, the two operations that use lambdas are both O(n), so use your imagination for how bad it would get if an algorithm was non- linear. To me, that kind of performance drain is simply unacceptable for use in library code unless that code is self-contained (i.e. the library isn't providing macros that do this as a user-interface to clients) and the n is relatively small.
That isn't an argument for total exclusion of the mechanism (or a similar mechanism), as non-library code could still use it if desired. However, the main context for code generation is in libraries or in application code using interfaces provided by libraries (i.e. macros published as interfaces by libraries). I'm simply not sure that 1) the implementation cost is worth the benefit, and 2) whether the tacit endorsement, by way of the existence of the mechanism, is a good idea.
I would guess that if you found maintaining the lambda interface to Chaos was too much work, particularly as you decide to add any new functionality to Chaos, you could drop it. Otherwise just documenting that it is much slower during compilation IMO would be adequate.
In reference to prototyping, there is a cost there as well. In the course of development, one of the things that I often do is strategically stop certain macros from expanding so that I can see what the partial results are. For example, in the following
#define SUM(seq) \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ SUM_O, seq, 0 \ )) \ /**/ #define SUM_O(x, y) CHAOS_PP_ADD(x, y
SUM( (1)(2)(3) )
...I've made two errors. The first is that I'm missing the 's' parameter (which is the recursion state of the library) in the SUM_O folding operation. In this case, the problem is somewhat easy to find, but I can see what's going on just by putting a period after the SUM_O in the definition of SUM.
#define SUM(seq) \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ SUM_O., seq, 0 \ )) \ /**/ #define SUM_O(x, y) CHAOS_PP_ADD(x, y
SUM( (1)(2)(3) )
This yields
=> SUM_O.(3, 3, SUM_O.(3, 2, SUM_O.(3, 1, 0)))
I have never seen this technique before for debugging macros. Is this something built-in to Chaos or something the compiler reacts to ?
Given that I'm familiar with Chaos' recursion mechanisms, I immediately spot my error and fix it.
#define SUM(seq) \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ SUM_O, seq, 0 \ )) \ /**/ #define SUM_O(s, x, y) CHAOS_PP_ADD(x, y
SUM( (1)(2)(3) )
This time, however, I run afoul of my second error, the missing closing parenthesis in the invocation of ADD. Here, GCC's error messages are even more unhelpful. I get
=> unterminated argument list invoking macro "CHAOS_IP_EXPR_3"
However, I know that the fold is generated correctly because of the debugging process for bug #1. Therefore, I strategically stop the expansion of ADD.
#define SUM(seq) \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ SUM_O, seq, 0 \ )) \ /**/ #define SUM_O(s, x, y) CHAOS_PP_ADD.(x, y
SUM( (1)(2)(3) )
However, I get the exact same error. Therefore, I know that the error is not occurring because ADD is getting nonsensical input.
Eventually, I figure out my obvious problem and fix it.
The issue here is not really this particular example so much as it is debugging techniques with preprocessor stuff. This example is trivial and obvious. Other times, the result is a mountain of garbage for output that makes template error messages look readable. (I'm sure anybody that has done any significant preprocessor generation has seen this.)
Whenever I have gotten error messages from incorrect macros, using Boost PP, I just break down the macro from the inside as much as possible, into test pieces, and turn on the compiler switch that gives me proprocessed output.
Some of the algorithms in Chaos make exponential use of the recursion backend in such a way that if machine memory was unlimited, it would literally take several millenia to finish printing the output to the screen. It is techniques like those above that can be used to isolate bug locations, condense and structure output, or make the output finite (in a practical sense) thus producing something that has a hope of being deciphered.
When lambda is involved, you simply cannot do things like that that easily. Because of that, whenever I'm illustrating the lambda mechanism, I virtually always prototype whatever I'm doing w/o lambda and then port it.
Understood. Thanks for the explanation.
On Mon, 29 Nov 2010 09:55:40 -0500, Edward Diener wrote:
The first file (w/lambda) takes about 49.4 seconds to preprocess with g++ on my Linux VM. The second file (w/o lambda) takes about 1.9 seconds to preprocess.
The 49.4 seconds really is slow. Despite my protestations about programmers complaining about compile times, if it took nearly a minute to compile a single source file I would find this very unusual. I am not sure what you mean by a Linux VM however. Are you running under VMWare, or something similar ? Maybe that explains the slow time as opposed to running natively. I will try it myself and see the difference in timing, but I admit I am a Windows person largely, although I have a number of Linux distros into which I occasionally boot, so I will try it under Windows with mingw/gcc.
Its a Linux distro running under VirtualBox on my laptop. Running outside the VM would be somewhat faster. The point being, however, is not the actual timings, but the ratio between them. The VM has enough RAM that disk swap is not kicking in.
I would guess that if you found maintaining the lambda interface to Chaos was too much work, particularly as you decide to add any new functionality to Chaos, you could drop it. Otherwise just documenting that it is much slower during compilation IMO would be adequate.
Maintaining the interface isn't terribly difficult.
#define SUM(seq) \ CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOLD_LEFT( \ SUM_O., seq, 0 \ )) \ /**/ #define SUM_O(x, y) CHAOS_PP_ADD(x, y
SUM( (1)(2)(3) )
This yields
=> SUM_O.(3, 3, SUM_O.(3, 2, SUM_O.(3, 1, 0)))
I have never seen this technique before for debugging macros. Is this something built-in to Chaos or something the compiler reacts to ?
It is nothing fancy; I simply passed "SUM_O." instead of "SUM_O" to the fold algorithm.
Whenever I have gotten error messages from incorrect macros, using Boost PP, I just break down the macro from the inside as much as possible, into test pieces, and turn on the compiler switch that gives me proprocessed output.
That does have to be done sometimes. Regards, Paul Mensonides
Whenever I have gotten error messages from incorrect macros, using Boost PP, I just break down the macro from the inside as much as possible, into test pieces, and turn on the compiler switch that gives me proprocessed output.
That does have to be done sometimes.
Do you still remember the macro tracing facilities in Wave? It should be usable for this kind of problems, really. Regards Hartmut --------------- http://boost-spirit.com
On 11/29/2010 10:52 AM, Hartmut Kaiser wrote:
Whenever I have gotten error messages from incorrect macros, using Boost PP, I just break down the macro from the inside as much as possible, into test pieces, and turn on the compiler switch that gives me proprocessed output.
That does have to be done sometimes.
Do you still remember the macro tracing facilities in Wave? It should be usable for this kind of problems, really.
Thanks for waking me up about this Wave facility. I will definitely use it when needed. I admit I am guilty at not looking at Wave and seeing what its functionality is, but I will do so now. I was aware of it in general, as having the ability to allow the end-user to programatically intercept the elements of a preprocessed file, but I had not specifically lloked at its functionality to see how I might use it for myself.
Hello, I use the C++ plugin for Eclipse to single-step through a macro evaluation. You can do this without compiling first, just while editing your source files and adding or modifying macro definitions. Very convenient, at least as long as you stick to a C++ standard. It also gives you an idea of the complexity of an expansion by counting the replacement steps. Regards Wolf Lammen -------- Original-Nachricht --------
Datum: Mon, 29 Nov 2010 15:17:43 -0500 Von: Edward Diener <eldiener@tropicsoft.com> An: boost@lists.boost.org Betreff: Re: [boost] [Preprocessor] Adding variadic macros support
On 11/29/2010 10:52 AM, Hartmut Kaiser wrote:
Whenever I have gotten error messages from incorrect macros, using Boost PP, I just break down the macro from the inside as much as possible, into test pieces, and turn on the compiler switch that gives me proprocessed output.
That does have to be done sometimes.
Do you still remember the macro tracing facilities in Wave? It should be usable for this kind of problems, really.
Thanks for waking me up about this Wave facility. I will definitely use it when needed. I admit I am guilty at not looking at Wave and seeing what its functionality is, but I will do so now. I was aware of it in general, as having the ability to allow the end-user to programatically intercept the elements of a preprocessed file, but I had not specifically lloked at its functionality to see how I might use it for myself.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
-- GRATIS! Movie-FLAT mit über 300 Videos. Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome
On 11/27/2010 6:36 AM, Paul Mensonides wrote: [...] [Apologies for somewhat hijacking this thread...although Chaos does have variadic macro support so I still think it's on-topic.] Paul: Thank you very much for explaining some of the capabilities and limitations of Chaos. I only had an inkling of its breadth from a brief look at the documentation folder. Can you comment at all on "Chaos' sister project", Order? E.g., what is its purpose, current state of development, etc.? - Jeff
On Sat, 27 Nov 2010 14:22:16 -0800, Jeffrey Lee Hellrung, Jr. wrote:
Can you comment at all on "Chaos' sister project", Order? E.g., what is its purpose, current state of development, etc.?
I can't say much; it's not my project. You'd have to ask Vesa to get a better answer. I believe that it is functional, but I'm fairly certain that it is not under any active development. I think that Vesa is pretty much tired of working with the C/C++ preprocessor (and C++ in general) as opposed to languages that have "real" macro mechanisms such as Scheme. He's a functional-language type of guy. Basically, Order is an interpreter that is implemented on top of a continuation machine. -Paul
On 11/27/2010 6:36 AM, Paul Mensonides wrote: [...]
===== Lambdas/Binding =====
Chaos has a pretty heavyweight lambda engine. For example (contrived),
#include<chaos/preprocessor.h>
CHAOS_PP_EXPR(CHAOS_PP_ENUM( 5, CHAOS_PP_PRIMITIVE_CAT_(class T, CHAOS_PP_ARG(1)) CHAOS_PP_WHEN_(CHAOS_PP_ARG(1))( CHAOS_PP_CAT_(= T, CHAOS_PP_DEC_(CHAOS_PP_ARG(1))) ) ))
...results in:
class T0, class T1 = T0, class T2 = T1, class T3 = T2, class T4 = T3
However, the problem with lambdas (with the preprocessor) is that they are considerably more expensive. No matter how you do it, it requires some sort of interpretation--which is slower--and simply having a macro is usually clearer. I've actually been considering their removal from Chaos for several years.
#define _(s, n) \ class T ## n \ CHAOS_PP_WHEN(n)(= CHAOS_PP_CAT(T, CHAOS_PP_DEC(n))) \ /**/
CHAOS_PP_EXPR(CHAOS_PP_ENUM(5, _))
#undef _
I believe the now-impossible-to-find "Avalanche Preprocessor library" that that a user posted a link to some time back used a somewhat different syntax for lambdas. Recalling strictly from memory, it would look something like CHAOS_PP_EXPR( CHAOS_PP_ENUM( 5, CHAOS_PP_BIND( CHAOS_PP_PRIMITIVE_CAT_, ( class T ), 1 ) CHAOS_PP_BIND( CHAOS_PP_WHEN_, 1, CHAOS_PP_BIND( CHAOS_PP_CAT_, ( = T ), CHAOS_PP_BIND( CHAOS_PP_DEC_, 1 ) ) ) In other words, there was some very heavily-used CHAOS_PP_BIND-like macro that essentially initiated a lambda sequence, which accepted the macro to bind followed by the arguments to bind. Arguments appearing as numerals (1, 2, etc.) were placeholders, while arguments appearing in parantheses were literals. I don't remember much of the syntax, semantics, or mechanics past that. What I wrote above is probably not considered preferable to using the existing Chaos lambda syntax, but maybe some combination of these ideas could be massaged into a better syntax with faster compile times, given willing investigative souls. - Jeff
On Sun, 28 Nov 2010 21:09:34 -0800, Jeffrey Lee Hellrung, Jr. wrote:
I believe the now-impossible-to-find "Avalanche Preprocessor library" that that a user posted a link to some time back used a somewhat different syntax for lambdas. Recalling strictly from memory, it would look something like
CHAOS_PP_EXPR( CHAOS_PP_ENUM( 5, CHAOS_PP_BIND( CHAOS_PP_PRIMITIVE_CAT_, ( class T ), 1 ) CHAOS_PP_BIND( CHAOS_PP_WHEN_, 1, CHAOS_PP_BIND( CHAOS_PP_CAT_, ( = T ), CHAOS_PP_BIND( CHAOS_PP_DEC_, 1 ) ) )
In other words, there was some very heavily-used CHAOS_PP_BIND-like macro that essentially initiated a lambda sequence, which accepted the macro to bind followed by the arguments to bind. Arguments appearing as numerals (1, 2, etc.) were placeholders, while arguments appearing in parantheses were literals. I don't remember much of the syntax, semantics, or mechanics past that. What I wrote above is probably not considered preferable to using the existing Chaos lambda syntax, but maybe some combination of these ideas could be massaged into a better syntax with faster compile times, given willing investigative souls.
The lambda expressions in Chaos are made up of a relatively few primitives. All (or, at least, almost all) library primitives define lambda bindings such as the following: #define CAT(a, ...) PRIMITIVE_CAT(a, __VA_ARGS__) #define CAT_ID() CAT #define CAT_ LAMBDA(CAT_ID)() The LAMBDA macro (as well as the other lambda operators such as ARG(1), ARG(2), etc.) expand to expressions such as the following for LAMBDA (CAT_ID)() (,,,,, (etc) ,,,, 0xLAMBDA, (CAT_ID))() The parser than has a "horizontal" limit on how far over it can parse an expression (the current limit is 50, so there is about 50 commas in the above). Otherwise, the parser has no (practical) limit on depth. It parses and substitutes general expressions. The main problem is a general problem with preprocessor metaprogramming. The parser cannot "get past" some input. You cannot safely use token- pasting against arbitrary input, as that leads to undefined behavior. Even if it didn't, you still couldn't get past other stuff like operators. E.g. CHAOS_PP_EXPR(CHAOS_PP_SEQ_FOR_EACH( (CHAOS_PP_ARG(1) + CHAOS_PP_ARG(2)), (a, b)(x, y)(p, q) )) This yields garbage output because their is no way for the parser to see past the '+' opreator to find the ARG(2). -Paul
Edward Diener <eldiener <at> tropicsoft.com> writes:
On 11/24/2010 3:22 AM, Mathias Gaunard wrote:
Variadic macros would allow to support tuples better in Boost.PP, as it would not be necessary to specify the size of the tuple, and it could be possible to provide other nice features around tuples.
Other possible applications could be to accept the sequence (a)(b, c, d)(e) as being the same as (a)((b, c, d))(e).
If you think other things could be improved in PP with variadic macros, feel free to point out where.
The question is: how should it be done? Should the new macros be named, say, BOOST_PPV_*? Should the macros in BOOST_PP_* be "overloaded" on arity (a possibility with variadic macros) so as to exhibit both the old and new interface?
Please see my variadic_macro_data library in the sandbox. I have already addressed the problem of uniting variadic macros with Boost PP there as well as providing the support for tuples which you suggest in your opening paragraph above.
Thank you very much for doing this. Just this week I ran across need to use variadic macros and was hunting the various 'solutions' for something that would get my job done. Essentially I was able to take a bunch of args and prefix them, build a list, and pass that list to the target function. Very nice indeed. I know my usage must be pretty basic, but being able to do it is extremely nice. Here's what I did: http://thell.pastebin.com/zJvXhHP0 By the way, I'm using VC 9 and it worked great. Outside of learning how some of the functions of both the VMD and PP libs worked out their expansions, this was as easy as it could get. -- Thell
On 11/25/2010 10:29 PM, Thell Fowler wrote:
Edward Diener<eldiener<at> tropicsoft.com> writes:
On 11/24/2010 3:22 AM, Mathias Gaunard wrote:
Variadic macros would allow to support tuples better in Boost.PP, as it would not be necessary to specify the size of the tuple, and it could be possible to provide other nice features around tuples.
Other possible applications could be to accept the sequence (a)(b, c, d)(e) as being the same as (a)((b, c, d))(e).
If you think other things could be improved in PP with variadic macros, feel free to point out where.
The question is: how should it be done? Should the new macros be named, say, BOOST_PPV_*? Should the macros in BOOST_PP_* be "overloaded" on arity (a possibility with variadic macros) so as to exhibit both the old and new interface?
Please see my variadic_macro_data library in the sandbox. I have already addressed the problem of uniting variadic macros with Boost PP there as well as providing the support for tuples which you suggest in your opening paragraph above.
Thank you very much for doing this. Just this week I ran across need to use variadic macros and was hunting the various 'solutions' for something that would get my job done. Essentially I was able to take a bunch of args and prefix them, build a list, and pass that list to the target function. Very nice indeed.
I know my usage must be pretty basic, but being able to do it is extremely nice. Here's what I did:
http://thell.pastebin.com/zJvXhHP0
By the way, I'm using VC 9 and it worked great. Outside of learning how some of the functions of both the VMD and PP libs worked out their expansions, this was as easy as it could get.
I am glad to hear that my library has practical uses. I am sure as others also use variadic macros for their syntactical ease of use, being able to use my library, as a way of using variadic macros with the great flexibility of Boost PP, will be valuable to them also.
On Wed, Nov 24, 2010 at 3:22 AM, Mathias Gaunard < mathias.gaunard@ens-lyon.org> wrote:
Other possible applications could be to accept the sequence (a)(b, c, d)(e) as being the same as (a)((b, c, d))(e).
I've recently done exactly this in my Generic and AutoFunction libraries, only instead of wrapping the elements with commas in extra parentheses, it wraps them all in extra parentheses. The macro takes a "variadic" sequence and yields the corresponding Boost.Preprocessor sequence. I'll link to the implementation on my next sandbox commit (which should be tonight). -- -Matt Calabrese
On 11/24/2010 2:58 PM, Matt Calabrese wrote:
On Wed, Nov 24, 2010 at 3:22 AM, Mathias Gaunard< mathias.gaunard@ens-lyon.org> wrote:
Other possible applications could be to accept the sequence (a)(b, c, d)(e) as being the same as (a)((b, c, d))(e).
I've recently done exactly this in my Generic and AutoFunction libraries, only instead of wrapping the elements with commas in extra parentheses, it wraps them all in extra parentheses. The macro takes a "variadic" sequence and yields the corresponding Boost.Preprocessor sequence. I'll link to the implementation on my next sandbox commit (which should be tonight).
One problem with variadic macro usage is that in a variadic macro itself you can only use variadic data once as the last parameter the macro takes. Of course you can use the variadic data in the expansion to the macro as you please, and my library easily lets you grab any token(s) of that variadic data and use it in the expansion. Please take a look at what my library has to offer and hopefully you can use some of the functionality in the Generic and Autofunction libraries to simplify syntax if you find it worthwhile. If you have any suggestions about my library I will be glad to hear them. Thanks for taking an interest in the library.
participants (10)
-
Edward Diener -
Hal Finkel -
Hartmut Kaiser -
Jeffrey Lee Hellrung, Jr. -
Mathias Gaunard -
Matt Calabrese -
Paul Mensonides -
Thell Fowler -
Vicente Botet -
Wolf Lammen