[config][preprocessor] proposal for GCC-XML support
Hello, This message proposes adding GCC-XML support to the boost libraries to enable wrapping of more code with pyste. Most of the changes will probably be in the config and preprocessor libraries, but there may be others. Here is an explanation of the motivation and what needs to be done: Boost's Pyste tool http://www.boost.org/libs/python/pyste uses my tool GCC-XML http://www.gccxml.org to parse C++ code and then generates Boost.Python wrappers. Users have experienced problems getting some boost code to be parsed by GCC-XML. See the problem description http://www.gccxml.org/pipermail/gccxml/2006-January/000696.html and my explanation of what GCC-XML is doing http://www.gccxml.org/pipermail/gccxml/2006-January/000697.html for details. GCC-XML has two basic parts. One is a patched GCC compiler that has extra code to dump out an XML description of the interface found in a translation unit. The other is a front-end executable and some support code I wrote to enable GCC-XML to parse source code in a way that is compatible with wrapper generators. Wrapping arbitrary C++ code requires the wrapper generator to know about the standard library and implementation-defined types and interfaces provided by the compiler that will be used to build the generated code. In order to provide this information from the source GCC-XML simulates the preprocessing environment of a given compiler. This means that even though the parser is GCC the preprocessor definitions look like another compiler. For example, if msvc71 is specified as the compiler to simulate then the include path will be setup to use the system headers provided by Visual Studio .NET 2003. Preprocessor definitions such as _MSC_VER will be set they way they are when the native preprocessor runs. Unless a GCC compiler is being simulated, __GNUC__ is NOT defined. The only way for code to know that it is being preprocessed by GCC-XML is to look for the definition __GCCXML__. This macro is defined to be the GCC-XML version number (major*10000+minor*100+patchlevel). In a perfect world all compilers would be standard conforming and would all parse valid C++ code. In the real world there are inconsistencies and boost has many places that use compiler-specific parser work-arounds. Since GCC-XML uses a GCC parser but appears to the preprocessor as the native compiler special knowledge is needed when choosing implementation alternatives for compiler workarounds. GCC-XML is effectively another compiler for which knowledge should be encoded into the boost libraries. When __GCCXML__ is defined any *parser* work-around should use the GCC alternative, while any *interface* work-around should use the native compiler's alternative. For example, if a particular boost feature is not available on MSVC 6 then it should not be available if _MSC_VER=1200 even if __GCCXML__ is defined. On the other hand if the feature is always available but its implementation switches among different syntax choices to support multiple compilers then the GCC choice should be used if __GCCXML__ is defined. Therefore only parser work-arounds need to be updated with knowledge of GCC-XML. If choice of a specific GCC parser version is needed each GCC-XML version always uses a single GCC parser version (currently 3.3) so there is a simple mapping from the __GCCXML__ value to the GCC parser version number. The following are points for discussion: 1.) Is there interest in adding this support to enable both wrapping of more boost code and of more user code that uses boost with pyste? The motivating case mentioned above is trying to wrap serialization code which in turn uses mpl. 2.) What is the best way to encode knowledge of GCC-XML in boost libraries? I suspect most of the knowledge can be put in the config library when it defines macros encoding the availability of parser features. 3.) Does anyone volunteer to do the work? It can probably be done incrementally as users request support for wrapping of more boost code. I'm willing to help but I cannot spend very much time on this because it is really a boost/pyste problem and not a GCC-XML problem. -Brad
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Brad King
This message proposes adding GCC-XML support to the boost libraries to enable wrapping of more code with pyste. Most of the changes will probably be in the config and preprocessor libraries, but there may be others. Here is an explanation of the motivation and what needs to be done:
How does this effect the preprocessor library--aside from avoiding workarounds that GCC will report as ill-formed? Changing that is simple enough for the preprocessor library. Already done--the configuration recognizes __GCCXML__ and defines BOOST_PP_CONFIG_FLAGS accordingly. Regards, Paul Mensonides
Paul Mensonides wrote:
This message proposes adding GCC-XML support to the boost libraries to enable wrapping of more code with pyste. Most of the changes will probably be in the config and preprocessor libraries, but there may be others. Here is an explanation of the motivation and what needs to be done:
How does this effect the preprocessor library--aside from avoiding workarounds that GCC will report as ill-formed?
I don't think it does, but Dave Abrahams asked me to get your attention by mentioning the preprocessor library: http://www.gccxml.org/pipermail/gccxml/2006-January/000698.html I think my post on the gccxml list used slightly confusing wording and Dave thought this required only preprocessor parsing work-arounds. However, I wasn't sure if he was implying that you were the right person to make all the changes. Sorry for the confusion.
Changing that is simple enough for the preprocessor library. Already done--the configuration recognizes __GCCXML__ and defines BOOST_PP_CONFIG_FLAGS accordingly.
Great, thanks. -Brad
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Brad King
How does this effect the preprocessor library--aside from avoiding workarounds that GCC will report as ill-formed?
I don't think it does, but Dave Abrahams asked me to get your attention by mentioning the preprocessor library:
http://www.gccxml.org/pipermail/gccxml/2006-January/000698.html
I think my post on the gccxml list used slightly confusing wording and Dave thought this required only preprocessor parsing work-arounds. However, I wasn't sure if he was implying that you were the right person to make all the changes. Sorry for the confusion.
Well, the pp-lib presents the same interface on all supported preprocessors. If something cannot be ported (via workarounds) to a particular preprocessor, the mechanism simply isn't exposed as an interface, though the pp-lib implementation itself may use it internally. There are a few internal interfaces that are really useful that aren't presented as external interfaces. Some of those are used by users anyway. That's fine; I've recommended it many times along with the portability warning. However, there is nothing in the pp-lib that gcc's preprocessor can't handle--it is a very good preprocessor--with the exception of workarounds present for broken preprocessors. Some of those workarounds involve code that is either illegal or utilitizes undefined behavior. Explicitly catching __GCCXML__ should prevent the library from using any of these workarounds in the pp-lib itself. However, I obviously can't guarantee anything about what clients of the pp-lib might be doing for portability.
Changing that is simple enough for the preprocessor library. Already done--the configuration recognizes __GCCXML__ and defines BOOST_PP_CONFIG_FLAGS accordingly.
Great, thanks.
No problem. Regards, Paul Mensonides
2.) What is the best way to encode knowledge of GCC-XML in boost libraries? I suspect most of the knowledge can be put in the config library when it defines macros encoding the availability of parser features.
I've added tentative support to Boost.Config, exactly how far that gets us remains to be seen :-) John.
John Maddock wrote:
2.) What is the best way to encode knowledge of GCC-XML in boost libraries? I suspect most of the knowledge can be put in the config library when it defines macros encoding the availability of parser features.
I've added tentative support to Boost.Config, exactly how far that gets us remains to be seen :-)
Thanks, I'll give it a try when I get a chance. There is one problem I see in gcc_xml.hpp: # if __GCCXML__ < 30400 # define BOOST_NO_IS_ABSTRACT # endif GCC-XML defines __GCCXML__ with its *own* version number, not that of the internal GCC parser. There is a simple mapping from GCC-XML version number to the GCC parser version number (currently always 3.3). However I can see that it would be a pain for every project to know about this mapping. I should add more definitions like __GCCXML_GNUC__ __GCCXML_GNUC_MINOR__ __GCCXML_GNUC_PATCHLEVEL__ so that the true GCC parser version is known. This also maintains the option to replace the parser with another compiler in the future. Do these names look okay to you John? Thanks, -Brad
GCC-XML defines __GCCXML__ with its *own* version number, not that of the internal GCC parser.
Oh, shucks :-(
There is a simple mapping from GCC-XML version number to the GCC parser version number (currently always 3.3). However I can see that it would be a pain for every project to know about this mapping. I should add more definitions like
__GCCXML_GNUC__ __GCCXML_GNUC_MINOR__ __GCCXML_GNUC_PATCHLEVEL__
so that the true GCC parser version is known. This also maintains the option to replace the parser with another compiler in the future. Do these names look okay to you John?
Sure, it's your tool you can call them anything you want :-) John.
John Maddock wrote:
Brad King wrote:
I should add more definitions like __GCCXML_GNUC__ __GCCXML_GNUC_MINOR__ __GCCXML_GNUC_PATCHLEVEL__ so that the true GCC parser version is known. Sure, it's your tool you can call them anything you want :-)
Okay, I've added these definitions to GCC-XML. It will be up to pyste to define them properly when invoking an older GCC-XML version that does not have this feature. Below is a clip from the GCC-XML man page after these changes that describes the definitions provided beyond the simulated compiler. Please update gcc_xml.hpp to use these definitions to switch among GCC parser versions. Thanks, -Brad The following extra C preprocessor definitions are provided: -D__GCCXML__=MMmmpp MM, mm, and pp are the major, minor, and patch versions of GCC-XML. This preprocessor symbol identifies GCC-XML to the source code as it is preprocessed. It can be used to enable GCC-XML-specific information. -D__GCCXML_GNUC__=3 Defined to internal GCC parser major version. -D__GCCXML_GNUC_MINOR__=3 Defined to internal GCC parser minor version. -D__GCCXML_GNUC_PATCHLEVEL__=2 Defined to internal GCC parser patchlevel.
participants (3)
-
Brad King -
John Maddock -
Paul Mensonides