Re: [boost] Release Tools Analysis Part I: Build systems
on Sun Aug 05 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
FWIW, I submitted a patch to SCons providing a python extension module around ucpp (http://pornin.nerim.net/ucpp/) to provide a 'more accurate' C/C++ dependency scanner, long before wave was around.
Does anyone know if ucpp is actually conforming? Until very recently, most preprocessors were badly broken (and maybe most still are). If it conforms, it would be interesting to benchmark it against Wave.
Steven refused that, believing that it could (and should) all be done in pure python.
Well, with a great deal of due respect to Steven, he's wrong. Sure it /can/ be done in pure python, but /should/ it be? Before we had insane C++ template metaprograms, the cost of preprocessing and lexing typically swamped that of any other C/C++ compilation phase, just because they had to deal with more data (individual characters). And now we have insane preprocessor metaprograms, so things have not gotten any easier on the C/C++ preprocessor.
That was many years ago, and the situation hasn't substantially changed since then.
Sounds familiar. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com The Astoria Seminar ==> http://www.astoriaseminar.com
David Abrahams wrote:
on Sun Aug 05 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
FWIW, I submitted a patch to SCons providing a python extension module around ucpp (http://pornin.nerim.net/ucpp/) to provide a 'more accurate' C/C++ dependency scanner, long before wave was around.
Does anyone know if ucpp is actually conforming? Until very recently, most preprocessors were badly broken (and maybe most still are). If it conforms, it would be interesting to benchmark it against Wave.
Actually, I had an argument with the ucpp author some years ago because ucpp stopped to parse boost code properly, which the ucpp author claimed was due to some different interpretation of the standard. That was when I decided to switch from ucpp to boost.wave in Synopsis...
Steven refused that, believing that it could (and should) all be done in pure python.
Well, with a great deal of due respect to Steven, he's wrong. Sure it /can/ be done in pure python, but /should/ it be? Before we had insane C++ template metaprograms, the cost of preprocessing and lexing typically swamped that of any other C/C++ compilation phase, just because they had to deal with more data (individual characters). And now we have insane preprocessor metaprograms, so things have not gotten any easier on the C/C++ preprocessor.
I agree with you. I think the technical question about the merits of pure-python programs aside, this is another case of the NIH syndrome.
That was many years ago, and the situation hasn't substantially changed since then.
Sounds familiar.
Indeed. :-( Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin...
David Abrahams wrote:
on Sun Aug 05 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
FWIW, I submitted a patch to SCons providing a python extension module around ucpp (http://pornin.nerim.net/ucpp/) to provide a 'more accurate' C/C++ dependency scanner, long before wave was around.
Does anyone know if ucpp is actually conforming? Until very recently, most preprocessors were badly broken (and maybe most still are). If it conforms, it would be interesting to benchmark it against Wave.
Wave probably will be slower than ucpp. Several reasons: - ucpp is 'cheating' and doesn't do correct preprocessing, which saves it some cycles - Wave never has been written with maximum performance in mind, even if a lot of effort has been invested recently to make it performant (if compared to gcc's preprocessor Wave is on average still 3 times slower) - Wave does more than pure preprocessing: it does C++ lexing, which adds some overhead OTOH, a real usecase in the Boost build system is a fine motivation and benchmark allowing to optimize Wave further. Regards Hartmut
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of David Abrahams Sent: Sunday, August 05, 2007 6:13 PM Subject: Re: [boost] Release Tools Analysis Part I: Build systems
Well, with a great deal of due respect to Steven, he's wrong. Sure it /can/ be done in pure python, but /should/ it be? Before we had insane C++ template metaprograms, the cost of preprocessing and lexing typically swamped that of any other C/C++ compilation phase, just because they had to deal with more data (individual characters). And now we have insane preprocessor metaprograms, so things have not gotten any easier on the C/C++ preprocessor.
I don't think, the maintainers position does matter here. Scons has an interface (a so called 'scanner' interface) to add tools of your choice for generating the implicit dependenies. cheers, aa -- Andreas Ames | Programmer | Comergo GmbH | ames AT avaya DOT com Sitz der Gesellschaft: Stuttgart Registergericht: Amtsgericht Stuttgart - HRB 22107 Geschäftsführer: Andreas von Meyer zu Knonow, Udo Bühler, Thomas Kreikemeier
Ames, Andreas (Andreas) wrote:
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of David Abrahams Sent: Sunday, August 05, 2007 6:13 PM Subject: Re: [boost] Release Tools Analysis Part I: Build systems
Well, with a great deal of due respect to Steven, he's wrong. Sure it /can/ be done in pure python, but /should/ it be? Before we had insane C++ template metaprograms, the cost of preprocessing and lexing typically swamped that of any other C/C++ compilation phase, just because they had to deal with more data (individual characters). And now we have insane preprocessor metaprograms, so things have not gotten any easier on the C/C++ preprocessor.
I don't think, the maintainers position does matter here. Scons has an interface (a so called 'scanner' interface) to add tools of your choice for generating the implicit dependenies.
That's not quite correct. That interface you are talking about isn't quite enough to let you plug in external scanners, since a real preprocessor such as boost.wave would need to interact with other tools to fetch preprocessor flags such as include search paths, predefined macros, and the like, without which it would fall over rather quickly. FWIW, Stefan -- ...ich hab' noch einen Koffer in Berlin...
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Stefan Seefeld Sent: Monday, August 06, 2007 12:33 PM Subject: Re: [boost] Release Tools Analysis Part I: Build systems
That's not quite correct. That interface you are talking about isn't quite enough to let you plug in external scanners, since a real preprocessor such as boost.wave would need to interact with other tools to fetch preprocessor flags such as include search paths, predefined macros, and the like, without which it would fall over rather quickly.
I have done it for IDL-preprocessing. You always have the build environment at your disposal to communicate the items you have mentioned. I don't claim it is perfect, but it is sth. to build upon, IMHO. cheers, aa -- Andreas Ames | Programmer | Comergo GmbH | ames AT avaya DOT com Sitz der Gesellschaft: Stuttgart Registergericht: Amtsgericht Stuttgart - HRB 22107 Geschäftsführer: Andreas von Meyer zu Knonow, Udo Bühler, Thomas Kreikemeier
Ames, Andreas (Andreas) wrote:
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Stefan Seefeld Sent: Monday, August 06, 2007 12:33 PM Subject: Re: [boost] Release Tools Analysis Part I: Build systems
That's not quite correct. That interface you are talking about isn't quite enough to let you plug in external scanners, since a real preprocessor such as boost.wave would need to interact with other tools to fetch preprocessor flags such as include search paths, predefined macros, and the like, without which it would fall over rather quickly.
I have done it for IDL-preprocessing. You always have the build environment at your disposal to communicate the items you have mentioned. I don't claim it is perfect, but it is sth. to build upon, IMHO.
For most C/C++ code you need system search paths, which depend on the compiler actually used. Finding user-provided headers isn't enough. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin...
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Stefan Seefeld Sent: Monday, August 06, 2007 12:48 PM Subject: Re: [boost] Release Tools Analysis Part I: Build systems
For most C/C++ code you need system search paths, which depend on the compiler actually used. Finding user-provided headers isn't enough.
For actual preprocessing, that's true. I'm not sure, you really want system headers in the list of implicit dependencies (and that means, I'm really not sure). Anyway, the paths to the system headers are highly platform- and tool- dependent (as you have mentioned) and should be communicated through the build environment, IMHO. cheers, aa -- Andreas Ames | Programmer | Comergo GmbH | ames AT avaya DOT com Sitz der Gesellschaft: Stuttgart Registergericht: Amtsgericht Stuttgart - HRB 22107 Geschäftsführer: Andreas von Meyer zu Knonow, Udo Bühler, Thomas Kreikemeier
Ames, Andreas (Andreas) wrote:
For most C/C++ code you need system search paths, which depend on the compiler actually used. Finding user-provided headers isn't enough.
For actual preprocessing, that's true. I'm not sure, you really want system headers in the list of implicit dependencies (and that means, I'm really not sure).
But there is no way of doing real dependency scanning without real preprocessing. That's the point. Whether you throw away the dependencies you consider stable is another matter.
Anyway, the paths to the system headers are highly platform- and tool- dependent (as you have mentioned) and should be communicated through the build environment, IMHO.
That's my point: SCons doesn't have any means to communicate these to the dependency scanner, so you can't easily plug in your own. (To get this back on-topic, I wonder whether boost.build has any means to communicate such variables to its own dependency scanner, i.e. how complicated it would be to hook a boost.wave-based preprocessor up with it.) Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin...
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Stefan Seefeld Sent: Monday, August 06, 2007 1:30 PM Subject: Re: [boost] Release Tools Analysis Part I: Build systems
That's my point: SCons doesn't have any means to communicate these to the dependency scanner, so you can't easily plug in your own.
Sorry for me being dense, I just don't get your point. The interface that scons uses to call the scannner looks like this: def __call__(self, node, env, path = ()): ... What I want to say is: Stuff the sys-include paths etc. into the build environment, that is communicated to the scanner through the 'env' parameter in the (member) function above. Then you can configure your own scanner however you want. For example, the msvc tool communicates those settings through a dict keyed 'ENV' within the build environment. It contains include and library paths, among other things. This is way off topic now, so I'll shut up. cheers, aa -- Andreas Ames | Programmer | Comergo GmbH | ames AT avaya DOT com Sitz der Gesellschaft: Stuttgart Registergericht: Amtsgericht Stuttgart - HRB 22107 Geschäftsführer: Andreas von Meyer zu Knonow, Udo Bühler, Thomas Kreikemeier
participants (4)
-
Ames, Andreas (Andreas) -
David Abrahams -
Hartmut Kaiser -
Stefan Seefeld