Re: [boost] Proposal for #pragma once support
On Tue, Jun 9, 2009 at 11:50 AM, <dherring@ll.mit.edu> wrote:
On Tue, 9 Jun 2009, Zachary Turner wrote:
Here are the build timings on my machine (Visual Studio 2008 Service Pack
2)
RedundantGuard - 1 second compile + link time PragmaOnce - 1 second compile + link time Nothing - 15 seconds compile + link time
...
So, I believe that boost compile times can benefit significantly from this.
Thoughts?
Could you run benchmarks on gcc? Will my compile times suffer for your improvement?
- Daniel
It seems someone else in the thread has already run benchmarks on GCC, but regardless of how the benchmarks did or did not turn out, I can say with 100% certainty that nobody will suffer. That's the exact reason I mentioned that compilers should be whitelisted based on which ones benefit from #pragma once, and only use #pragma once for said compilers. #if defined(_MSC_VER) && (_MSC_VER >= 1020) #pragma once #endif That's all it takes. Everybody benefits. Of course I don't have licenses for every C++ compiler around, but I'm sure there are other compilres in which it benefits also. So I would need some assistance from other people who do have other compilers to help find out which compilers it does offer an improvement on. And of course, the default would be to NOT use pragma once, erring on the side of caution. If you see any downsides to this approach let me know, but it seems like a win/win for everyone. To make it even more explicit, there could even be a global macro, which defaults to undefined, called BOOST_USE_PRAGMA_ONCE. The programmer would be forced to explicitly declare this macro before including any boost headers, and if it is enabled, boost would use pragma once.
Zachary Turner wrote:
It seems someone else in the thread has already run benchmarks on GCC, but regardless of how the benchmarks did or did not turn out, I can say with 100% certainty that nobody will suffer. That's the exact reason I mentioned that compilers should be whitelisted based on which ones benefit from #pragma once, and only use #pragma once for said compilers.
#if defined(_MSC_VER) && (_MSC_VER >= 1020) #pragma once #endif
That's all it takes. Everybody benefits. Of course I don't have licenses for every C++ compiler around, but I'm sure there are other compilres in which it benefits also. So I would need some assistance from other people who do have other compilers to help find out which compilers it does offer an improvement on.
I think one sure-fire way to get most people to at least try it would be to write a script that adds this to each header in their boost installation. I'd be willing to try it (make sure it runs on Linux!) -- Sohail Somani http://uint32t.blogspot.com
On Tue, Jun 9, 2009 at 11:19 AM, Zachary Turner <divisortheory@gmail.com> wrote:
#if defined(_MSC_VER) && (_MSC_VER >= 1020) #pragma once #endif
That's all it takes. Everybody benefits.
I'm sorry but what problem are we trying to solve with this? Does compiling Boost take too much time? If the answer is yes, is #pragrma once the best solution we can come up with? :) Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
On Tue, Jun 9, 2009 at 2:00 PM, Emil Dotchevski <emildotchevski@gmail.com>wrote:
On Tue, Jun 9, 2009 at 11:19 AM, Zachary Turner <divisortheory@gmail.com> wrote:
#if defined(_MSC_VER) && (_MSC_VER >= 1020) #pragma once #endif
That's all it takes. Everybody benefits.
I'm sorry but what problem are we trying to solve with this? Does compiling Boost take too much time? If the answer is yes, is #pragrma once the best solution we can come up with? :)
It's not necessarily the "best" solution one can come up with, it's just one of many solutions that all address the problem of compilation time from a different angle. If we can come up with other solutions that's obviously great, but I don't see any reason to ignore one solution that can then be stacked with other solutions for even greater benefits. No matter how many other solutions are implemented to reduce compilation time, #pragma once will _always_ expand on that by reducing compilation times even further for certain compilers. That being said, compilation of boost itself isn't even the real use case, since a user of boost typically just compiles it once and is done. The real use case is for people using boost in their own projects, i.e. including lots of boost header files at various locations in their program. In this case, it has potential to make a huge impact, since developers frequently build code tens of times per day. Even the slightest change to a header file and a rebuild of the project to test the change has the potential to trigger a rebuild of numerous other files, so there's no reason not to take every measure possible to reduce compilation time.
On 9 Jun 2009, at 20:08, Zachary Turner wrote:
That being said, compilation of boost itself isn't even the real use case, since a user of boost typically just compiles it once and is done. The real use case is for people using boost in their own projects, i.e. including lots of boost header files at various locations in their program. In this case, it has potential to make a huge impact, since developers frequently build code tens of times per day. Even the slightest change to a header file and a rebuild of the project to test the change has the potential to trigger a rebuild of numerous other files, so there's no reason not to take every measure possible to reduce compilation time.
While it would be a bit of work, I'd be very interested in seeing convincing numbers to this effect: If you add #pragma once (with suitable guards) to boost (or a substantial section of it), then prove that the resulting code produces speed improvements, then you would convince people! One thing to watch out for, in case you don't notice in advance: Some headers should NOT have #pragma once. Not very many, but such files exist. You'll probably want to try to find some way of guessing if a file already has a standard include guard (shouldn't be too hard to detect), and only #pragma once such files.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
2009/6/9 Christopher Jefferson <chris@bubblescope.net>:
One thing to watch out for, in case you don't notice in advance: Some headers should NOT have #pragma once. Not very many, but such files exist. You'll probably want to try to find some way of guessing if a file already has a standard include guard (shouldn't be too hard to detect), and only #pragma once such files.
Which is exactly what GCC does to not need #pragma once =) (And that check can be made fast by allowing the pathological cases to fall back to the normal include guard behaviour, as that's still safe, whereas #pragma once always has to go the slow route of canonizing all the paths -- which isn't even always possible -- to be safe.) ~ Scott
On Tue, Jun 9, 2009 at 7:58 PM, Scott McMurray<me22.ca+boost@gmail.com> wrote:
2009/6/9 Christopher Jefferson <chris@bubblescope.net>:
One thing to watch out for, in case you don't notice in advance: Some headers should NOT have #pragma once. Not very many, but such files exist. You'll probably want to try to find some way of guessing if a file already has a standard include guard (shouldn't be too hard to detect), and only #pragma once such files.
Which is exactly what GCC does to not need #pragma once =)
(And that check can be made fast by allowing the pathological cases to fall back to the normal include guard behaviour, as that's still safe, whereas #pragma once always has to go the slow route of canonizing all the paths -- which isn't even always possible -- to be safe.)
You're making the same point I was thinking about earlier, which is that in theory #pragma once shouldn't be faster than include guards. So how about this: #include <boost/detail/workaround.hpp> #if defined(BOOST_MSVC) && BOOST_WORKAROUND(BOOST_MSVC, < 1500) #pragma once #endif Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
2009/6/9 Emil Dotchevski <emildotchevski@gmail.com>:
You're making the same point I was thinking about earlier, which is that in theory #pragma once shouldn't be faster than include guards. So how about this:
#include <boost/detail/workaround.hpp> #if defined(BOOST_MSVC) && BOOST_WORKAROUND(BOOST_MSVC, < 1500) #pragma once #endif
Other than the fact that this should just be solved for once and for all in MSVC, rather than forcing us to pollute our code, I don't see a problem with just using this: #ifdef BOOST_ENABLE_PRAGMA_ONCE #pragma once #endif That way anyone who wants it can have it, and it's obvious by looking at the headers how to get it, but we're not encouraging its use.
On Tue, Jun 9, 2009 at 8:33 PM, Scott McMurray<me22.ca+boost@gmail.com> wrote:
2009/6/9 Emil Dotchevski <emildotchevski@gmail.com>:
You're making the same point I was thinking about earlier, which is that in theory #pragma once shouldn't be faster than include guards. So how about this:
#include <boost/detail/workaround.hpp> #if defined(BOOST_MSVC) && BOOST_WORKAROUND(BOOST_MSVC, < 1500) #pragma once #endif
Other than the fact that this should just be solved for once and for all in MSVC, rather than forcing us to pollute our code, I don't see a problem with just using this:
#ifdef BOOST_ENABLE_PRAGMA_ONCE #pragma once #endif
I'm not sure you got my drift, using BOOST_WORKAROUND effectively flags the better #pragma once performance on MSVC as a bug. :) Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
I'm not sure you got my drift, using BOOST_WORKAROUND effectively flags the better #pragma once performance on MSVC as a bug. :)
Sure, but it requires #including boost/workaround.hpp in every header, which is surely not such a great idea? In fact wouldn't the extra #includes outweigh the use of pragma once? Speculatively yours, John.
On Thu, Jun 11, 2009 at 1:34 AM, John Maddock<john@johnmaddock.co.uk> wrote:
I'm not sure you got my drift, using BOOST_WORKAROUND effectively flags the better #pragma once performance on MSVC as a bug. :)
Sure, but it requires #including boost/workaround.hpp in every header, which is surely not such a great idea?
In fact wouldn't the extra #includes outweigh the use of pragma once?
Yeah, I wast mostly joking :) but then again, in my mind improving build times by using pragma once is a joke too. If we want to deal with the problem of slow builds we should be removing stuff from headers, instead of making it easier to swallow all that crap. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
On Thu, Jun 11, 2009 at 12:51 PM, Emil Dotchevski <emildotchevski@gmail.com>wrote:
On Thu, Jun 11, 2009 at 1:34 AM, John Maddock<john@johnmaddock.co.uk> wrote:
I'm not sure you got my drift, using BOOST_WORKAROUND effectively flags the better #pragma once performance on MSVC as a bug. :)
Sure, but it requires #including boost/workaround.hpp in every header, which is surely not such a great idea?
In fact wouldn't the extra #includes outweigh the use of pragma once?
Yeah, I wast mostly joking :) but then again, in my mind improving build times by using pragma once is a joke too. If we want to deal with the problem of slow builds we should be removing stuff from headers, instead of making it easier to swallow all that crap.
Even if we do remove a bunch of stuff from headers, it can still be made faster. Now matter how many ways you approach the problem, choosing to ignore one specific way that actually makes a serious difference simply because it only works on a subset of compilers is fundamentally inconsistent with the notion of developing a cross platform C++ library that works on every compiler. In an ideal world(?), every compiler for every platform would be 100% standards compliance and provide 0 compiler specific extensions. Then the issue wouldn't come up at all. But a fact of life when developing for multiple compilers is that you have to deal with compiler specific quirks. I disagree with the notion that MSVC is "fundamentally a joke" (paraphrased) as others have put it simply because it provides a compiler extension via the *C++ standard* way of providing compiler extensions. GCC, while a great compiler, has its quirks too. There are compiler specific newsgroups to nitpick about what you love / hate about specific compilers, but in the meantime the purpose here is to solve real world problems people face when using boost with whatever compiler it is they're using. It's easy to downplay the importance of fast build times. But it's not so easy anymore once you've worked on a product for which it is actually impossible to do nightly builds because it takes longer than a night to complete a build.
On Thu, Jun 11, 2009 at 11:15 AM, Zachary Turner<divisortheory@gmail.com> wrote:
On Thu, Jun 11, 2009 at 12:51 PM, Emil Dotchevski <emildotchevski@gmail.com>wrote:
On Thu, Jun 11, 2009 at 1:34 AM, John Maddock<john@johnmaddock.co.uk> wrote:
I'm not sure you got my drift, using BOOST_WORKAROUND effectively flags the better #pragma once performance on MSVC as a bug. :)
Sure, but it requires #including boost/workaround.hpp in every header, which is surely not such a great idea?
In fact wouldn't the extra #includes outweigh the use of pragma once?
Yeah, I wast mostly joking :) but then again, in my mind improving build times by using pragma once is a joke too. If we want to deal with the problem of slow builds we should be removing stuff from headers, instead of making it easier to swallow all that crap.
Even if we do remove a bunch of stuff from headers, it can still be made faster. Now matter how many ways you approach the problem, choosing to ignore one specific way that actually makes a serious difference simply because it only works on a subset of compilers is fundamentally inconsistent with the notion of developing a cross platform C++ library that works on every compiler.
In a company I worked at, it took ~40 minutes to do a full rebuild of the project I worked on. Their solution was to install IncrediBuild, and overnight build times went down to ~7 minutes total. That's fast right? Worth doing? More than #pragma once? Except that two years later the build times were up to over 30 minutes again _with_ IncrediBuild. :) Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
On Thu, Jun 11, 2009 at 1:34 PM, Emil Dotchevski <emildotchevski@gmail.com>wrote:
On Thu, Jun 11, 2009 at 11:15 AM, Zachary Turner<divisortheory@gmail.com> wrote:
Even if we do remove a bunch of stuff from headers, it can still be made faster. Now matter how many ways you approach the problem, choosing to ignore one specific way that actually makes a serious difference simply because it only works on a subset of compilers is fundamentally inconsistent with the notion of developing a cross platform C++ library that works on every compiler.
In a company I worked at, it took ~40 minutes to do a full rebuild of the project I worked on. Their solution was to install IncrediBuild, and overnight build times went down to ~7 minutes total. That's fast right? Worth doing? More than #pragma once? Except that two years later the build times were up to over 30 minutes again _with_ IncrediBuild. :)
I guess I just don't see where the "more than" comes from. If something is worth doing it's worth doing, independent of what else may or may not be worth doing. Whether something gives a "bigger" performance improvement is beside the point. For one thing, removing lots of stuff from existing headers as you suggest is a much larger undertaking than simply adding a few lines of code at the top of (almost) every header file. Does the cost/benefit ratio make it worth doing? More than #pragma once? ;-)
#pragma once is useful in Windows for the same reason they have the pre-compiled header abomination: slow directory-level file system operations and a really bad system header legacy. On the point that some headers should not use this mechanism: in addition to headers designed to instantiate something in-place, some headers include optional components activated by a user-specified define. The idempotent and flag-independent portions can be enclosed in a preprocessor include guard but the optional part should be left outside the guard. An example is the non-ANSI manifest constant set in <math.h> which Microsoft chose to enable by defining a symbol before including the header. By leaving that conditional block of definitions out of the include guard and *not* specifying pragma once for that file they ensured that #include <something.innocuous.that.includes.math.h> #define USE_MATH_DEFINES #include <math.h> works as expected. Another example: #define BOOST_TEST_MAIN #include "boost/test/unit_test.hpp" This causes a boilerplate "main" to be instantiated in your source file. If this were done outside the include guard then we could make unit_test.hpp (without main) part of the pre-compiled header and then re-include it in one compilation unit with the macro defined. #pragma once would make that impossible. By the way, the Microsoft include "optimization" story gets even goofier. They also have a magic cookie comment that you can include as the first line of a header that causes dependency analysis to ignore the file that contains the cookie and any subordinate dependencies. This was hacked into their C/C++ compiler to make incremental builds tolerable in the presence of the IDE-maintained headers defining resource IDs and such. Needless to say it causes all kinds of problems. One of which is that other consumers of those files do not honor the cookie in their dependency analysis. There is also a "minimal rebuild" mechanism mentioned in the compiler switch documentation that seems to have been intended to do a syntax-aware intra-compilation-unit dependency analysis. I have no idea what it really does but its existence is just one more clue as to how bad the performance of this compiler/file system/header suite combination is. -swn
Stephen Nuchia wrote On Wednesday, June 10, 2009 10:46 AM
#pragma once is useful in Windows for the same reason they have the pre-compiled header abomination: slow directory-level file system operations and a really bad system header legacy.
On the point that some headers should not use this mechanism: in addition to headers designed to instantiate something in- place, some headers include optional components activated by a user-specified define. The idempotent and flag-independent portions can be enclosed in a preprocessor include guard but the optional part should be left outside the guard.
[snip examples of optional code outside guard] [snip examples of Microsoft magic speedup mechanisms] These all just reinforce the idea that an automated solution is only a first pass. Building and testing must occur to find the headers that should not use #pragma once before the changes are committed, assuming this effort comes to fruition. Since making the changes, building successfully, and testing successfully, must occur in order to provide meaningful test results, special cases like those you've noted will have been addressed. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
Emil Dotchevski schrieb am Dienstag 09 Juni 2009 um 21:00:
On Tue, Jun 9, 2009 at 11:19 AM, Zachary Turner <divisortheory@gmail.com> wrote:
#if defined(_MSC_VER) && (_MSC_VER >= 1020) #pragma once #endif
That's all it takes. Everybody benefits.
I'm sorry but what problem are we trying to solve with this? Does compiling Boost take too much time? If the answer is yes, is #pragrma once the best solution we can come up with? :)
While having no experience with "#pragma once", my impression is that it has an positive effect if the filesystem is slow when touching many small files, which I painfully experience on MS.Windows (at least on XP, dunno if Vista and Windows7 suffer from this as well). The point of Zachary's approach is being a non intrusive no brainer solution. All the user has to do is to add a definition to the command line via his/her build system. Well, given that there is an measurable effect on real boost using code. I suggest we wait until Zachary wrote the script and did benchmarks on some of the plenty files at the libs test dirs. -- Maik
participants (9)
-
Christopher Jefferson -
Emil Dotchevski -
John Maddock -
Maik Beckmann -
Scott McMurray -
Sohail Somani -
Stephen Nuchia -
Stewart, Robert -
Zachary Turner