Should Boost adopt pimpl/d-pointer as much as it can?
Hello, There are recently lots of discussions about current development issues. I would like to point to one small but extremely important issue (from user point of view). Boost has too many logic in header part of library, by meaning too much I mean painfully too much. I think that Boost should adopt as policy: 1) All implementation related code **should** be put in sources. 2) Any non-template based class should have opaque/d/pimpl pointer in its body. Why? I give you a simple example: one of the pearls of Boost: Boost.Asio is one of most painful libraries to use. I created a simple example of echo server written with Boost.Asio and written with a small pimpl-wrapped Boost.Asio (I had written myself as proof of concept - that uses boost.asio in implementation): http://art-blog.no-ip.info/files/asio_pimpl.tar.gz These are results of compilation times with gcc-4.3 (seconds): -O0 -O2 -O3 -O2+link asio 2.56 4.65 4.96 4.75 pimpl 0.59 0.85 0.89 0.94 diff *4.3 *5.5 *5.6 *5.05 And this is trivial echo server. When the code gets larger and bigger it becomes even more painful. For me, compilation of components based on Asio is total nightmare because of unacceptable compilation times. Another issue is executable size. I had took same echo sever placed in 10 object compiled under different namespaces once I did it with Boost.Asio and then I did it with pimpl-ed Asio. Result (liked executable size with gcc-4.3): Build Size (-Os stripped) Asio: 245K Pimpl: 190K ---------------------------- Conclusions: ============ There are too many good libraries (Asio as good example) that use headers too much or headers only, this terribly increases compilation times and makes executable sizes much bigger (this is especially relevant for embedded systems). Should Boost wellcome pimpled libraries more? I think yes. It would solve two problems: 1. It would give faster compilation times. 2. It would allow to provide changes easily without breaking compatibility (even would allow to provide backward ABI compatibility) Dear Boosters. What do you think about? Artyom
Artyom wrote:
Boost has too many logic in header part of library, by meaning too much I mean painfully too much.
I think that Boost should adopt as policy:
1) All implementation related code **should** be put in sources. 2) Any non-template based class should have opaque/d/pimpl pointer in its body.
I created a simple example of echo server written with Boost.Asio and written with a small pimpl-wrapped Boost.Asio
I think that what you have done is the right solution - you, the end user, can wrap things in pimpls where you believe it is beneficial. It's fairly easy to convert a non-pimpl interface into a pimpl one, espcially if you need only a subset of it; on the other hand, it's impossible to convert a pimpl'd interface back into a non-pimpled one. Phil.
From: "Phil Endecott" <spam_from_boost_dev@chezphil.org>
Artyom wrote: 2) Any non-template based class should have opaque/d/pimpl pointer in its body.
I think that what you have done is the right solution - you, the end user, can wrap things in pimpls where you believe it is beneficial.
I am far from convinced that it is always the right solution to leave that descion to the end-user.
It's fairly easy to convert a non-pimpl interface into a pimpl one, espcially if you need only a subset of it;
I personally spent quite a bit of time looking at the idom and I am under strong impression that converting to the pimpl interface *correctly* is not *that* straightforward. More so, I am not convinced that giving that *option* (or leaving that burden) to the user is such a good idea.
on the other hand, it's impossible to convert a pimpl'd interface back into a non-pimpled one.
In my experience non-pimpled implementations do make sense. Say, for classes of limited complexity (like struct { int a, b; }) where passing by value is more efficient. Due to growing complexity of s/w the overhead of pimpl is often negligible and, therefore, an intelligent decision can be make by the implementer. However, pimpl's benefits are substantial, more importantly for *both* the end-user *and* the library developer. Pimpl's important properties are the separation of the interface and the implementation *and* implementation hiding. The implementation hiding is crucial for industrial-grade s/w (which Boost has become) as it frees implementer's hands. The user benefits as well, as he/she might not even need to recompile to deploy a new shared/dynamic library. I've been deploying Pimpl for a while -- eliminated a lot of hassle as far as implementer-user is concerned. Best, V.
I created a simple example of echo server written with Boost.Asio and written with a small pimpl-wrapped Boost.Asio
<snip> It's fairly easy to convert a non-pimpl interface into a pimpl one, espcially if you need only a subset of it;
You are wrong. I would even say, you are absolutely wrong. Have you ever tried to "pimplize" Boost.Asio even with small subset? I'm thinking about this more then a year. I'm active used of Asio in production environment and I still had to learn **a lot** about structure of Asio and implement few dozen pimpl interfaces in various libraries in order to write **any** kind of wrapper for Asio. So do not expect from average Boost user to wrap Asio or even simpler libraries. Artyom
On 03/21/2010 08:09 AM, Artyom wrote:
I created a simple example of echo server written with Boost.Asio and written with a small pimpl-wrapped Boost.Asio
<snip> It's fairly easy to convert a non-pimpl interface into a pimpl one, espcially if you need only a subset of it;
You are wrong. I would even say, you are absolutely wrong. Have you ever tried to "pimplize" Boost.Asio even with small subset? I'm thinking about this more then a year. I'm active used of Asio in production environment and I still had to learn **a lot** about structure of Asio and implement few dozen pimpl interfaces in various libraries in order to write **any** kind of wrapper for Asio.
So do not expect from average Boost user to wrap Asio or even simpler libraries.
I think pimpl should be used on by case basis, because it doesn't make sense in every library Boost has. In the particular case of ASIO, it may be reasonable to provide such interface _in_addition_ to the current header-only, if it is possible to create one without sacrificing genericity.
I think pimpl should be used on by case basis, because it doesn't make sense in every library Boost has.
I agree, but I think that there should be policy that recommends doing that for relevant parts. Just like today: Boost recommends using Boost.Test framework but it is up to library developer to decide whether it would use it or not. For example: I do not expect from Boost.Function, Boost.Bind or Boost.Pheonix to separate implementation from interfaces. However libraries like Asio, Thread, Regex, Interprocess and many others should. I don't see any reason why, memory allocation algorithm of Interprocess should be in header libraries at all? If something requires templates for flexibility, it should be in headers, but if it does not? Put it in source library and never recompile, especially when these parts of code huge and non-trivial! Artyom
Artyom wrote:
I think pimpl should be used on by case basis, because it doesn't make sense in every library Boost has.
I agree, but I think that there should be policy that recommends doing that for relevant parts.
Just like today: Boost recommends using Boost.Test framework but it is up to library developer to decide whether it would use it or not.
For example: I do not expect from Boost.Function, Boost.Bind or Boost.Pheonix to separate implementation from interfaces.
However libraries like Asio, Thread, Regex, Interprocess and many others should.
I don't see any reason why, memory allocation algorithm of Interprocess should be in header libraries at all?
If something requires templates for flexibility, it should be in headers, but if it does not? Put it in source library and never recompile, especially when these parts of code huge and non-trivial!
If you look at the archives of this mailing list, you will see that there was a concerted effort to keep libraries header-only in order to avoid the myriad of problems involved in ensuring that boost libraries are compiled with ALL required settings to be compatible with the users' build targets. In my experience - which I think the archives mirror - header-only libraries are easier to incorporate into existing build procedures, and enjoy a higher adoption rate due to that fact. Promoting a specific solution of pimpl/d to the implied problem of compile times will bring along as many negatives as it tries to alleviate. Jeff
<snip>
Artyom wrote:
I think pimpl should be used on by case basis, because it doesn't make sense in every library Boost has.
<snip>
I agree, but I think that there should be policy that recommends doing that for relevant parts.
<snip> If you look at the archives of this mailing list, you will see that there was a concerted effort to keep libraries header-only in order to avoid the myriad of problems involved in ensuring that boost libraries are compiled with ALL required settings to be compatible with the users' build targets.
I would only add one single notice: [Sarcasm] I would only rephrase you a little: in order to avoid the myriad of problems involved in ensuring that boost libraries are compiled with **one specific** compiler on **one** specific operating system where compiler run-times are not backward compatible between releases and even between debug/release builds, where is no standard naming conventions for various libraries types even exists, and building shared library requires extraordinary skills and lots of non-trivial configuration macros in order to get it done right. [/Sarcasm] Please, no offense all MSVC/MS Windows users, I agree, you (and me too) should not suffer this hell every day... ----- But still, I think that creation of header only libraries just in order to simplify build procedures, IMHO, are just not justified. Libraries are not designed for simple users they are designed for developers that should be capable to solve these problems. Best, Artyom
on 21.03.2010 at 23:16 Artyom wrote :
[Sarcasm] I would only rephrase you a little:
in order to avoid the myriad of problems involved in ensuring that boost libraries are compiled with **one specific** compiler on **one** specific operating system where compiler run-times are not backward compatible between releases and even between debug/release builds, where is no standard naming conventions for various libraries types even exists, and building shared library requires extraordinary skills and lots of non-trivial configuration macros in order to get it done right. [/Sarcasm] indeed this is the reality we are living in
But still, I think that creation of header only libraries just in order to simplify build procedures, IMHO, are just not justified.
Libraries are not designed for simple users they are designed for developers that should be capable to solve these problems. i strongly diagree with you i think one should strive to make his library open to as many users as it is possible, not only pro developers that is the scourge of modern software engineering (as i understand it) modern tools are so complicated that only very experienced programmers can handle it please don't make things worse than they are!
-- Pavel
Libraries are not designed for simple users they are designed for developers that should be capable to solve these problems. i strongly diagree with you i think one should strive to make his library open to as many users as it is possible, not only pro developers that is the scourge of modern software engineering (as i understand it) modern tools are so complicated that only very experienced programmers can handle it please don't make things worse than they are!
Is is really that difficult to work a little bit more on the documentation for the Getting Started Guide? I would dedicate some of my time if a small task force could be organized to improve it, if that's the major obstacle for moving code from .hpp to .cpp. I've been managing our companies build script (if you can call it that, it's very tiny) for boost so I've ran into to most of the common problems during the years that repeatedly gets into the mailing lists (for Windows platform, only). I find this argument that boost being too difficult to build as an old myth that is not actually true. If it -is- true, the correct solution must be to improve it, and not to force users to recompile boost sources every time they make a change to their own code. My thoughts, christian
Christian Holmquist wrote:
I find this argument that boost being too difficult to build as an old myth that is not actually true. If it -is- true, the correct solution must be to improve it, and not to force users to recompile boost sources every time they make a change to their own code.
I've always held that using, and hence compiling, Boost libraries should be as easy as just including the sources in your project. And I think most Boost libraries fall into that mode. Perhaps helping out to identify where that's ot the case and finding ways to make it that way would be beneficial to the whole Boost community. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org (msn) - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail
i strongly diagree with you i think one should strive to make his library open to as many users as it is possible, not only pro developers that is the scourge of modern software engineering (as i understand it) modern tools are so complicated that only very experienced programmers can handle it please don't make things worse than they are!
A lot of professional developers (and corporations) refuse to use boost because of the increased compile times in client code. Focusing on people who are not comfortable linking against other libraries (a pretty fundamental part of C/C++ development) to me focuses on the entirely wrong set of developers. Boost.Thread is a great example where there's no reason for it to be header only. Compare: #include <boost/thread/mutex.hpp> int main(int argc, char** argv) { } time /usr/bin/g++ -O3 test_thread_compile_time.cpp -o test_thread_compile_time real 0m0.492s user 0m0.420s sys 0m0.020s to the pthread equivalent (whose header includes far more than just mutices): #include <pthread.h> int main(int argc, char** argv) { } time /usr/bin/g++ -O3 test_pthread_compile_time.cpp -o test_pthread_compile_time real 0m0.091s user 0m0.020s sys 0m0.020s This is without even *using* anything from Boost.Thread, just including that one header -- it pulls in 227 other boost headers, 322 total. Changing that header to boost/thread/thread.hpp increases that time to ~0.9 seconds. The actual method of separation (pimpl, implementation in a library, etc.) doesn't matter, and in some cases it doesn't make sense, but it would be nice if there were a policy to try and keep compilation times for clients as low as is reasonably possible. Josh
on 23.03.2010 at 1:22 Josh Faust wrote :
i strongly diagree with you i think one should strive to make his library open to as many users as it is possible, not only pro developers [blah blah blah] A lot of professional developers (and corporations) refuse to use boost because of the increased compile times in client code. Focusing on people who are not comfortable linking against other libraries (a pretty fundamental part of C/C++ development) to me focuses on the entirely wrong set of developers.
[Boost.Thread vs. pthreads compilation example]
was that an argument? if so i say that separating declaration and implementation for non template code is a natural practice in C and C++ development i.e. declaration -- in headers (*.h), implementation -- in *.cpp or *.c personally i like the compile-once-link-any-time-you-want principle especially it fits well with msvc set of tools once you have a lib and the header you can put a file in your /include dir with the following contents //file "universe_formula" #include "path_to_actual_header/formula.h" #ifdef NDEBUG #pragma comment(lib, "universef.lib") #else #pragma comment(lib, "universefd.lib") #endif now if you #include <universe_formula> linker will be automatically informed about the package with symbols to resolve oh and i forgot that you need to put the .lib file(s) in your per project/global /lib dir -- Pavel
On Tue, Mar 23, 2010 at 10:24 AM, DE <satan66613@yandex.ru> wrote:
if so i say that separating declaration and implementation for non template code is a natural practice in C and C++ development i.e. declaration -- in headers (*.h), implementation -- in *.cpp or *.c personally i like the compile-once-link-any-time-you-want principle
especially it fits well with msvc set of tools once you have a lib and the header you can put a file in your /include dir with the following contents
Boost Build owns MSVC projects in that department. :) Assuming lib foo has a Jamfile, you just say "I use lib foo" and everything that should happen automatically does: you get the correct include paths added, the correct lib files linked, and if you're running a test -- the correct DLL/SO paths added. On any platform and any compiler. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
was that an argument? if so i say that separating declaration and implementation for non template code is a natural practice in C and C++ development i.e. declaration -- in headers (*.h), implementation -- in *.cpp or *.c personally i like the compile-once-link-any-time-you-want principle
Your comment made it sound like you wanted things to be header-only as much as possible. Auto-linking is not header only -- it only works on msvc, and as you said it still requires you to set up your lib paths. Josh
on 23.03.2010 at 20:48 Josh Faust wrote :
was that an argument? if so i say that separating declaration and implementation for non template code is a natural practice in C and C++ development i.e. declaration -- in headers (*.h), implementation -- in *.cpp or *.c personally i like the compile-once-link-any-time-you-want principle Your comment made it sound like you wanted things to be header-only as much as possible. Auto-linking is not header only -- it only works on msvc, and as you said it still requires you to set up your lib paths. no my comment was specifically on the following words:
on 21.03.2010 at 23:16 Artyom wrote :
Libraries are not designed for simple users they are designed for developers that should be capable to solve these problems.
on header only libraries, again: everything is good in it's season like some people say if a lib has a set of some handy functions which inline perfectly then there is perfect reason to make it header only but if there are huge non-template member/non-member functions -- it's better to separate the declaration part and implementation all this imho of course -- Pavel
On Sun, Mar 21, 2010 at 8:14 AM, Jeff Flinn <TriumphSprint2000@hotmail.com> wrote:
If you look at the archives of this mailing list, you will see that there was a concerted effort to keep libraries header-only in order to avoid the myriad of problems involved in ensuring that boost libraries are compiled with ALL required settings to be compatible with the users' build targets. In my experience - which I think the archives mirror - header-only libraries are easier to incorporate into existing build procedures, and enjoy a higher adoption rate due to that fact.
This line of thought misses the point of physically decoupling implementation details from interfaces. It *is* easier not to do it, this doesn't prove that it's better not to do it. A credit card with low introductory interest rate also enjoys higher adoption rate due to that fact, but it doesn't mean it's necessarily a good long term choice. The point of avoiding physical coupling is that changes in implementation details don't require user code to recompile. An important side effect is that the need for the ABIs to match is greatly reduced; so you *can* have a particular library compiled with different compiler options and that's OK. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
On 21 March 2010 18:37, Emil Dotchevski <emildotchevski@gmail.com> wrote:
This line of thought misses the point of physically decoupling implementation details from interfaces. It *is* easier not to do it, this doesn't prove that it's better not to do it.
But worse is better. "Just copy the headers" makes replication of the "virus" much easier.
AMDG Artyom wrote:
There are recently lots of discussions about current development issues.
I would like to point to one small but extremely important issue (from user point of view).
Boost has too many logic in header part of library, by meaning too much I mean painfully too much.
I think that Boost should adopt as policy:
1) All implementation related code **should** be put in sources. 2) Any non-template based class should have opaque/d/pimpl pointer in its body.
I am against such a policy. Such decisions should be left to the library author and the review process.
Why? I give you a simple example: one of the pearls of Boost: Boost.Asio is one of most painful libraries to use.
I created a simple example of echo server written with Boost.Asio and written with a small pimpl-wrapped Boost.Asio (I had written myself as proof of concept - that uses boost.asio in implementation):
http://art-blog.no-ip.info/files/asio_pimpl.tar.gz
These are results of compilation times with gcc-4.3 (seconds):
-O0 -O2 -O3 -O2+link asio 2.56 4.65 4.96 4.75 pimpl 0.59 0.85 0.89 0.94 diff *4.3 *5.5 *5.6 *5.05
And this is trivial echo server. When the code gets larger and bigger it becomes even more painful.
For me, compilation of components based on Asio is total nightmare because of unacceptable compilation times.
Another issue is executable size. I had took same echo sever placed in 10 object compiled under different namespaces once I did it with Boost.Asio and then I did it with pimpl-ed Asio.
Result (liked executable size with gcc-4.3):
Build Size (-Os stripped) Asio: 245K Pimpl: 190K
----------------------------
Conclusions: ============
There are too many good libraries (Asio as good example) that use headers too much or headers only, this terribly increases compilation times and makes executable sizes much bigger (this is especially relevant for embedded systems).
Should Boost wellcome pimpled libraries more? I think yes.
It would solve two problems:
1. It would give faster compilation times. 2. It would allow to provide changes easily without breaking compatibility (even would allow to provide backward ABI compatibility)
Dear Boosters. What do you think about?
I think making a general policy about this would be misguided. Looking over the list of libraries at There are many more libraries for which pimpls are either impossible or nonsensical than libraries for which they might make sense. Not to mention that there seems to be quite a bit of disagreement around here about whether having separate source is a good thing. In Christ, Steven Watanabe
On Sat, Mar 20, 2010 at 7:01 PM, Steven Watanabe <watanabesj@gmail.com> wrote:
AMDG
Artyom wrote:
There are recently lots of discussions about current development issues.
I would like to point to one small but extremely important issue (from user point of view).
Boost has too many logic in header part of library, by meaning too much I mean painfully too much.
I think that Boost should adopt as policy:
1) All implementation related code **should** be put in sources. 2) Any non-template based class should have opaque/d/pimpl pointer in its body.
I am against such a policy. Such decisions should be left to the library author and the review process.
In principle you're right, however most, even brilliant programmers, don't think that physical coupling is a big problem. That's the real problem, and it can only be addressed by a global requirement. But that's pointless in the case of Boost. First, most people will disagree with such a policy; we'll have endless arguments whether a "simple" function should be inline (it shouldn't if that's at all possible). Second, once physical coupling has you, it's extremely difficult to remove (the opposite -- introducing coupling to buy performance -- is always trivial.) Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Artyom wrote:
Why? I give you a simple example: one of the pearls of Boost: Boost.Asio is one of most painful libraries to use. [because it is header-only]
I believe asio is header-only because the author finds that more practical to use and deploy. It is him you should try to convince otherwise.
Result (liked executable size with gcc-4.3):
Build Size (-Os stripped) Asio: 245K Pimpl: 190K
Could you try this while whole program optimization (gcc 4.5+) turned on and tell me whether there is still a difference?
I believe asio is header-only because the author finds that more practical to use and deploy.
It is him you should try to convince otherwise.
It is not just ASIO, but many other libraries.
Result (liked executable size with gcc-4.3):
Build Size (-Os stripped) Asio: 245K Pimpl: 190K
Could you try this while whole program optimization (gcc 4.5+) turned on and tell me whether there is still a difference?
1. I do turned on full optimization (for size -Os), 2. 4.3 is quite latest and I don't have access to 4.5 (it is not even released yet...) 3. The issue is not compiler but maybe linker itself. What happens: Each sample server I compiled implemented its own "select loop", wait queue, even dispatcher, mutexes etc, etc. Each module compiles independently, it does not know if there is another module which implements same routines -- so there is lots of work for linker to find shared routines and merge them. I'm even not sure that according to standard it suppose to do such things. Artyom
Artyom, Did you try precompiling header files? My results are: inclusion 2,6 s (-02+link) precompilation 1,45 s (-02+link) diff *1.8* gcc 4.2, boost 1.41 Alexander Churanov
On Tue, Mar 23, 2010 at 12:00 PM, Alexander Churanov <alexanderchuranov@gmail.com> wrote:
Artyom,
Did you try precompiling header files?
Precompiling headers is usually a bad idea. For it to work well, you have to #include many things in a single header, which increases coupling. The fastest headers are those you don't #include. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
"Emil Dotchevski" <emildotchevski@gmail.com> wrote in message news:c72e33ce1003231213g6fd7b0f7i495206197eb4a24c@mail.gmail.com...
Precompiling headers is usually a bad idea. For it to work well, you have to #include many things in a single header, which increases coupling.
How exactly does it increase coupling? ...I yet have to find a case where including all 3rd party ('stable') library headers into one big precompiled header (that is force included into all compilation units) does not significantly (like by a third) reduce compilation times (with MSVC)... -- "What Huxley teaches is that in the age of advanced technology, spiritual devastation is more likely to come from an enemy with a smiling face than from one whose countenance exudes suspicion and hate." Neil Postman
Did you try precompiling header files?
I didn't -- I've only ever used pch with msvc so I didn't think to try it.
My results are:
inclusion 2,6 s (-02+link) precompilation 1,45 s (-02+link) diff *1.8*
1.45 seconds is still a large amount of time to add for inclusion of a single mutex header. As a concrete example, including mutex.hpp pulls in 50 date_time headers even if I'm not going to use timed_mutex. If timed_mutex were implemented in a cpp file, system_time could be forward-declared. The locks also don't look like they need the definition of system_time. I'll have to do some experimentation with this stuff and submit a patch if I can get compile times down. I don't want to derail this thread into something Boost.Thread-specific. Josh
From: Alexander Churanov <alexanderchuranov@gmail.com> Subject: Re: [boost] Should Boost adopt pimpl/d-pointer as much as it can? To: boost@lists.boost.org Date: Tuesday, March 23, 2010, 9:00 PM Artyom,
Did you try precompiling header files?
My results are:
inclusion 2,6 s (-02+link) precompilation 1,45 s (-02+link) diff *1.8*
gcc 4.2, boost 1.41
Alexander Churanov
There is one big issue with precompiled headers -- you only have one such header. So it is all or nothing, This is problematic, I tinkered a lot with PCH and various configurations. So far the best: - Use forward declarations - Include only that you need Gives much better performance then - Include everything in PCH - Include PCH everywhere. Artyom
On 24 Mar 2010, at 16:05, Andrey Semashev wrote:
On 03/24/2010 12:16 AM, Artyom wrote:
There is one big issue with precompiled headers -- you only have one such header.
Not true with GCC, AFAIK. The compiler will look for PCH before including the real header.
But it will only ever look for the PCH as the first header, for example given: #include "a.hpp" #include "b.hpp" It will look for a pre-compiled a.hpp, but not a pre-compiled b.hpp. This is because (among other problems) a.hpp may #define some symbols which are then used in b.hpp. Chris
On 03/24/2010 07:05 PM, Andrey Semashev wrote:
On 03/24/2010 12:16 AM, Artyom wrote:
There is one big issue with precompiled headers -- you only have one such header.
Not true with GCC, AFAIK. The compiler will look for PCH before including the real header.
Ah, looks like I'm not quite right. It does limit to a single PCH per compilation. But you can use different PCHs for different compilation units.
Folks, I usually generate a single header file that just includes other headers as a part of build process, and then include it using the "-include" compiler command-line option, this solves the issues with one precompiled header limit and the requirement for such a header to be the first. As a bonus, the source code remains unchanged, but compilation speeds up, build time drops to 60% of the original duration. Alexander Churanov
"Artyom" <artyomtnk@yahoo.com> wrote in message news:834975.41282.qm@web110205.mail.gq1.yahoo.com...
I think that Boost should adopt as policy:
1) All implementation related code **should** be put in sources.
I would agree, because most non-header-only libraries can probably AFAIK be easily converted to a header only simply by treating .cpp files as .inl/.ipp files that get included at the bottom of the header if a configuration option/macro specifies that a header only library is wanted (of course special care needs to be taken for static objects, static/inline functions and private/anonymous namespaces).
2) Any non-template based class should have opaque/d/pimpl pointer in its body.
I would most strongly disagree. The reason is, of course, efficiency. Let's take for example memory mapped files. You just want to map a file into memory and get a pair of pointers that mark the mapped memory range. For that boost gives us the iostreams::mapped_file class. Then you see that unfortunately this class uses the pimpl idiom, I really do not know why because, on Windows, the pointer to the mapped file view is at the same time also the handle to the view that also internaly holds the handle to the file mapping and the file being mapped (as it clearly stated on MSDN) so the only thing you need to hold as a member variable is a plain old simple void/char pointer...Additionaly you can query the size to have the pointer pair and a container interface. Even if you had to hold two additional handles it still makes no sense to use a pimpl because handles are also just plain pointers. But, the mentioned iostreams::mapped_file class uses a shared_ptr<> (even though a scoped_ptr would be enough) with all its nice and insignificant virtual functions, RTTI, double memory allocations, locking and indirect deleting. Before someone raises the "you are a premature optimizer and therefore evil" and/or the "you are doing disk access and going into kernel, who cares about pimpl overhead" sign posts let us look at the real world results that show where does this 'by the book ignoring of efficiency until a profiler says otherwise' take us. To be correct, the pimp+shared_ptr<> are not the only cause for the following results. The mentioned class is plaged by various other unnecessary C++-ifications (e.g. std::string usage when in fact all the underlying OS implementations work with plain char pointers). Anyways, to measure the space and time overhead of boost::iostreams::mapped_file (from boost 1.40) class I've used a custom created class (WinMemoryMappedFile, included in the attachment) that does just what I need (map a file, throw on error, and give me a begin and an end pointer) without all the shared_ptr, std::string, std::ios_base 'goodies'. Results: --SPACE-- - an empty project with an empty main function: 45 056 bytes (yes MSVC CRT is 'evil') - void main(...) { WinMemoryMappedFile f( "C:/boot.ini" ); std::ptrdiff_t size( f.end() - f.begin() ); } 46 080 bytes - void main(...) { boost::iostreams::mapped_file f( "C:/boot.ini" ); std::ptrdiff_t size( f.end() - f.begin() ); } 58 880 bytes so...to get the result of calling 6 different OS API calls and to throw an exception boost::iostreams::mapped_file brings in >13824< bytes of code (that's ~13 times more than WinMemoryMappedFile)... --TIME-- Spinning the above two lines 100000 times takes: - WinMemoryMappedFile ~1.71 seconds - boost::iostreams::mapped_file ~1.96 seconds ...that's about 15% slower/longer... I'd say all this quite clearly: - shows that avoiding 'premature optimization' is actually 'causing premature pessimization' - says no to forcing pimpls or any other type of abstraction or 'C++-ification' "on people" if it is not actually necessary... Otherwise I'might begin to understand Linus when he says "Quite frankly, even if the choice of C were to do *nothing* but keep the C++ programmers out, that in itself would be a huge reason to use C." (and I really do _not_ like C). In this particular case, for me the choice was simply not to use boost but my own implementation, which is said IMO...
These are results of compilation times with gcc-4.3 (seconds):
-O0 -O2 -O3 -O2+link asio 2.56 4.65 4.96 4.75 pimpl 0.59 0.85 0.89 0.94 diff *4.3 *5.5 *5.6 *5.05
Did you perhaps try to just move non-templated implementation into .cpps, without pimpling?
For me, compilation of components based on Asio is total nightmare because of unacceptable compilation times.
When using precompiled headers (which is always ;) I do not notice any compile time performance hit from just including a boost header (except when doing a rebuild of course :), using it however is a different thing of course if it is template based (but then you cannot pimpl it anyway)...
Another issue is executable size. I had took same echo sever placed in 10 object compiled under different namespaces once I did it with Boost.Asio and then I did it with pimpl-ed Asio.
Result (liked executable size with gcc-4.3):
Build Size (-Os stripped) Asio: 245K Pimpl: 190K
Hmm...this is strange...as if your compiler and linker are seriously 'broken' or 'weak' at throwing out unwanted things/merging same functions...If gcc has a 'link time code generation' option try using it... Using a pimpl based implementation should always give a slower and larger code (even if completely insignificantly) simply because of its overhead....
Conclusions: ============ makes executable sizes much bigger (this is especially relevant for embedded systems).
Actually, with a proper linker, pimpling makes your code bigger...
Should Boost wellcome pimpled libraries more? I think yes.
Only if provided as optional wrappers around otherwise 'lean' libraries...
2. It would allow to provide changes easily without breaking compatibility (even would allow to provide backward ABI compatibility)
Qt uses pimpling for just about everything, the result (not nearly only because of that of course): it has grown to an unbelievably humongous bloated monster that compiles (for hours) to a size of several server OS kernels... -- "What Huxley teaches is that in the age of advanced technology, spiritual devastation is more likely to come from an enemy with a smiling face than from one whose countenance exudes suspicion and hate." Neil Postman begin 666 WinMemoryMappedFile.hpp M+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O M+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\-"B\O+PT*+R\O M(%QF:6QE(%=I;DUE;6]R>4UA<'!E9$9I;&4N:'!P#0HO+R\@+2TM+2TM+2TM M+2TM+2TM+2TM+2TM+2TM+2TM+2T-"B\O+PT*+R\O($-O<'ER:6=H=" H8RD@ M,C Q,"X@1&]M86=O:B!387)I8RX-"B\O+PT*+R\O+R\O+R\O+R\O+R\O+R\O M+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O+R\O M+R\O+R\O+R\O+R\O+R\O+R\-"B\O+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM M+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM M+2TM+2TM+2TM#0HC<')A9VUA(&]N8V4-"B-I9FYD968@5VEN365M;W)Y36%P M<&5D1FEL95]H<'!?7S=",T%&-#(V7SDP1#)?-$1!15]!,#8P7T0X.30T1C T M14-%0@T*(V1E9FEN92!7:6Y-96UO<GE-87!P961&:6QE7VAP<%]?-T(S048T M,C9?.3!$,E\T1$%%7T$P-C!?1#@Y-#1&,#1%0T5"#0HO+RTM+2TM+2TM+2TM M+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM M+2TM+2TM+2TM+2TM+2TM+2TM+2TM+0T*(VEN8VQU9&4@(F)O;W-T+V%S<V5R M="YH<' B#0HC:6YC;'5D92 B8F]O<W0O;F]N8V]P>6%B;&4N:'!P(@T*(VEN M8VQU9&4@(F)O;W-T+W)A;F=E+VET97)A=&]R7W)A;F=E+FAP<"(-"@T*(VEN M8VQU9&4@(G=I;F1O=W,N:"(-"B\O+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM M+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM M+2TM+2TM+2TM#0H-"F-L87-S(%=I;DUE;6]R>4UA<'!E9$9I;&4-"B @(" Z M#0H@(" @<'5B;&EC("!B;V]S=#HZ:71E<F%T;W)?<F%N9V4\=6YS:6=N960@ M8VAA<B J/BP-"B @("!P<FEV871E(&)O;W-T.CIN;VYC;W!Y86)L90T*>PT* M<'5B;&EC.@T*(" @(%=I;DUE;6]R>4UA<'!E9$9I;&4H(&-H87(@8V]N<W0@ M*B!F:6QE3F%M92 I.PT*(" @('Y7:6Y-96UO<GE-87!P961&:6QE*"D[#0H- M"@T*<')I=F%T93H-"B @("!I=&5R871O<E]R86YG92 F(&%D=F%N8V5?8F5G M:6XH(&1I9F9E<F5N8V5?='EP92 I.PT*(" @(&ET97)A=&]R7W)A;F=E("8@ M861V86YC95]E;F0@("@@9&EF9F5R96YC95]T>7!E("D[#0H-"GT[#0H-"FEN M;&EN90T*5VEN365M;W)Y36%P<&5D1FEL93HZ5VEN365M;W)Y36%P<&5D1FEL M92@@8VAA<B!C;VYS=" J(&-O;G-T(&9I;&5.86UE("D-"GL-"B @("!"3T]3 M5%]!4U-%4E0H(&9I;&5.86UE("D[#0H-"B @("!(04Y$3$4@8V]N<W0@9FEL M95]H86YD;&4-"B @(" H#0H@(" @(" @(#HZ0W)E871E1FEL94$-"B @(" @ M(" @* T*(" @(" @(" @(" @9FEL94YA;64L($=%3D5224-?4D5!1"!\($=% M3D5224-?5U))5$4L($9)3$5?4TA!4D5?4D5!1"P@,"P@3U!%3E]!3%=!65,L M($9)3$5?05144DE"551%7U1%35!/4D%262P@, T*(" @(" @(" I#0H@(" @ M*3L-"B @("!"3T]35%]!4U-%4E0H("@@9FEL95]H86YD;&4@/3T@24Y604Q) M1%](04Y$3$5?5D%,544@*2!\?" H(#HZ1V5T3&%S=$5R<F]R*"D@/3T@3D]? M15)23U(@*2!\?" H(#HZ1V5T3&%S=$5R<F]R*"D@/3T@15)23U)?04Q214%$ M65]%6$E35%,@*2 I.PT*#0H@(" @+R\@270@:7,@)T]+)R!T;R!S96YD(&YU M;&PO:6YV86QI9"!H86YD;&5S('1O(%=I;F1O=W,@9G5N8W1I;VYS("AT:&5Y M('=I;&P-"B @(" O+R!S:6UP;'D@9F%I;"DL('1H:7,@<VEM<&QI9FEE<R!E M<G)O<B!H86YD;&EN9R H:70@:7,@96YO=6=H('1O(&=O('1H<F]U9V@-"B @ M(" O+R!A;&P@=&AE(&QO9VEC+"!I;G-P96-T('1H92!F:6YA;"!R97-U;'0@ M86YD('1H96X@=&AR;W<@;VX@97)R;W(I+@T*#0H@(" @+R\@0W)E871E1FEL M94UA<'!I;F<@86-C97!T<R!)3E9!3$E$7TA!3D1,15]604Q512!A<R!V86QI M9"!I;G!U="!B=70@;VYL>2!I9@T*(" @("\O('1H92!S:7IE('!A<F%M971E M<B!I<R!N;W0@;G5L;"X@5&AE(&9O;&QO=VEN9R!C86QL('=I;&P@=&AE<F5F M;W)E(&%L<V\-"B @(" O+R!F86EL(&5V96X@:68@9FEL95]H86YD;&4@/3T@ M24Y604Q)1%](04Y$3$5?5D%,544L(&IU<W0@87,@:7,@9&5S:7)E9"X-"B @ M("!(04Y$3$4@8V]N<W0@9FEL95]M87!P:6YG7VAA;F1L90T*(" @("@-"B @ M(" @(" @.CI#<F5A=&5&:6QE36%P<&EN9R@@9FEL95]H86YD;&4L(# L(%!! M1T5?4D5!1%=2251%+" P+" P+" P("D-"B @(" I.PT*(" @($)/3U-47T%3 M4T525 T*(" @("@-"B @(" @(" @* T*(" @(" @(" @(" @(69I;&5?;6%P M<&EN9U]H86YD;&4@)B8-"B @(" @(" @(" @("@-"B @(" @(" @(" @(" @ M(" H(" @9FEL95]H86YD;&4@(3T@24Y604Q)1%](04Y$3$5?5D%,544@*2!\ M? T*(" @(" @(" @(" @(" @("@@*"!F:6QE7VAA;F1L92 ]/2!)3E9!3$E$ M7TA!3D1,15]604Q512 I("8F("@@.CI'971,87-T17)R;W(H*2 ]/2!%4E)/ M4E])3E9!3$E$7U!!4D%-151%4B I("D-"B @(" @(" @(" @("D-"B @(" @ M(" @*0T*(" @(" @(" @(" @?'P-"B @(" @(" @*" Z.D=E=$QA<W1%<G)O M<B@I(#T]($Y/7T524D]2("D-"B @(" I.PT*#0H@(" @:71E<F%T;W(@8V]N M<W0@=FEE=U]S=&%R="@@<W1A=&EC7V-A<W0\:71E<F%T;W(^*" Z.DUA<%9I M97=/9D9I;&4H(&9I;&5?;6%P<&EN9U]H86YD;&4L($9)3$5?34%07T%,3%]! M0T-%4U,L(# L(# L(# @*2 I("D[#0H@(" @#0H@(" @1%=/4D0@8V]N<W0@ M9FEL95]S:7IE*" Z.D=E=$9I;&53:7IE*"!F:6QE7VAA;F1L92P@," I("D[ M#0H@(" @0D]/4U1?05-315)4*" H(&9I;&5?<VEZ92 A/2 M,2 I('Q\("@@ M9FEL95]H86YD;&4@/3T@24Y604Q)1%](04Y$3$5?5D%,544@*2!\?" H(#HZ M1V5T3&%S=$5R<F]R*"D@/3T@3D]?15)23U(@*2 I.PT*#0H@(" @+R\@26UP M;&5M96YT871I;VX@;F]T93H-"B @(" O+R!-87!P960@=FEE=W,@:&]L9"!I M;G1E<FYA;"!R969E<F5N8V5S('1O('1H92!F;VQL;W=I;F<@:&%N9&QE<R!S M;R!W92!D;PT*(" @("\O(&YO="!N965D(&AO;&0O<W1O<F4@=&AE;2!O=7)S M96QV97,Z#0H@(" @+R\@:'1T<#HO+VUS9&XN;6EC<F]S;V9T+F-O;2]E;BUU M<R]L:6)R87)Y+V%A,S8V-3,W*%93+C@U*2YA<W!X#0H@(" @+R\@(" @(" @ M(" @(" @(" @(" @(" @(" @(" @(" @(" @(" @(" @*#(V+C S+C(P,3 N M*2 H1&]M86=O:B!387)I8RD-"B @("!"3T]35%]615))1EDH(#HZ0VQO<V5( M86YD;&4H(&9I;&5?:&%N9&QE(" @(" @(" @*2!\?" H(&9I;&5?:&%N9&QE M(#T]($E.5D%,241?2$%.1$Q%7U9!3%5%("D@*3L-"B @("!"3T]35%]615)) M1EDH(#HZ0VQO<V5(86YD;&4H(&9I;&5?;6%P<&EN9U]H86YD;&4@*2!\?" A M9FEL95]M87!P:6YG7VAA;F1L92 @(" @(" @(" @(" @(" @(" @*3L-"@T* M(" @(&EF("@@(79I97=?<W1A<G0@*0T*(" @(" @("!T:')O=R!S=&0Z.F5X M8V5P=&EO;B@@(D9I;&4@;65M;W)Y(&UA<'!I;F<@9F%I;&5D+B(@*3L-"@T* M(" @('1Y<&5D968@='EP92!M>5]R86YG93L-"B @("!M>5]R86YG93HZ;W!E M<F%T;W(@/2 H(&UY7W)A;F=E*"!V:65W7W-T87)T+"!V:65W7W-T87)T("L@ M9FEL95]S:7IE("D@*3L-"GT-"@T*:6YL:6YE#0I7:6Y-96UO<GE-87!P961& M:6QE.CI^5VEN365M;W)Y36%P<&5D1FEL92@I#0I[#0H@(" @0D]/4U1?5D52 M2499*" Z.E5N;6%P5FEE=T]F1FEL92@@8F5G:6XH*2 I("D[#0I]#0H-"@T* M+R\M+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM M+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2TM+2T-"B-E;F1I9B O =+R!7:6Y-96UO<GE-87!P961&:6QE7VAP< T*#0H` ` end
2) Any non-template based class should have opaque/d/pimpl pointer in its body.
I would most strongly disagree. The reason is, of course, efficiency.
Ok, let it make more clear. If you want Boost to be ever stable library as Qt, GLib, STL you need this. This would allow to extend the class in future without breaking ABI. I do not suggest moving every member to pimpl, I suggest to have it in order to add new members when you need. See this example in CppCMS class http::cookie: Header: class CPPCMS_API cookie { public: std::string name() const; std::string value() const; .... void name(std::string n); void value(std::string v); ... cookie(); ~cookie(); cookie(cookie const &); cookie const &operator=(cookie const &); private: // for future use struct data; util::copy_ptr<data> d; // real members std::string name_; std::string value_; ... }; And in implementation struct data is defined as struct data{}; and it is not even allocated when class created. But, if I do want to add a new member without breaking any binary compatibility I can! This is very important. So you do not have to use pimpl by default everywhere. But you need an option to have it.
Did you perhaps try to just move non-templated implementation into .cpps, without pimpling?
No I didn't it would be too hard, ASIO is very templetized implementation, Most of classes I implemented as template specialization (to make it look like ASIO).
When using precompiled headers (which is always ;) I do not notice any compile time performance hit
You can take a look on this list and find many pros and cons against precompiled headers. From my experience it is better to reduce size of includes then use such headers. Also may compilers do not support them or even not support them properly. For exampple I had issues with GCC perecompiled headers when was using Boost.Asio with 4.1 or even newer versions.
Hmm...this is strange...as if your compiler and linker are seriously 'broken' or 'weak' at throwing out unwanted things/merging same functions...If gcc has a 'link time code generation' option try using it...
In fact most of linkers I used do not have such thing. I mean they know to throw unused functions but not merge two functions implemented in different objects. If MSVC has good for him.
Qt uses pimpling for just about everything, the result (not nearly only because of that of course)
Ohhh I'm really glad you mentioned Qt... Qt is one of the finest libraries I had used. It is totally different from Boost in they behavior. One of the greatest things about Qt... I may compile executable for Linux and just give it... It would run on other Linux that have Qt... I can upgrade Qt for years and my program would run without recompilation. Why? Qt provides stable ABI. This is one of greatest features it has! Have you tried to distribute a compiled software liked with Boost? It is impossible! Because Boost releases new version every few month and they are not compatible even at API (not talking about ABI) level. I wish Boost would learn something from Qt...
it has grown to an unbelievably humongous bloated monster that compiles (for hours) to a size of several server OS kernels...
Just to mention. Have you tryed to compile full Boost? It compiles for hours. But forget. What Qt brings you? GUI, Threading, Networking, XML, Webkit HTML Engine SQL, OpenGL, and more... 2/3 of this even does not exist in Boost. Artyom
On 26 Mar 2010, at 14:48, Artyom wrote:
2) Any non-template based class should have opaque/d/pimpl pointer in its body.
I would most strongly disagree. The reason is, of course, efficiency.
Ok, let it make more clear. If you want Boost to be ever stable library as Qt, GLib, STL you need this. This would allow to extend the class in future without breaking ABI.
I do not know of any implementation of the STL which is pimpl. With the exception of some parts of string and iostreams, they are also all header-only. pimpl is over-rated for not breaking the ABI. Chris
Artyom wrote:
Have you tried to distribute a compiled software liked with Boost? It is impossible! Because Boost releases new version every few month and they are not compatible even at API (not talking about ABI) level.
Using only static boost libraries solves that problem with reasonable safety whenever you decide to upgrade the boost version you are using. We actually do not even compile the dynamic libraries to prevent mishaps. --> Mika Heiskanen
participants (17)
-
Alexander Churanov -
Andrey Semashev -
Artyom -
Christian Holmquist -
Christopher Jefferson -
DE -
Domagoj Saric -
Emil Dotchevski -
Jeff Flinn -
Josh Faust -
Mathias Gaunard -
Mika Heiskanen -
Phil Endecott -
Rene Rivera -
Scott McMurray -
Steven Watanabe -
Vladimir Batov