[Modularization] A new approach to header modularization
I've started to put together a wiki page based on our ongoing Header Modularization discussions. See http://svn.boost.org/trac/boost/wiki/HeaderModularization I listed eight approaches to meeting the Goals/Objectives/Needs/Wants, and was about to identify the weaknesses of each approach when I hit on approach nine: Move (permanently) each library's root/libs/libname contents into root/boost/libname for the library. In other words, instead of moving the boost/... tree headers to the libs/... tree, merge the libs/... tree into the boost/... tree. For example, Boost.Filesystem is currently organized like this: root/ boost/ filesystem.hpp filesystem/ config.hpp convenience.hpp exception.hpp fstream.hpp operations.hpp path.hpp libs/ filesystem/ build/ ... CMakeLists.txt doc/ ... example/ ... index.html ... module.cmake src/ ... test/ ... After the reorganization, filesystem would be organized like this: root/ boost/ filesystem.hpp filesystem/ config.hpp convenience.hpp exception.hpp fstream.hpp operations.hpp path.hpp build/ ... CMakeLists.txt doc/ ... example/ ... index.html ... module.cmake src/ ... test/ ... In theory, the root/libs/... tree isn't needed anymore. In practice we might want to keep a root/libs/libname/index.html entry for each library so docs links don't break. Apologies if someone already suggested this approach. It is so simple I can't believe someone didn't already suggest it. Unless I'm missing something, this is the only approach that appears to do fairly well for all of the Goals/Objectives/Needs/Wants, and very well for most of them. It would require some rework of scripts and other setups that depend on the current organization of libs/..., but that seems minor compared to the long-term benefits. Comments? --Beman
On Wed, May 27, 2009 at 01:06:47PM -0400, Beman Dawes wrote:
Unless I'm missing something, this is the only approach that appears to do fairly well for all of the Goals/Objectives/Needs/Wants, and very well for most of them. It would require some rework of scripts and other setups that depend on the current organization of libs/..., but that seems minor compared to the long-term benefits.
Comments?
I wouldn't be surprised if the following installation procedure was common among users of the library: bjam stage to target/lib copy boost-1.xx.y/boost to target/include With the proposed reorg, the resulting include dir would be full of source and build gunk, and would force the builder to use `bjam install' instead, which produces lots of bulky intermediaries and takes ages to deploy the headers. -- Lars Viklund | zao@acc.umu.se | 070-310 47 07
In no particular order: - There is still the toplevel filesystem.hpp and friends to deal with. You can't move libraries around as a unit (which is the desired capability, right?). - You can't tell what headers don't belong to any 'project'. - It is a lot harder to recursively search for function 'foo' in the headers; you're going to get hits from examples, tests, docs, etc. - It makes it possible to include things you're not supposed to, e.g. private headers from other projects. There is also the potential for name collisions between directories under libs/X and boost/X, You would want some naming convention: boost/ filesystem.hpp filesystem/ O_o/ <---- build/ test/ doc/ or maybe something pythonic like __src__/. - Users expect headers under include/ and source under src/, this will surprise them. - I'm not aware of any prior art. - Will boost.build in fact be able to differentiate between headers, build products, and source files for the sake of installation? - I still like it more than svn:externals -t
troy d. straszheim wrote:
In no particular order:
- There is still the toplevel filesystem.hpp and friends to deal with. You can't move libraries around as a unit (which is the desired capability, right?).
- You can't tell what headers don't belong to any 'project'.
- It is a lot harder to recursively search for function 'foo' in the headers; you're going to get hits from examples, tests, docs, etc.
- It makes it possible to include things you're not supposed to, e.g. private headers from other projects. There is also the potential for name collisions between directories under libs/X and boost/X, You would want some naming convention:
boost/ filesystem.hpp filesystem/ O_o/ <---- build/ test/ doc/
or maybe something pythonic like __src__/.
- Users expect headers under include/ and source under src/, this will surprise them.
- I'm not aware of any prior art.
- Will boost.build in fact be able to differentiate between headers, build products, and source files for the sake of installation?
Yes, of course. I don't see anything particular hard here. But I am also concerned about this approach for other reasons you list above. - Volodya
On Thu, May 28, 2009 at 10:53 AM, troy d. straszheim <troy@resophonic.com> wrote:
In no particular order:
- There is still the toplevel filesystem.hpp and friends to deal with.
The way to deal with that is to migrate those to within the library's header directory. As boost grows, filling the boost/ with headers becomes less and less attractive. Of course we have to leave the current headers there for compatibiltiy, but that location could be deprecated and eventually only <boost/filesystem/filesystem.hpp> supported.
You can't move libraries around as a unit (which is the desired capability, right?).
You can except for the current boost/ .hpp files. Whether that's a problem or not depends on whether we are willing to move to all headers in the library's home. That may well be an eventual need regardless of the modularization approach.
- You can't tell what headers don't belong to any 'project'.
Same as now. The headers in boost/ are a problem.
- It is a lot harder to recursively search for function 'foo' in the headers; you're going to get hits from examples, tests, docs, etc.
The command gets a bit more complicated. But "a lot harder":-?
- It makes it possible to include things you're not supposed to, e.g. private headers from other projects.
That's true already. If you know what the overall directory structure is, you can always include files not meant for public use.
There is also the potential for name collisions between directories under libs/X and boost/X, You would want some naming convention:
boost/ filesystem.hpp filesystem/ O_o/ <---- build/ test/ doc/
Yes, you could do that. But we can also reserve certan names (doc/src/test/example/build). That would go a long way.
or maybe something pythonic like __src__/.
- Users expect headers under include/ and source under src/, this will surprise them.
Yep. Just like it surprises them now that there is no include/. They get over it.
- I'm not aware of any prior art.
Neither am I, except many years ago in some C projects I once worked on.
- Will boost.build in fact be able to differentiate between headers, build products, and source files for the sake of installation?
Apparently.
- I still like it more than svn:externals
The more I think about svn:externals, the more I think it is the wrong tool for this particular job. If there was a svn:hardlink, that would be a strong contender. Some source control systems do support such functionality. I wonder how hard it would be to add to Subversion? --Beman
Beman Dawes wrote On Thursday, May 28, 2009 5:07 PM
On Thu, May 28, 2009 at 10:53 AM, troy d. straszheim <troy@resophonic.com> wrote:
- There is still the toplevel filesystem.hpp and friends to deal with.
The way to deal with that is to migrate those to within the library's header directory.
+1
As boost grows, filling the boost/ with headers becomes less and less attractive. Of course we have to leave the current headers there for compatibiltiy, but that location could be deprecated and eventually only <boost/filesystem/filesystem.hpp> supported.
Rather than being redundant, why not make it <boost/library/all.hpp>, where "library" varies?
You can't move libraries around as a unit (which is the desired capability, right?).
You can except for the current boost/ .hpp files. Whether that's a problem or not depends on whether we are willing to move to all headers in the library's home. That may well be an eventual need regardless of the modularization approach.
Those wanting to take advantage of the modularization will change their include directives from boost/library.hpp to boost/library/all.hpp.
There is also the potential for name collisions between directories under libs/X and boost/X, You would want some naming convention:
boost/ filesystem.hpp filesystem/ O_o/ <---- build/ test/ doc/
Yes, you could do that. But we can also reserve certan names (doc/src/test/example/build). That would go a long way.
+1
or maybe something pythonic like __src__/.
-1
- Users expect headers under include/ and source under src/, this will surprise them.
Yep. Just like it surprises them now that there is no include/. They get over it.
It never hurts to follow convention when possible and reasonable. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
On Fri, May 29, 2009 at 7:52 AM, Stewart, Robert <Robert.Stewart@sig.com> wrote:
Beman Dawes wrote
...
As boost grows, filling the boost/ with headers becomes less and less attractive. Of course we have to leave the current headers there for compatibiltiy, but that location could be deprecated and eventually only <boost/filesystem/filesystem.hpp> supported.
Rather than being redundant, why not make it <boost/library/all.hpp>, where "library" varies?
Interesting. I like it! --Beman
on Fri May 29 2009, "Stewart, Robert" <Robert.Stewart-AT-sig.com> wrote:
Rather than being redundant, why not make it <boost/library/all.hpp>, where "library" varies?
+1 -- Dave Abrahams BoostPro Computing http://www.boostpro.com
On Thursday 28 May 2009 04:53:18 pm troy d. straszheim wrote:
In no particular order:
- There is still the toplevel filesystem.hpp and friends to deal with. You can't move libraries around as a unit (which is the desired capability, right?).
- You can't tell what headers don't belong to any 'project'.
- It is a lot harder to recursively search for function 'foo' in the headers; you're going to get hits from examples, tests, docs, etc.
- It makes it possible to include things you're not supposed to, e.g. private headers from other projects.
One possible way to do this is to stage/install public headers in similar ways as is done with libs: - stage/install all modified headers - build libs with include paths set to staging/install area - stage/install libs - build and run tests -- Bjørn
On Thu, May 28, 2009 at 5:35 AM, Lars Viklund <zao@acc.umu.se> wrote:
On Wed, May 27, 2009 at 01:06:47PM -0400, Beman Dawes wrote:
Unless I'm missing something, this is the only approach that appears to do fairly well for all of the Goals/Objectives/Needs/Wants, and very well for most of them. It would require some rework of scripts and other setups that depend on the current organization of libs/..., but that seems minor compared to the long-term benefits.
Comments?
I wouldn't be surprised if the following installation procedure was common among users of the library:
bjam stage to target/lib copy boost-1.xx.y/boost to target/include
With the proposed reorg, the resulting include dir would be full of source and build gunk, and would force the builder to use `bjam install' instead, which produces lots of bulky intermediaries and takes ages to deploy the headers.
I don't see that as the worst thing in the world. Also, as we move to installers with prebuilt binaries, the whole completion of the user install problem changes. --Beman
Beman Dawes wrote:
[snip] root/ boost/ filesystem.hpp filesystem/ config.hpp convenience.hpp exception.hpp fstream.hpp operations.hpp path.hpp build/ ... CMakeLists.txt doc/ ... example/ ... index.html ... module.cmake src/ ... test/ ...
Looks ugly for my taste, sorry.
In theory, the root/libs/... tree isn't needed anymore. In practice we might want to keep a root/libs/libname/index.html entry for each library so docs links don't break.
Apologies if someone already suggested this approach. It is so simple I can't believe someone didn't already suggest it.
Unless I'm missing something, this is the only approach that appears to do fairly well for all of the Goals/Objectives/Needs/Wants,
It doesn't make dependencies of header-only libraries explicit.
[snip]
On Thu, May 28, 2009 at 10:31 AM, Ilya Sokolov <ilyasokol@gmail.com> wrote:
Beman Dawes wrote:
[snip] root/ boost/ filesystem.hpp filesystem/ config.hpp convenience.hpp exception.hpp fstream.hpp operations.hpp path.hpp build/ ... CMakeLists.txt doc/ ... example/ ... index.html ... module.cmake src/ ... test/ ...
Looks ugly for my taste, sorry.
It is somewhat ugly. But the alternatives, including our current scheme, are also somewhat ugly.
In theory, the root/libs/... tree isn't needed anymore. In practice we might want to keep a root/libs/libname/index.html entry for each library so docs links don't break.
Apologies if someone already suggested this approach. It is so simple I can't believe someone didn't already suggest it.
Unless I'm missing something, this is the only approach that appears to do fairly well for all of the Goals/Objectives/Needs/Wants,
It doesn't make dependencies of header-only libraries explicit.
Dependencies are a separate issue. AFAICS, that's also true for the alternatives too. --Beman
2009/5/28 Beman Dawes <bdawes@acm.org>:
On Thu, May 28, 2009 at 10:31 AM, Ilya Sokolov <ilyasokol@gmail.com> wrote:
Beman Dawes wrote:
[snip] root/ boost/ filesystem.hpp filesystem/ config.hpp convenience.hpp exception.hpp fstream.hpp operations.hpp path.hpp build/ ... CMakeLists.txt doc/ ... example/ ... index.html ... module.cmake src/ ... test/ ...
Looks ugly for my taste, sorry.
It is somewhat ugly. But the alternatives, including our current scheme, are also somewhat ugly.
I'd prefer it if the non-header files were all in a single directory with a standard name. That way if I list the files in boost/filesystem there'll be less noise. And it'll make it easier to use the unix command 'find'. So; boost/filesystem.hpp boost/filesystem/config.hpp boost/filesystem/lib/index.html boost/filesystem/lib/example/ boost/filesystem/lib/test/ boost/filesystem/lib/src/ Replacing 'lib' with whatever you fancy. Although, you can add me to the people who are currently happy enough with the status quo. Daniel
On Thu, May 28, 2009 at 2:31 PM, Daniel James <daniel_james@fmail.co.uk> wrote:
Although, you can add me to the people who are currently happy enough with the status quo.
Me too, I don't understand what would this reorganization improve. What would make sense for me personally would be an effort to reduce physical coupling in Boost, by moving as much code as possible from headers to CPP files. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
On 28 May 2009, at 22:45, Emil Dotchevski wrote:
On Thu, May 28, 2009 at 2:31 PM, Daniel James <daniel_james@fmail.co.uk
wrote: Although, you can add me to the people who are currently happy enough with the status quo.
Me too, I don't understand what would this reorganization improve. What would make sense for me personally would be an effort to reduce physical coupling in Boost, by moving as much code as possible from headers to CPP files.
Out of interest, how would this help? The majority of libraries make no attempt to maintain a stable API from version to version, so the only gain I could see would be reduced compile time. Useful yes, but not obvious helping maintence? Also, so many (all?) libraries use templates, which won't go into cpp files. Chris
On Thu, May 28, 2009 at 3:34 PM, Christopher Jefferson <chris@bubblescope.net> wrote:
On 28 May 2009, at 22:45, Emil Dotchevski wrote:
On Thu, May 28, 2009 at 2:31 PM, Daniel James <daniel_james@fmail.co.uk> wrote:
Although, you can add me to the people who are currently happy enough with the status quo.
Me too, I don't understand what would this reorganization improve. What would make sense for me personally would be an effort to reduce physical coupling in Boost, by moving as much code as possible from headers to CPP files.
Out of interest, how would this help? The majority of libraries make no attempt to maintain a stable API from version to version, so the only gain I could see would be reduced compile time. Useful yes, but not obvious helping maintence?
It would help maintenance in two ways: - by minimizing the number of compilation units that need to be recompiled when making a change - by minimizing the amount of code that's visible to the compiler for each compilation unit when it does need to be recompiled
Also, so many (all?) libraries use templates, which won't go into cpp files.
Many function templates have parts that don't depend on template parameters and could be separated in a cpp file. Also, many Boost headers contain regular (non-template) inline functions, because there's pressure from users (and developers) to keep Boost libraries header-only. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski escribió:
What would make sense for me personally would be an effort to reduce physical coupling in Boost, by moving as much code as possible from headers to CPP files.
[...]
Many function templates have parts that don't depend on template parameters and could be separated in a cpp file. Also, many Boost headers contain regular (non-template) inline functions, because there's pressure from users (and developers) to keep Boost libraries header-only.
Why there's pressure from users to keep Boost libs header-only? Boost is for the users, so their reasons should be given proper weight. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
On Thu, May 28, 2009 at 10:50 PM, <joaquin@tid.es> wrote:
Emil Dotchevski escribió:
What would make sense for me personally would be an effort to reduce physical coupling in Boost, by moving as much code as possible from headers to CPP files.
[...]
Many function templates have parts that don't depend on template parameters and could be separated in a cpp file. Also, many Boost headers contain regular (non-template) inline functions, because there's pressure from users (and developers) to keep Boost libraries header-only.
Why there's pressure from users to keep Boost libs header-only? Boost is for the users, so their reasons should be given proper weight.
One can not create good design (in general, not just in software) by asking the users what would they like. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski <emildotchevski <at> gmail.com> writes:
On Thu, May 28, 2009 at 10:50 PM, <joaquin <at> tid.es> wrote:
Why there's pressure from users to keep Boost libs header-only? Boost is for the users, so their reasons should be given proper weight.
One can not create good design (in general, not just in software) by asking the users what would they like.
I'd rather not go into discussing such far-reaching issues as design theory, but you haven't asked my question: What are the particular reasons why users demand header-only libraries? Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
On Thu, May 28, 2009 at 11:15 PM, Joaquin M Lopez Munoz <joaquin@tid.es> wrote:
Emil Dotchevski <emildotchevski <at> gmail.com> writes:
On Thu, May 28, 2009 at 10:50 PM, <joaquin <at> tid.es> wrote:
Why there's pressure from users to keep Boost libs header-only? Boost is for the users, so their reasons should be given proper weight.
One can not create good design (in general, not just in software) by asking the users what would they like.
I'd rather not go into discussing such far-reaching issues as design theory, but you haven't asked my question: What are the particular reasons why users demand header-only libraries?
Probably different users have different reasons. One reason might be that it makes Boost easier to install initially. What's your point? Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski escribió:
On Thu, May 28, 2009 at 11:15 PM, Joaquin M Lopez Munoz <joaquin@tid.es> wrote:
Emil Dotchevski <emildotchevski <at> gmail.com> writes:
On Thu, May 28, 2009 at 10:50 PM, <joaquin <at> tid.es> wrote:
Why there's pressure from users to keep Boost libs header-only? Boost is for the users, so their reasons should be given proper weight.
One can not create good design (in general, not just in software) by asking the users what would they like.
I'd rather not go into discussing such far-reaching issues as design theory, but you haven't asked my question: What are the particular reasons why users demand header-only libraries?
Probably different users have different reasons. One reason might be that it makes Boost easier to install initially. What's your point?
My point is that if there's pressure from users to have header-only libs (as you and also I recognize) I think the least we can do is try to understand and analyze these reasons to see their merit. If pressure were the other way around (i.e. users demanding that code be moved out of .hpps as much as possible) we wouldn't be having this sort of discussions. As a user, I can describe *my* reasons to favor header-only libs: 1. The whole bjam-driven building process is nontrivial and time and space consuming. 2. If autolinking is not available, picking up the right lib variant is not trivial. 3. Bulding libs selectively is not as easy as it might seem, due to the fact that interlib dependencies might force you to build libB when using libA, and you don't know in advance. 4. I'm not concerned about ABI issues given that, to start with, no ABI compatibility guarantees are provided across Boost versions. This is not to say that I'd like *every* lib to be header-only; but I'd say the benefits of moving to a link-based lib should be balanced against points 1-4. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
joaquin@tid.es wrote:
Emil Dotchevski escribió:
On Thu, May 28, 2009 at 11:15 PM, Joaquin M Lopez Munoz <joaquin@tid.es> wrote:
Emil Dotchevski <emildotchevski <at> gmail.com> writes:
On Thu, May 28, 2009 at 10:50 PM, <joaquin <at> tid.es> wrote:
Why there's pressure from users to keep Boost libs header-only? Boost is for the users, so their reasons should be given proper weight.
One can not create good design (in general, not just in software) by asking the users what would they like.
I'd rather not go into discussing such far-reaching issues as design theory, but you haven't asked my question: What are the particular reasons why users demand header-only libraries?
Probably different users have different reasons. One reason might be that it makes Boost easier to install initially. What's your point?
My point is that if there's pressure from users to have header-only libs (as you and also I recognize) I think the least we can do is try to understand and analyze these reasons to see their merit. If pressure were the other way around (i.e. users demanding that code be moved out of .hpps as much as possible) we wouldn't be having this sort of discussions.
It it my understanding (based on actually supporting users on IRC and mailing lists for years), that the users who are mostly concerned about linking are either: 1. Users who just crossed the chasm between the two popular platforms, in either direction. 2. Users who don't understand the difference between headers and libraries in general, and don't know how to use their IDE. 3. Users who don't know C++. Users in (1) group will adapt quickly. Users in groups (2) and (3) probably won't be able to use Boost effectively until they do some other learning.
As a user, I can describe *my* reasons to favor header-only libs:
1. The whole bjam-driven building process is nontrivial and time and space consuming.
In 1.39, it is two commands in 1.39, on popular platforms. And processors are fast these days.
2. If autolinking is not available, picking up the right lib variant is not trivial.
It is not an issue on GNU/Linux.
3. Bulding libs selectively is not as easy as it might seem, due to the fact that interlib dependencies might force you to build libB when using libA, and you don't know in advance.
And you should not care. If you add --with-filesystem, then the system library will be built automatically.
4. I'm not concerned about ABI issues given that, to start with, no ABI compatibility guarantees are provided across Boost versions.
The fact that there's no ABI compatibility (or API, either), is a big problem in itself. - Volodya
On 29 May 2009, at 07:49, Vladimir Prus wrote:
joaquin@tid.es wrote:
Emil Dotchevski escribió:
On Thu, May 28, 2009 at 11:15 PM, Joaquin M Lopez Munoz <joaquin@tid.es
wrote:
Emil Dotchevski <emildotchevski <at> gmail.com> writes:
On Thu, May 28, 2009 at 10:50 PM, <joaquin <at> tid.es> wrote:
Why there's pressure from users to keep Boost libs header-only? Boost is for the users, so their reasons should be given proper weight.
One can not create good design (in general, not just in software) by asking the users what would they like.
I'd rather not go into discussing such far-reaching issues as design theory, but you haven't asked my question: What are the particular reasons why users demand header-only libraries?
Probably different users have different reasons. One reason might be that it makes Boost easier to install initially. What's your point?
My point is that if there's pressure from users to have header-only libs (as you and also I recognize) I think the least we can do is try to understand and analyze these reasons to see their merit. If pressure were the other way around (i.e. users demanding that code be moved out of .hpps as much as possible) we wouldn't be having this sort of discussions.
It it my understanding (based on actually supporting users on IRC and mailing lists for years), that the users who are mostly concerned about linking are either:
1. Users who just crossed the chasm between the two popular platforms, in either direction.
2. Users who don't understand the difference between headers and libraries in general, and don't know how to use their IDE.
3. Users who don't know C++.
Users in (1) group will adapt quickly. Users in groups (2) and (3) probably won't be able to use Boost effectively until they do some other learning.
I have consistently found that boost seems to cause more problems for developers than all the other libraries I use put together, in terms of building and linking. I work with a large number of developers on users on many different OSes and processors, so we distribute as source. Some users use packaged copies of boost distributed with their system, other compile from source.
As a user, I can describe *my* reasons to favor header-only libs:
1. The whole bjam-driven building process is nontrivial and time and space consuming.
In 1.39, it is two commands in 1.39, on popular platforms. And processors are fast these days.
Is if ./configure && make?
3. Bulding libs selectively is not as easy as it might seem, due to the fact that interlib dependencies might force you to build libB when using libA, and you don't know in advance.
And you should not care. If you add --with-filesystem, then the system library will be built automatically.
Most users aren't going to go around looking for options, they will assume the default ones do the right thing. I do love boost, which is why I've fought to make sure we continue using it, but compared to other libraries it is often a pain. Much of the pain goes away for header-only libraries. Chris
Christopher Jefferson wrote:
On 29 May 2009, at 07:49, Vladimir Prus wrote:
joaquin@tid.es wrote:
Emil Dotchevski escribió:
On Thu, May 28, 2009 at 11:15 PM, Joaquin M Lopez Munoz <joaquin@tid.es
wrote:
Emil Dotchevski <emildotchevski <at> gmail.com> writes:
On Thu, May 28, 2009 at 10:50 PM, <joaquin <at> tid.es> wrote:
> Why there's pressure from users to keep Boost libs header-only? > Boost is > for the users, so their reasons should be given proper weight. > One can not create good design (in general, not just in software) by asking the users what would they like.
I'd rather not go into discussing such far-reaching issues as design theory, but you haven't asked my question: What are the particular reasons why users demand header-only libraries?
Probably different users have different reasons. One reason might be that it makes Boost easier to install initially. What's your point?
My point is that if there's pressure from users to have header-only libs (as you and also I recognize) I think the least we can do is try to understand and analyze these reasons to see their merit. If pressure were the other way around (i.e. users demanding that code be moved out of .hpps as much as possible) we wouldn't be having this sort of discussions.
It it my understanding (based on actually supporting users on IRC and mailing lists for years), that the users who are mostly concerned about linking are either:
1. Users who just crossed the chasm between the two popular platforms, in either direction.
2. Users who don't understand the difference between headers and libraries in general, and don't know how to use their IDE.
3. Users who don't know C++.
Users in (1) group will adapt quickly. Users in groups (2) and (3) probably won't be able to use Boost effectively until they do some other learning.
I have consistently found that boost seems to cause more problems for developers than all the other libraries I use put together, in terms of building and linking.
I work with a large number of developers on users on many different OSes and processors, so we distribute as source. Some users use packaged copies of boost distributed with their system, other compile from source.
And, what exact problems are you having, and do those problems happen with 1.39 and have you reported them?
As a user, I can describe *my* reasons to favor header-only libs:
1. The whole bjam-driven building process is nontrivial and time and space consuming.
In 1.39, it is two commands in 1.39, on popular platforms. And processors are fast these days.
Is if ./configure && make?
No, it's "./bootstrap.sh" and "bjam".
3. Bulding libs selectively is not as easy as it might seem, due to the fact that interlib dependencies might force you to build libB when using libA, and you don't know in advance.
And you should not care. If you add --with-filesystem, then the system library will be built automatically.
Most users aren't going to go around looking for options, they will assume the default ones do the right thing.
The defaults indeed do the right thing. I was replying to a statement as if user *explicitly wants* to build specific subset of libraries, it has to do something non-trivial. If you don't want to do anyting nontrivial, all you need is: ./bootstrap.sh ./bjam Exactly that, without any arguments whatsoever. - Volodya
On Thu, May 28, 2009 at 11:39 PM, <joaquin@tid.es> wrote:
Emil Dotchevski escribió:
On Thu, May 28, 2009 at 11:15 PM, Joaquin M Lopez Munoz <joaquin@tid.es> wrote: My point is that if there's pressure from users to have header-only libs (as you and also I recognize) I think the least we can do is try to understand and analyze these reasons to see their merit. If pressure were the other way around (i.e. users demanding that code be moved out of .hpps as much as possible) we wouldn't be having this sort of discussions.
As a user, I can describe *my* reasons to favor header-only libs:
1. The whole bjam-driven building process is nontrivial and time and space consuming. 2. If autolinking is not available, picking up the right lib variant is not trivial. 3. Bulding libs selectively is not as easy as it might seem, due to the fact that interlib dependencies might force you to build libB when using libA, and you don't know in advance.
These are indicative of build system problems. It's true, you can work around build system inadequacies by avoiding building libraries.
4. I'm not concerned about ABI issues given that, to start with, no ABI compatibility guarantees are provided across Boost versions.
Sure, ABI compatibility is not the issue.
This is not to say that I'd like *every* lib to be header-only; but I'd say the benefits of moving to a link-based lib should be balanced against points 1-4.
Assume for a moment that 1-3 were solved at the build system level. What reasons do you have for putting code in headers? I can think of only two: - Code that must be there, e.g. templates - Performance reasons: 1) inlining functions and 2) not using pimpl in code that is provably critical (the price is greater physical coupling.) Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski escribió:
On Thu, May 28, 2009 at 11:39 PM, <joaquin@tid.es> wrote:
As a user, I can describe *my* reasons to favor header-only libs:
1. The whole bjam-driven building process is nontrivial and time and space consuming. 2. If autolinking is not available, picking up the right lib variant is not trivial. 3. Bulding libs selectively is not as easy as it might seem, due to the fact that interlib dependencies might force you to build libB when using libA, and you don't know in advance.
[...]
This is not to say that I'd like *every* lib to be header-only; but I'd say the benefits of moving to a link-based lib should be balanced against points 1-4.
Assume for a moment that 1-3 were solved at the build system level.
Assuming 1-3 were solved is tantamount to assuming that building libs is as painless as not having to build them. Under these conditions of course I'd have no reason to prefer one solution to the other.
What reasons do you have for putting code in headers? I can think of only two:
- Code that must be there, e.g. templates
- Performance reasons: 1) inlining functions and 2) not using pimpl in code that is provably critical (the price is greater physical coupling.)
These are points for a lib developer to evaluate, not the Boost user. Remember we were talking of reasons why users prefer header-only. Again, I've listed here my personal reasons (as a user). I think it could be instructive to try to gather more insight as to why our users prefer header-only libs, putting aside our own arguments as Boost authors. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
On Fri, May 29, 2009 at 12:28 AM, <joaquin@tid.es> wrote:
Emil Dotchevski escribió:
On Thu, May 28, 2009 at 11:39 PM, <joaquin@tid.es> wrote:
As a user, I can describe *my* reasons to favor header-only libs:
1. The whole bjam-driven building process is nontrivial and time and space consuming. 2. If autolinking is not available, picking up the right lib variant is not trivial. 3. Bulding libs selectively is not as easy as it might seem, due to the fact that interlib dependencies might force you to build libB when using libA, and you don't know in advance.
[...]
This is not to say that I'd like *every* lib to be header-only; but I'd say the benefits of moving to a link-based lib should be balanced against points 1-4.
Assume for a moment that 1-3 were solved at the build system level.
Assuming 1-3 were solved is tantamount to assuming that building libs is as painless as not having to build them. Under these conditions of course I'd have no reason to prefer one solution to the other.
One way to make building libs as painless as not having to build them is to use Boost Build itself to build your programs. It makes the presence of cpps/libs an implementation detail. In my own code repository, I have many libs and many cpp files and many header-only libs and I don't keep track which is which -- Boost Build figures it out for me. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
On 29 May 2009, at 08:36, Emil Dotchevski wrote:
On Fri, May 29, 2009 at 12:28 AM, <joaquin@tid.es> wrote:
Emil Dotchevski escribió:
On Thu, May 28, 2009 at 11:39 PM, <joaquin@tid.es> wrote:
As a user, I can describe *my* reasons to favor header-only libs:
1. The whole bjam-driven building process is nontrivial and time and space consuming. 2. If autolinking is not available, picking up the right lib variant is not trivial. 3. Bulding libs selectively is not as easy as it might seem, due to the fact that interlib dependencies might force you to build libB when using libA, and you don't know in advance.
[...]
This is not to say that I'd like *every* lib to be header-only; but I'd say the benefits of moving to a link-based lib should be balanced against points 1-4.
Assume for a moment that 1-3 were solved at the build system level.
Assuming 1-3 were solved is tantamount to assuming that building libs is as painless as not having to build them. Under these conditions of course I'd have no reason to prefer one solution to the other.
One way to make building libs as painless as not having to build them is to use Boost Build itself to build your programs. It makes the presence of cpps/libs an implementation detail. In my own code repository, I have many libs and many cpp files and many header-only libs and I don't keep track which is which -- Boost Build figures it out for me.
Expecting people to change their build system just for one library doesn't seem sensible! What if some other library decided we should use their specialised build system? Also, I don't believe boost build supports all the features of cmake I'm currently using anyway. Chris
On Fri, May 29, 2009 at 12:49 AM, Christopher Jefferson <chris@bubblescope.net> wrote:
On 29 May 2009, at 08:36, Emil Dotchevski wrote:
On Fri, May 29, 2009 at 12:28 AM, <joaquin@tid.es> wrote:
Emil Dotchevski escribió:
On Thu, May 28, 2009 at 11:39 PM, <joaquin@tid.es> wrote:
As a user, I can describe *my* reasons to favor header-only libs:
1. The whole bjam-driven building process is nontrivial and time and space consuming. 2. If autolinking is not available, picking up the right lib variant is not trivial. 3. Bulding libs selectively is not as easy as it might seem, due to the fact that interlib dependencies might force you to build libB when using libA, and you don't know in advance.
[...]
This is not to say that I'd like *every* lib to be header-only; but I'd say the benefits of moving to a link-based lib should be balanced against points 1-4.
Assume for a moment that 1-3 were solved at the build system level.
Assuming 1-3 were solved is tantamount to assuming that building libs is as painless as not having to build them. Under these conditions of course I'd have no reason to prefer one solution to the other.
One way to make building libs as painless as not having to build them is to use Boost Build itself to build your programs. It makes the presence of cpps/libs an implementation detail. In my own code repository, I have many libs and many cpp files and many header-only libs and I don't keep track which is which -- Boost Build figures it out for me.
Expecting people to change their build system just for one library doesn't seem sensible! What if some other library decided we should use their specialised build system?
I didn't point at Boost Build because it builds boost, but because it's a good build system that solves the 1-3 issues that were listed. All I'm saying is that if using Boost Filesystem (which requires building) is harder for you than using Boost Bind (which doesn't), then maybe you ought to be thinking about changing your build system. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski wrote On Friday, May 29, 2009 3:37 AM
One way to make building libs as painless as not having to build them is to use Boost Build itself to build your programs. It makes the presence of cpps/libs an implementation detail. In my own code repository, I have many libs and many cpp files and many header-only libs and I don't keep track which is which -- Boost Build figures it out for me.
That's simply not going to happen for most Boost users, so it's a non-starter. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
------- Original message -------
From: Stewart, Robert <Robert.Stewart@sig.com> To: boost@lists.boost.org Sent: 29.5.'09, 8:54
Emil Dotchevski wrote On Friday, May 29, 2009 3:37 AM
One way to make building libs as painless as not having to build them is to use Boost Build itself to build your programs. It makes the presence of cpps/libs an implementation detail. In my own code repository, I have many libs and many cpp files and many header-only libs and I don't keep track which is which -- Boost Build figures it out for me.
That's simply not going to happen for most Boost users, so it's a non-starter.
But boost.build can be used to copy only the headers of the requested libraries on install. And make bcp deprecated.
_____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com
On Fri, May 29, 2009 at 4:54 AM, Stewart, Robert <Robert.Stewart@sig.com> wrote:
Emil Dotchevski wrote On Friday, May 29, 2009 3:37 AM
One way to make building libs as painless as not having to build them is to use Boost Build itself to build your programs. It makes the presence of cpps/libs an implementation detail. In my own code repository, I have many libs and many cpp files and many header-only libs and I don't keep track which is which -- Boost Build figures it out for me.
That's simply not going to happen for most Boost users, so it's a non-starter.
What is not going to happen for most Boost users? To use a good build system? And the solution to the "my build system sucks, I don't want to build the libraries I use" problem is to put everything in headers? Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
Emil Dotchevski wrote: On Friday, May 29, 2009 8:32 AM
On Fri, May 29, 2009 at 4:54 AM, Stewart, Robert <Robert.Stewart@sig.com> wrote:
Emil Dotchevski wrote On Friday, May 29, 2009 3:37 AM
One way to make building libs as painless as not having to build them is to use Boost Build itself to build your programs. It makes the presence of cpps/libs an implementation detail. In my own code repository, I have many libs and many cpp files and many header-only libs and I don't keep track which is which -- Boost Build figures it out for me.
That's simply not going to happen for most Boost users, so it's a non-starter.
What is not going to happen for most Boost users? To use a good build system?
Folks are not going to change build systems to accommodate Boost. If making building Boost easier was one of many reasons to switch to using Boost.Build, then it may be enough to justify the switch. However, there are usually a great many reasons to select a build system and making Boost easier to build is not likely one of them.
And the solution to the "my build system sucks, I don't want to build the libraries I use" problem is to put everything in headers?
You have misstated the problem. It isn't, "my build system sucks," but, "Boost uses an unusual build approach that doesn't easily fit into other build systems and approaches." The solution to the problem you stated might well be use Boost.Build, but the solution to the other is to make building Boost less unusual and easier to fit into other systems. (Which, I think, is increasingly the case.) Boost does not play well in our corporate build system, so we have to finagle a public, shared build to which we all link (for the non-header-only libraries). Getting libraries built in the right place with a known name and incorporating their naming convention into our build system was non-trivial, but works, though things change with each release. This precludes individual teams and projects from using independent versions of Boost; the build system owner must build what is needed and share it. The good news is we make it work, and building more non-header libraries is hardly more difficult than building those we now do. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
Emil Dotchevski wrote:
On Fri, May 29, 2009 at 4:54 AM, Stewart, Robert <Robert.Stewart@sig.com> wrote:
Emil Dotchevski wrote On Friday, May 29, 2009 3:37 AM
One way to make building libs as painless as not having to build them is to use Boost Build itself to build your programs. It makes the presence of cpps/libs an implementation detail. In my own code repository, I have many libs and many cpp files and many header-only libs and I don't keep track which is which -- Boost Build figures it out for me. That's simply not going to happen for most Boost users, so it's a non-starter.
What is not going to happen for most Boost users? To use a good build system?
I'm sure that is not what Robert was implying - in fact quite the opposite. The user may already be using a good build system, just not Boost Build. --> Mika Heiskanen
On Fri, 2009-05-29 at 09:28 +0200, joaquin@tid.es wrote:
- Code that must be there, e.g. templates
- Performance reasons: 1) inlining functions and 2) not using pimpl in code that is provably critical (the price is greater physical coupling.)
These are points for a lib developer to evaluate, not the Boost user. Remember we were talking of reasons why users prefer header-only.
Again, I've listed here my personal reasons (as a user). I think it could be instructive to try to gather more insight as to why our users prefer header-only libs, putting aside our own arguments as Boost authors.
Well - here is input from one boost user. I think single issue yes/no vote referendums produce some remarkably silly answers, as do poorly selected sets of questions in surveys. I could easily reply to a survey like this: Do you want header only libs (no need to "build" boost)? yes Do you want shorter builds? Yes Do you want to be able to use "system" or otherwise separately built/installed shared libraries rather than deploying libraries with each product etc? Yes Do you want to deal with the versioning and compatibility problems that can sometimes arise with shared libraries? No etc... The above noisy results are a result of asking questions that have too much to do with implementation. I don't care how the thing works so long as it works well and is easy to use. I suspect that goes for most users. To me easy to use means (things irrelevant to this thread elided): easy to build easy to deploy If you expect users to answer *how* to do those things I think you are giving users to much credit and/or trying to pass the buck on a hard problem :-) Now, if you avoid the need to build, you can't get easier than that - so header only it is - right? And if you don't need to deploy anything but the users own code - then can't get easier tan that - done! Header only is the winner! As far as it goes, I think that is actually a pretty accurate answer. However, the problem is that I also want my projects to build in a reasonable time, I don't want, when I build a whole suite of projects all using a common library, to have to build a common library N times once per project). Surely it is clear that the header only approach ultimately does not scale? Surely it is also clear that shared lib compatibility, versioning etc is a problem that can and has been solved? My biggest build and deployment problems have nothing to do with boost. Boost Build's uniqueness is not (now that the out of the box defaults are good and the basic knobs to twiddle are better documented) a significant impediment for basic use in my opinion at least. Its unfortunate that so much work has had to be done (and a big thanks to those responsible) to make Boost object code libraries easily (relatively) usable and deployable on a very common platform that makes these things remarkably hard. Please don't give up now! Darryl.
On 29 May 2009, at 08:28, joaquin@tid.es wrote:
These are points for a lib developer to evaluate, not the Boost user. Remember we were talking of reasons why users prefer header-only.
Again, I've listed here my personal reasons (as a user). I think it could be instructive to try to gather more insight as to why our users prefer header-only libs, putting aside our own arguments as Boost authors.
I have looked through a number of interactions we have had with boost on my project's mailing list, to find exactly why we have been having problems with boost. I acknolwedge the current e-mail is extremely bias to UNIXy systems, I don't know about windows. The problems can be broadly split into two categories: 1) ./configure doesn't act "like configure" (as it's actually just a wrapper over bjam). This seems to have got much better of late, with the new ./bootstrap.sh making things much clearer. 2) compiling boost with a short command line is hard. If I want someone to compile using gmp and libz (for example) I tell them to drop "-lgmp -lz" on the end of the g++ command, or I hardwire '-lgmp -lz' into a tiny makefile. Assuming these two libraries are installed, that will cover 99% of systems I come across in common usage. Now lets imagine I want to include functional (a header-only library). #include <boost/functional.hpp> // Fails #include <boost-1_39/boost/functional.hpp> // Fails with bizarre errors So I have to add: -I/usr/local/include/boost-1_39 Which of course won't work if I then go to another computer, where boost isn't in /usr/local/include The problem is exacerbated by libraries, where I have to type (on my computer) -L/usr/local/lib/libboost_thread-xgcc40-mt-1_38.a This is little no chance of that line being valid on another system, and it's hard to describe to people how to generate it. In summation, I think the biggest problem I've had with boost is in the 'small simple problems' category. Two suggestions: 1) I've seen some systems provide a script which can be executed like: g++ my_file ` boostdefs --include --library ` Which produce the appropriate flags. Exactly how should a system should work is of course up for debate. 2) Why not provide a 'libboost' which includes all the libraries linked together. If they are dynamically linked this shouldn't create a large overhead. Chris
Christopher Jefferson wrote:
I have looked through a number of interactions we have had with boost on my project's mailing list, to find exactly why we have been having problems with boost. I acknolwedge the current e-mail is extremely bias to UNIXy systems, I don't know about windows.
The problems can be broadly split into two categories:
1) ./configure doesn't act "like configure" (as it's actually just a wrapper over bjam). This seems to have got much better of late, with the new ./bootstrap.sh making things much clearer.
2) compiling boost with a short command line is hard.
If I want someone to compile using gmp and libz (for example) I tell them to drop "-lgmp -lz" on the end of the g++ command, or I hardwire '-lgmp -lz' into a tiny makefile. Assuming these two libraries are installed, that will cover 99% of systems I come across in common usage.
Now lets imagine I want to include functional (a header-only library).
#include <boost/functional.hpp> // Fails #include <boost-1_39/boost/functional.hpp> // Fails with bizarre errors
So I have to add:
-I/usr/local/include/boost-1_39
Which of course won't work if I then go to another computer, where boost isn't in /usr/local/include
The problem is exacerbated by libraries, where I have to type (on my computer)
-L/usr/local/lib/libboost_thread-xgcc40-mt-1_38.a
This is little no chance of that line being valid on another system, and it's hard to describe to people how to generate it.
In summation, I think the biggest problem I've had with boost is in the 'small simple problems' category. Two suggestions:
1) I've seen some systems provide a script which can be executed like:
g++ my_file ` boostdefs --include --library `
Which produce the appropriate flags. Exactly how should a system should work is of course up for debate.
There were proposals to add pkg-config generation. However, pkg-config is fairly limited tool -- it does not even allow to return different compiler flags for static and shared libraries. So, pkg-config is only good if you have selected a single variant of Boost that you care about. But in that case, you might as well build with --layout=system and add -lboost_thread -- no decoration at all. The idea of using a custom 'boostdefs' tool has came up before, but again, it's not clear how useful it is when you build several variants. Say, if you use boostdefs --include --library thread --release --threading=single then your project will not build if single-threaded version of Boost is not installed, and we're back to 'how to make my project to build'. I actually believe that on Linux, using --layout=system by default is the right approach. On Windows, there are some reasons to build several variants with fancy naming, but there, autolink just handles everything.
2) Why not provide a 'libboost' which includes all the libraries linked together. If they are dynamically linked this shouldn't create a large overhead.
This is interesting idea -- and easily implementable on Linux. Anybody wishes to comment? I am not sure if this is possible/reasonable on Windows. - Volodya
On 3 Jun 2009, at 15:10, Vladimir Prus wrote:
In summation, I think the biggest problem I've had with boost is in the 'small simple problems' category. Two suggestions:
1) I've seen some systems provide a script which can be executed like:
g++ my_file ` boostdefs --include --library `
Which produce the appropriate flags. Exactly how should a system should work is of course up for debate.
There were proposals to add pkg-config generation. However, pkg- config is fairly limited tool ...
then your project will not build if single-threaded version of Boost is not installed, and we're back to 'how to make my project to build'. I actually believe that on Linux, using --layout=system by default is the right approach. On Windows, there are some reasons to build several variants with fancy naming, but there, autolink just handles everything.
I hadn't looked at --layout=system. That looks like exactly how I would like things to be. While some users might want more complicated layouts, I think making the default do the simple, obvious thing is a nice idea. Chris
2) Why not provide a 'libboost' which includes all the libraries linked together. If they are dynamically linked this shouldn't create a large overhead.
This is interesting idea -- and easily implementable on Linux. Anybody wishes to comment? I am not sure if this is possible/reasonable on Windows.
- Volodya
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Wednesday 03 June 2009 15:00:43 Christopher Jefferson wrote:
Now lets imagine I want to include functional (a header-only library).
#include <boost/functional.hpp> // Fails #include <boost-1_39/boost/functional.hpp> // Fails with bizarre errors
So I have to add:
-I/usr/local/include/boost-1_39
Which of course won't work if I then go to another computer, where boost isn't in /usr/local/include
I know this problem. A well-written Makefile should use variables like CFLAGS, CXXFLAGS, CPPFLAGS and LDFLAGS. You can then create a simple script export CXX=g++-4.4 export CC=gcc-4.4 export CPPFLAGS="-I /home/uli/include -I /home/uli/devel/Boost-trunk" export LDFLAGS="-L /home/uli/lib" which sets up the environment accordingly. This requires people to understand what these flags mean, but I don't think they can't learn that, and the knowledge is even useful elsewhere.
2) Why not provide a 'libboost' which includes all the libraries linked together. If they are dynamically linked this shouldn't create a large overhead.
Actually, this is an interesting new idea. At least, it would make several things simpler. Boost historically allows users to use parts of it without having the whole forced upon them, I guess some people see this as an important feature. However, the global boost-all library wouldn't prevent that, it would just be an alternative. I would create a TRAC ticket for that, at least that would serve as central collection point for the pros and cons of that design. Uli
On Wed, Jun 3, 2009 at 9:59 AM, Ulrich Eckhardt <doomster@knuut.de> wrote:
2) Why not provide a 'libboost' which includes all the libraries linked together. If they are dynamically linked this shouldn't create a large overhead.
Actually, this is an interesting new idea. At least, it would make several things simpler. Boost historically allows users to use parts of it without having the whole forced upon them, I guess some people see this as an important feature. However, the global boost-all library wouldn't prevent that, it would just be an alternative. I would create a TRAC ticket for that, at least that would serve as central collection point for the pros and cons of that design.
Read the subject line. I wouldn't call piling everything together modularization. :) Also, wouldn't this increase the size of the executable? I'm not an expert but I think that by default GCC doesn't remove unused code because a .so loaded later could try to link with it. This would put even more pressure on making libraries header-only. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
On 3 Jun 2009, at 18:10, Emil Dotchevski wrote:
On Wed, Jun 3, 2009 at 9:59 AM, Ulrich Eckhardt <doomster@knuut.de> wrote:
2) Why not provide a 'libboost' which includes all the libraries linked together. If they are dynamically linked this shouldn't create a large overhead.
Actually, this is an interesting new idea. At least, it would make several things simpler. Boost historically allows users to use parts of it without having the whole forced upon them, I guess some people see this as an important feature. However, the global boost-all library wouldn't prevent that, it would just be an alternative. I would create a TRAC ticket for that, at least that would serve as central collection point for the pros and cons of that design.
Read the subject line. I wouldn't call piling everything together modularization. :)
Also, wouldn't this increase the size of the executable? I'm not an expert but I think that by default GCC doesn't remove unused code because a .so loaded later could try to link with it. This would put even more pressure on making libraries header-only.
I'm certainly not suggesting that a separate libraries be removed. Any user who wants separate libraries for reduced space can use the existing ones. Chris
On 3 Jun 2009, at 17:59, Ulrich Eckhardt wrote:
On Wednesday 03 June 2009 15:00:43 Christopher Jefferson wrote:
Now lets imagine I want to include functional (a header-only library).
#include <boost/functional.hpp> // Fails #include <boost-1_39/boost/functional.hpp> // Fails with bizarre errors
So I have to add:
-I/usr/local/include/boost-1_39
Which of course won't work if I then go to another computer, where boost isn't in /usr/local/include
I know this problem. A well-written Makefile should use variables like CFLAGS, CXXFLAGS, CPPFLAGS and LDFLAGS. You can then create a simple script
export CXX=g++-4.4 export CC=gcc-4.4 export CPPFLAGS="-I /home/uli/include -I /home/uli/devel/Boost-trunk" export LDFLAGS="-L /home/uli/lib"
which sets up the environment accordingly. This requires people to understand what these flags mean, but I don't think they can't learn that, and the knowledge is even useful elsewhere.
That still doesn't solve the problem that the libraries' name is mangled strangely with the default build system (although not with -- layout=system it seems). Also, if I just want to transmit a file without a Makefile, for many libraries I still don't need even that. Chris
On Friday 29 May 2009 08:39:22 joaquin@tid.es wrote:
As a user, I can describe *my* reasons to favor header-only libs:
1. The whole bjam-driven building process is nontrivial and time and space consuming. 2. If autolinking is not available, picking up the right lib variant is not trivial. 3. Bulding libs selectively is not as easy as it might seem, due to the fact that interlib dependencies might force you to build libB when using libA, and you don't know in advance. 4. I'm not concerned about ABI issues given that, to start with, no ABI compatibility guarantees are provided across Boost versions.
This is not to say that I'd like *every* lib to be header-only; but I'd say the benefits of moving to a link-based lib should be balanced against points 1-4.
You can have your cake and eat it: https://svn.boost.org/trac/boost/ticket/3103 It requires one unusual thing, #including a .cpp file, but otherwise the usage is dead simple. Uli
Daniel James wrote On Thursday, May 28, 2009 5:32 PM
I'd prefer it if the non-header files were all in a single directory with a standard name. That way if I list the files in boost/filesystem there'll be less noise. And it'll make it easier to use the unix command 'find'. So;
boost/filesystem.hpp boost/filesystem/config.hpp boost/filesystem/lib/index.html boost/filesystem/lib/example/ boost/filesystem/lib/test/ boost/filesystem/lib/src/
Replacing 'lib' with whatever you fancy.
+1 _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
On Thu, May 28, 2009 at 5:31 PM, Daniel James <daniel_james@fmail.co.uk> wrote:
2009/5/28 Beman Dawes <bdawes@acm.org>:
On Thu, May 28, 2009 at 10:31 AM, Ilya Sokolov <ilyasokol@gmail.com> wrote: ...
I'd prefer it if the non-header files were all in a single directory with a standard name. That way if I list the files in boost/filesystem there'll be less noise. And it'll make it easier to use the unix command 'find'. So;
boost/filesystem.hpp boost/filesystem/config.hpp boost/filesystem/lib/index.html boost/filesystem/lib/example/ boost/filesystem/lib/test/ boost/filesystem/lib/src/
Replacing 'lib' with whatever you fancy.
Good idea. Yes, that would be a help.
Although, you can add me to the people who are currently happy enough with the status quo.
We certainly want to tread carefully. But we really do need to see if there is a new approach with enough benefits to warrant a change. --Beman
Beman Dawes wrote:
On Thu, May 28, 2009 at 10:31 AM, Ilya Sokolov <ilyasokol@gmail.com> wrote:
It doesn't make dependencies of header-only libraries explicit.
Dependencies are a separate issue. AFAICS, that's also true for the alternatives too.
No. If we had a build system target for each library that installs its headers to the 'boost' directory before building we would be forced to explicitly state the dependencies: (pseudocode) # Boost.SmartPtr jamfile synchronize sync_headers : [ glob-tree ../include/*.hpp ] : ../../boost ; lib smart_ptr : : <dependency>sync_headers ; # Boost.Filesystem jamfile lib filesystem : [ glob-tree ../src/*.cpp ] /boost//system /boost//smart_ptr # NOTE ... ;
Beman Dawes wrote:
I've started to put together a wiki page based on our ongoing Header Modularization discussions.
See http://svn.boost.org/trac/boost/wiki/HeaderModularization
I am afraid I still don't see clear, and detailed list of goals -- separate from implementation concerns and requirements. So far, the only clearly obvious points that require directory layout changes are: 1. Easier switch/merge (1 command instead of 2) 2. Easier deployment of prospective libraries... oh wait, a prospective library can just the same layout as Boost, and be unpacked on top of it. Everything else (like installing subset of Boost) can be handled by necessary build system changes, probably with additional metadata. So, are we in this just for 1? - Volodya
Beman Dawes wrote:
I've started to put together a wiki page based on our ongoing Header Modularization discussions.
See http://svn.boost.org/trac/boost/wiki/HeaderModularization
I listed eight approaches to meeting the Goals/Objectives/Needs/Wants, and was about to identify the weaknesses of each approach when I hit on approach nine:
Move (permanently) each library's root/libs/libname contents into root/boost/libname for the library.
In other words, instead of moving the boost/... tree headers to the libs/... tree, merge the libs/... tree into the boost/... tree.
For example, Boost.Filesystem is currently organized like this:
root/ boost/ filesystem.hpp filesystem/ config.hpp convenience.hpp exception.hpp fstream.hpp operations.hpp path.hpp libs/ filesystem/ build/ ... CMakeLists.txt doc/ ... example/ ... index.html ... module.cmake src/ ... test/ ...
After the reorganization, filesystem would be organized like this:
root/ boost/ filesystem.hpp filesystem/ config.hpp convenience.hpp exception.hpp fstream.hpp operations.hpp path.hpp build/ ... CMakeLists.txt doc/ ... example/ ... index.html ... module.cmake src/ ... test/ ...
In theory, the root/libs/... tree isn't needed anymore. In practice we might want to keep a root/libs/libname/index.html entry for each library so docs links don't break.
Apologies if someone already suggested this approach. It is so simple I can't believe someone didn't already suggest it.
Unless I'm missing something, this is the only approach that appears to do fairly well for all of the Goals/Objectives/Needs/Wants, and very well for most of them. It would require some rework of scripts and other setups that depend on the current organization of libs/..., but that seems minor compared to the long-term benefits.
Comments?
This is on the right track!!! Here's my incremental 2 cents root/ boost/ filesystem.hpp filesystem/ config.hpp convenience.hpp - uh-oh looks like this is taken care of by ../filesystem.hpp exception.hpp fstream.hpp operations.hpp path.hpp index.html - link to doc/html/index.html bjam/ build/ Jamfile.v2 test/ Jamfile.v2 cmake/ build/ module.cmake test/ ?.... vc7ide/ - or whatever build/ solutions and projects test/ solutions and projects doc/ html/ index.html ... - maybe generated from docbook pdf/ ?. pdf - maybe gerated from docbook ... docbook/ ?.xml - maybe generated by qbk .... qbk/ ... libs/ - was src filesystem.cpp .... binaries/ libboost..... boost.... / whatever ... bjam/ Jamfile.v2 - to build library results.html - build by bjam build cmake/ cmake.module vc7ide/ - or whatever solutions and projects test/ test_filesystem.cpp .... binaries/ test_filesystem.exe ... bjam/ Jamfile.v2 - to build library results.html - build by bjam build cmake/ cmake.module vc7ide/ - or whatever solutions and projects example/ example_filesystem.cpp .... binaries/ example_filesystem.exe ... bjam/ Jamfile.v2 - to build and run examples results.html - build by bjam build cmake/ cmake.module vc7ide/ - or whatever performance/ or whatever example_filesystem.cpp .... binaries/ example_filesystem.exe ... bjam/ Jamfile.v2 - to build library results.html - build by bjam build cmake/ cmake.module vc7ide/ - or whatever Other points There should be NO files in root/boost. All these should be moved to root/boost/utilities. root/boost/utilities would ahve the same structure as if it were any other library. Basically it is "catch all" library for miscelaneous components that are "too small" to rate a whole tree. To build/run any section - e.g. performance with bjam build cd root/boost/filesystem/performance/bjam bjam_build.sh or bjam_build.bat or whatever #generates results.html Building anything would invoke builds on it's prerequistes Building all libaries would just walk all the libraries. Any thing already update wouldn't be repeated. I'm not crazy about using fancy features which are coupled with some particular tool. I'm specifically referring to using things like svnexternals hardlinks to "hide" some "real" directory structure behind some "public" one. and other assorted tricks. Such "tricks" seem appealing at the beginning. but they tend to make things hard to understand when things go (inevitably) wrong. That's my 2 cents. BTW - I'm very happy with the current SVN setup. It has worked extremely well for me. I would be extremely reluctant to abandon somethng that has worked as advertised.
--Beman
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
participants (18)
-
Beman Dawes -
Bjørn Roald -
Christopher Jefferson -
Daniel James -
Darryl Green -
David Abrahams -
Emil Dotchevski -
Felipe Magno de Almeida -
Ilya Sokolov -
Joaquin M Lopez Munoz -
joaquin＠tid.es
-
Lars Viklund -
Mika Heiskanen -
Robert Ramey -
Stewart, Robert -
troy d. straszheim -
Ulrich Eckhardt -
Vladimir Prus