To split, or not to split, or something else? RE: type_traits rewrite, modularization, dependencies, etc.
I keep reading emails about the effort to reduce dependencies, and to split libraries into core/simple + whole/big.. And I keep thinking it's making things more complex than they need to be. I can see the need to compartmentalize the parts of a library that have minimal to little dependencies. So here's a suggestions for structure of managing this. I see options: 1. In each library allow for a "core" set of headers that are still in a core subdir. But that we can tell is a subset of the library that has the minimal dependencies. For example we would have: * libs/<lib>/include/boost/<lib>/core/.. * libs/<lib>/include/boost/<lib>/<the-rest>.. This of course assumes that it's headers only that would be core. Second option is to have each library have to explicit sublibs/parts. "core" and "full". Structure would be: * libs/<lib>/core/(build, doc, include, meta, test) * libs/<lib>/full/(build, doc, include, meta, test) Note, both core and full would be in the same git repo to make life easier for library authors and users that visit a library git repo on github. As it's only one place for everything for that one library. As a user, release manager, testing manager, and library author this would be easy to understand and manage than the current "some random set of libraries have core or not and in different places". -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
Rene Rivera wrote:
I keep reading emails about the effort to reduce dependencies, and to split libraries into core/simple + whole/big..
We need to keep this into perspective. There's much e-mail traffic but out of 118 or so libraries we want to split 2.5 into core+non-core -- serialization, range, perhaps mpl. We also want to split hash out of functional, and common_type out of type_traits, and maybe fpclassify/sign out of math in the future, but those aren't core/non-core splits. So standardizing core/non-core splits would not be a priority for me. To get a rough idea what we're trying to accomplish, look at the bold parts of http://www.pdimov.com/tmp/report-develop-c3bb6eb/module-weights.html and see what tends to predominate - it's the same few libraries over and over. This is not a general problem.
Peter Dimov-2 wrote
and see what tends to predominate - it's the same few libraries over and over. This is not a general problem.
I agree. That is I see it as mostly as problem specific to the serialization library as an optional component that other libraries provide access to. Our dependency concept (and our headers) don't really capture the "optional" aspect to this. In any case, I think a solution more or less specific to the serialization library would suffice. Of course I don't want something overly disruptive since I'm the one who has to deal with it. Also I don't want something overly general that everyone has to deal with even though the shouldn't have to. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/To-split-or-not-to-split-or-something-els... Sent from the Boost - Dev mailing list archive at Nabble.com.
On 3 February 2015 at 10:49, Peter Dimov
Rene Rivera wrote:
I keep reading emails about the effort to reduce dependencies, and to
split libraries into core/simple + whole/big..
We need to keep this into perspective. There's much e-mail traffic but out of 118 or so libraries we want to split 2.5 into core+non-core -- serialization, range, perhaps mpl. We also want to split hash out of functional, and common_type out of type_traits, and maybe fpclassify/sign out of math in the future, but those aren't core/non-core splits. So standardizing core/non-core splits would not be a priority for me.
To get a rough idea what we're trying to accomplish, look at the bold parts of
http://www.pdimov.com/tmp/report-develop-c3bb6eb/module-weights.html
and see what tends to predominate - it's the same few libraries over and over. This is not a general problem.
I'm happy to solve the Boost.Range dependencies. I suspect this is far simpler than for Boost.Serialization. I think code simply needs to move to other libraries. For example the Range Adapter for regexp could move to Boost.RegExp if it is reasonable for it to depend on Boost.Range. I've publicly stated previously that code that is in Boost.Range would be better elsewhere. The problem has been that as a maintainer the access to other libraries has been limited. I think there is a pattern to the libraries that hit the top of the list. They are very generic libraries that interoperate and extend other Boost libraries. I think we had chosen to make optional dependencies based upon header inclusion, and this was a valid decision at the time. I also understand that the direction of C++ modules makes it appear that it would be prudent to solve these dependencies in another manner, so that we are ready to adopt them rather than being forced to start the deprecation process very later and delay the adoption of modules. We can't be sure we are exactly correct, but the general direction for modules is clear enough to motivate action now. I have no resistance to assisting with this effort. I'd like to ensure my users get proper time to adjust via the proper deprecation procedure, but I'm open to making changes. Please feel free to enlist my help. Regards, Neil Groves
Neil Groves wrote:
I also understand that the direction of C++ modules makes it appear that it would be prudent to solve these dependencies in another manner, so that we are ready to adopt them rather than being forced to start the deprecation process very later and delay the adoption of modules. We can't be sure we are exactly correct, but the general direction for modules is clear enough to motivate action now.
Please feel free to enlist my help.
You might want to tell Robert that. Thanks, Steve.
Neil Groves wrote:
I'm happy to solve the Boost.Range dependencies. I suspect this is far simpler than for Boost.Serialization. I think code simply needs to move to other libraries. For example the Range Adapter for regexp could move to Boost.RegExp if it is reasonable for it to depend on Boost.Range.
The most straightforward way to proceed with Boost.Range is to split (approximately) everything in boost/range.hpp into a range_core module. I don't remember Range's history very clearly, but I suspect that this is what Range originally was, before acquiring the additional bells and whistles. Range's reverse dependencies: http://www.pdimov.com/tmp/report-develop-c3bb6eb/range.html#reverse-dependen... show that most libraries only depend on this 'core' part. Isolating it precisely would take some additional work though, as f.ex. as_literal.hpp is not included in boost/range.hpp, but does seem part of the core.
On Tue, Feb 3, 2015 at 4:49 AM, Peter Dimov
Rene Rivera wrote:
I keep reading emails about the effort to reduce dependencies, and to
split libraries into core/simple + whole/big..
We need to keep this into perspective. There's much e-mail traffic but out of 118 or so libraries we want to split 2.5 into core+non-core -- serialization, range, perhaps mpl. We also want to split hash out of functional, and common_type out of type_traits, and maybe fpclassify/sign out of math in the future, but those aren't core/non-core splits. So standardizing core/non-core splits would not be a priority for me.
I guess I'm more of a stickler about structure then :-) I look at the (small) special cases in build scripts and I cringe. I also think that if it's a problem now, it will continue to be a problem in the future for new libraries. So it's better to have an agreed upon structure (and documentation) that we can point new authors to. This is al optional anyway. It's just that I would prefer to have *one* option instead of *N* options for each library (even if N is small). -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
Rene Rivera wrote:
I guess I'm more of a stickler about structure then :-) I look at the (small) special cases in build scripts and I cringe. I also think that if it's a problem now, it will continue to be a problem in the future for new libraries. So it's better to have an agreed upon structure (and documentation) that we can point new authors to. This is al optional anyway. It's just that I would prefer to have *one* option instead of *N* options for each library (even if N is small).
I prefer zero options. :-) I especially disprefer nested options, that is, libs/X and libs/X/Y. But ideally, it should all be libs/X, no options at all.
Rene Rivera-2 wrote
I guess I'm more of a stickler about structure then :-) I look at the (small) special cases in build scripts and I cringe. I also think that if it's a problem now, it will continue to be a problem in the future for new libraries. So it's better to have an agreed upon structure (and documentation) that we can point new authors to. This is al optional anyway. It's just that I would prefer to have *one* option instead of *N* options for each library (even if N is small).
Thinking out loud using the serialization library and date_time as a problematic test case. <code> date_time date_time/gregorian_serialize serialization/split_free serialization/serialization strong_typedef pfto access serialization/nvp ... /similar to the above but more complex </code> The short version is that including anything from boost/serialization shouldn't trigger anything from boost/archive. boost/archive is where all the havoc begins in creating a deep dependency. So someone might propose the following: a) make boost/serialization and boost/archive separate modules b) move a couple of things from the serialization library (where they don't belong) such as optional.hpp to the appropriate library. These two things would make a noticeable impact on the current dependency reports. This would mean that the module structure is still "regular" so it wouldn't have to affect your testing scripts. a) looks to be possible - but not trivial. I would raise questions about which module should contain which tests etc. A whole new round of horsing around with bjam scripts - which I really, really, really hate doing. Although when I made the library I created this division - mainly to keep the thing from turning into a huge ball of wax - this was not appreciated at the time. I don't know how many - if any - secret connections I may have made over the last 12 years. I also don't know if any such secret connections are unavoidable. This would also require some extra documentation explaining the rationale behind all this. On the upside, this document would be couched in general terms so that it would a pattern for future libraries which touch upon this problem. Basically, using this as a pattern would be the general solution and your testing setup wouldn't have any (hopefully) special cases. So far no one has proposed this. Which is OK from my perspective. I also would like to see explored (maybe on a different thread) J Maddocks observation regarding "bridge libraries" and my "bridge header" idea. Maybe it's not necessary if other libraries don't import anything from the "archive" side of the library. Maybe the boost/archive should be "private" in that it would be prohibited form being included by any other library but reserved for applications. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/To-split-or-not-to-split-or-something-els... Sent from the Boost - Dev mailing list archive at Nabble.com.
Robert Ramey wrote:
a) make boost/serialization and boost/archive separate modules ... So far no one has proposed this.
I have. I'd only move _some_ of boost/serialization into a new module and not all of it, but the principle is generally the same. The part that you need to make a type serializable in module A, everything else in module B. (One could also retire pfto.hpp while one's at it, but that's a side issue.)
On Feb 3, 2015, at 10:49 AM, Peter Dimov
wrote: http://www.pdimov.com/tmp/report-develop-c3bb6eb/module-weights.html
Just curious, which tool generates th html report?
Ahmed Charles wrote:
On Feb 3, 2015, at 10:49 AM, Peter Dimov
wrote: http://www.pdimov.com/tmp/report-develop-c3bb6eb/module-weights.html
Just curious, which tool generates th html report?
tools/boostdep/examples/report.bat.
participants (6)
-
Ahmed Charles
-
Neil Groves
-
Peter Dimov
-
Rene Rivera
-
Robert Ramey
-
Stephen Kelly