The latest dependency report shows chrono as having a dependency on math. See http://www.pdimov.com/tmp/report-develop-3b5a9f4/module-overview.html math is really big. In testing bpm, an install of timer (which depends on chrono) slowed noticeably when it got to math. Presumably the build is also much slower, although I haven't tested that yet. Is the math dependency really needed? Thanks, --Beman
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Beman Dawes Sent: 09 January 2015 15:14 To: Boost Developers List Subject: [boost] [chrono] Dependency on math?
The latest dependency report shows chrono as having a dependency on math.
See http://www.pdimov.com/tmp/report-develop-3b5a9f4/module- overview.html
Apologies for this naive outsider question, but could the dependency graph not be a lot shallower if packages are defined less greedy. As I understand it, if library X depends on library Y and Y depends on Z, then X is said to depend on Z. I can understand as packages are created based on the library dependency graph: [ To package X we must package all libraries on which X depends.] However, the reality is that the file-dependencies of X do not necessarily include Z. Why would you not package only the files upon which the desired library depends? I suppose the idea of distributing partial libraries is not very attractive. But how about distinguishing the packaging of libraries with and without their dependencies. [ To package X "with dependencies" we must package all libraries "without dependencies" on which it depends in terms of file the file-dependency graph (not the library dependency graph).] In this scenario the inclusion of a single Math header in Chrono would not be as disastrous. The alternatives seem to be to duplicate the code, or to use some judgement to split Math into Math.Core and Math.More. It also seems more in line with how you would treat external dependencies (only package zlib if zlib is being used), or not? Beman's question of "with test" and "without tests" seems to be in the same direction, but different. Please enlighten me if this is an old and passed topic. Thanks, Alex
alex wrote:
As I understand it, if library X depends on library Y and Y depends on Z, then X is said to depend on Z. I can understand as packages are created based on the library dependency graph:
[ To package X we must package all libraries on which X depends.]
However, the reality is that the file-dependencies of X do not necessarily include Z. Why would you not package only the files upon which the desired library depends?
Because the package for X only contains X's files, not its dependencies. If X depends on a file from Y, then Y's package must be installed to get Y's file. And since Y is now installed, if it requires building, Z needs to be installed as well; but even if it doesn't require building, having Y installed but unusable is not good from usability point of view.
Peter Dimov wrote
Sent: 09 January 2015 16:42 To: boost@lists.boost.org Subject: Re: [boost] [chrono] Dependency on math?
alex wrote:
As I understand it, if library X depends on library Y and Y depends on Z, then X is said to depend on Z. I can understand as packages are created based on the library dependency graph:
[ To package X we must package all libraries on which X depends.]
However, the reality is that the file-dependencies of X do not necessarily include Z. Why would you not package only the files upon which the desired library depends?
Because the package for X only contains X's files, not its dependencies. If X depends on a file from Y, then Y's package must be installed to get Y's file. And since Y is now installed, if it requires building, Z needs to be installed as well; but even if it doesn't require building, having Y installed but unusable is not good from usability point of view.
Thanks for clarifying. I must say I am not convinced by the usability argument. Having just the files of Y and not the build makes library X usable. And X is what the user asked for. So, from a usability point of view it is perfect. Y must not be build, but there are good arguments for Y to be build. However there are also counter arguments as there clearly is an unhealthy incentive now to fragment libraries, to duplicate code and to fix things that are not broken.
On January 9, 2015 5:43:07 PM EST, alex <alexhighviz@hotmail.com> wrote:
Peter Dimov wrote
Sent: 09 January 2015 16:42 To: boost@lists.boost.org Subject: Re: [boost] [chrono] Dependency on math?
alex wrote:
As I understand it, if library X depends on library Y and Y depends on Z, then X is said to depend on Z. I can understand as packages are created based on the library dependency graph:
[ To package X we must package all libraries on which X depends.]
However, the reality is that the file-dependencies of X do not necessarily include Z. Why would you not package only the files upon which the desired library depends?
Because the package for X only contains X's files, not its dependencies. If X depends on a file from Y, then Y's package must be installed to get Y's file. And since Y is now installed, if it requires building, Z needs to be installed as well; but even if it doesn't require building, having Y installed but unusable is not good from usability point of view.
Thanks for clarifying. I must say I am not convinced by the usability argument. Having just the files of Y and not the build makes library X usable. And X is what the user asked for. So, from a usability point of view it is perfect.
How do you know Y is usable at that point? To get what you want, a packaging tool has to be given much more information. If Y depends upon a header from Z, how does the tool know whether that Z header implies the need to build Z? Without that, the tool must assume that Z will need to be built. The same applies to the Y headers upon which X depends so Y must be installed ready to build. If the tool knows that a library is header only, then it could be more selective about which files it installs. However, that leads to complicated dependency tracking of the installed files. If the user gets only a partial Y and a partial Z to satisfy X, and the user asks to install Y, the tool must install the rest of Y. But if the user then uninstalls Y, the tool must leave the parts of Y that were needed originally for X whilst removing the rest of Y. Since users can alter the contents of the installation directories, the tool must actually track all files it installs in order to be able to reinstall missing files to correct user mistakes, too. ___ Rob (Sent from my portable computation engine)
Thanks for clarifying. I must say I am not convinced by the usability argument. Having just the files of Y and not the build makes library X usable. And X is what the user asked for. So, from a usability point of view it is perfect.
How do you know Y is usable at that point? To get what you want, a packaging tool has to be given much more information. If Y depends upon a header from Z, how does the tool know whether that Z header implies the need to build Z? Without that, the tool must assume that Z will need to be built. The same applies to the Y headers upon which X depends so Y must be installed ready to build.
Is it really that much more information that needs to be given? If you would just tell the package manager which header files require building and which not, would that not enable it to remove many dependencies based on the assumption that all headers require building. If such information were absent you could always revert to the original assumption that everything needs to be build.
If the tool knows that a library is header only,
selective about which files it installs. However, that leads to complicated dependency tracking of the installed files. If the user gets only a
Well, large parts of BOOST are header-only and libraries that are not header-only often only rely on being built for some specific functionality. then it could be more partial Y It would be very complicated if you wanted to optimize and really copy only the files that are essential. However I suggest to only get complete libraries, but also only get libraries that are actually depended (based on recursive file dependencies) on and not all libraries that are depended on based on recursive library dependencies.
and a partial Z to satisfy X, and the user asks to install Y, the tool must install the rest of Y. But if the user then uninstalls Y, the tool must leave the parts of Y that were needed originally for X whilst removing the rest of Y.
That is why I thought it would be good to always get all files of a library, even if not all files are needed.
Since users can alter the contents of the installation directories, the tool must actually track all files it installs in order to be able to reinstall missing files to correct user mistakes, too.
Rob
On January 10, 2015 11:04:29 AM EST, alex <alexhighviz@hotmail.com> wrote:
To get what you want, a packaging tool has to be given much more information. If Y depends
header from Z, how does the tool know whether that Z header implies
upon a the
need to build Z? Without that, the tool must assume that Z will need to be built. The same applies to the Y headers upon which X depends so Y must be installed ready to build.
Is it really that much more information that needs to be given? If you would just tell the package manager which header files require building and which not, would that not enable it to remove many dependencies based on the assumption that all headers require building. If such information were absent you could always revert to the original assumption that everything needs to be build.
There must be information on each file, at least if the library isn't marked header only. Both pieces of information can become stale after just one maintenance change, so it is brittle. There's also the question of where to store that information. Is it per host? per user?
If the tool knows that a library is header only,
Well, large parts of BOOST are header-only and libraries that are not header-only often only rely on being built for some specific functionality.
selective about which files it installs. However, that leads to complicated dependency tracking of the installed files. If the user gets only a
then it could be more partial Y
It would be very complicated if you wanted to optimize and really copy only the files that are essential. However I suggest to only get complete libraries, but also only get libraries that are actually depended (based on recursive file dependencies) on and not all libraries that are depended on based on recursive library dependencies.
and a partial Z to satisfy X, and the user asks to install Y, the tool must install the rest of Y. But if the user then uninstalls Y, the tool must leave the parts of Y that were needed originally for X whilst removing the rest of Y.
That is why I thought it would be good to always get all files of a library, even if not all files are needed.
I was trying to imply that, indirectly, but the tool will need to be smart enough to know when the user expressly wanted Y versus needing Y because of X.
Since users can alter the contents of the installation directories, the tool must actually track all files it installs in order to be able to reinstall missing files to correct user mistakes, too.
That would still be an issue. ___ Rob (Sent from my portable computation engine)
If you would just tell the package manager which header files require building and which not....
There must be information on each file, at least if the library isn't
marked
header only. Both pieces of information can become stale after just one maintenance change, so it is brittle.
Yes you would need information for each file, but you could have defaults for the library or directories. Would you call a code change that introduces a requirement to build 'maintenance' ? It would be a breaking change for users that did not build before. A code change that removes a requirement to build but not marked as such would not cause a problem; it just means that sometimes the library would be built superfluously.
There's also the question of where to store that information. Is it per host? per user?
Isn't it obvious that it should be per host? The user shouldn't be expected to know?
That is why I thought it would be good to always get all files of a library, even if not all files are needed.
I was trying to imply that, indirectly, but the tool will need to be smart enough to know when the user expressly wanted Y versus needing Y because of X.
Isn't the whole idea that the users express what they want and then get the package containing what they need?
Since users can alter the contents of the installation directories, the tool must actually track all files it installs in order to be able to reinstall missing files to correct user mistakes, too.
That would still be an issue.
From you responses I gather that resolving dependencies at file level would cause too many complications for built libraries but not so for header-only
Yes, but very solvable and I would think it already needs to do this anyway. libraries. Why not at least take advantage of the header-only status of a large number of libraries? I suppose this would create an incentive to split existing libraries into a built and header-only library, but that is not so bad is it?
On January 10, 2015 4:43:20 PM EST, alex <alexhighviz@hotmail.com> wrote:
If you would just tell the package manager which header files require building and which not....
There must be information on each file, at least if the library isn't marked header only. Both pieces of information can become stale after just one maintenance change, so it is brittle.
Yes you would need information for each file, but you could have defaults for the library or directories.
Would you call a code change that introduces a requirement to build 'maintenance' ? It would be a breaking change for users that did not build before.
If the library was header only and now must be built, that's a breaking change and might make the maintainer recall the need to change the metadata for the library. Detecting the missed metadata change will be relative easy: link failures well occur if someone installs and use that package. There's no way to automate checking that unless test runners is the packaging tool for each library under test. Changing one file from not requiring building a library to requiring that it be built, for a library that already needed to be built is a likely failure case. The maintainer is likely to forget to change the metadata for that file and detecting its omission will be difficult.
A code change that removes a requirement to build but not marked as such would not cause a problem; it just means that sometimes the library would be built superfluously.
Sure
There's also the question of where to store that information. Is it per host? per user?
Isn't it obvious that it should be per host? The user shouldn't be expected to know?
That is not obvious. Users can install Boost on a host that already has another version installed by an administrator, for example. A user might install more than one version.
That is why I thought it would be good to always get all files of a library, even if not all files are needed.
I was trying to imply that, indirectly, but the tool will need to be smart enough to know when the user expressly wanted Y versus needing Y because of X.
Isn't the whole idea that the users express what they want and then get the package containing what they need?
Yes, but you're not thinking about users changing their minds, it seems.
Since users can alter the contents of the installation directories, the tool must actually track all files it installs in order to be able to reinstall missing files to correct user mistakes, too.
That would still be an issue.
Yes, but very solvable and I would think it already needs to do this anyway.
That implies an extra directory associated with the tree to hold that information. That data can be stale relative to the filesystem which implies an integrity checker and a healing mode, etc. These are all complications which Peter, or even Boost, may not wish to undertake. However, if you're volunteering, I'm sure Peter would welcome your help.
From you responses I gather that resolving dependencies at file level would cause too many complications for built libraries but not so for header-only libraries. Why not at least take advantage of the header-only status of a large number of libraries? I suppose this would create an incentive to split existing libraries into a built and header-only library, but that is not so bad is it?
That's a nice first step, but even then you're relying on library maintainers to remember yet another thing to update, and that for an infrequent state change. ___ Rob (Sent from my portable computation engine)
From your responses I gather that resolving dependencies at file level would cause too many complications for built libraries but not so for header-only libraries. Why not at least take advantage of the header-only status of a large number of libraries? I suppose this would create an incentive to split existing libraries into a built and header-only library, but that is not so bad is it?
That's a nice first step, but even then you're relying on library maintainers to remember yet another thing to update, and that for an infrequent state change. ___ Rob
Thanks for clearing it up for me. My conclusion is that there is scope for reducing dependencies by being more selective in deciding what libraries to install. However, there are minor complications for header-only libraries and major complications for built libraries. I find this relevant because some of the current efforts to reduce dependencies seem problematic too.
The latest dependency report shows chrono as having a dependency on math.
See http://www.pdimov.com/tmp/report-develop-3b5a9f4/module-overview.html
math is really big. In testing bpm, an install of timer (which depends on chrono) slowed noticeably when it got to math. Presumably the build is also much slower, although I haven't tested that yet.
Is the math dependency really needed?
Longer term the better solution would be to repackage Math as several smaller packages. However, Paul and I have worked hard to try to make a wide range of functionality part of a coherent whole, and I would hate to loose that. Which is my plea to not just hack it apart in a hurry ;-) Of course since we haven't had any kind of package management until now, this has basically been a non-issue up till now. John.
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of John Maddock Sent: 09 January 2015 17:25 To: boost@lists.boost.org Subject: Re: [boost] [chrono] Dependency on math?
The latest dependency report shows chrono as having a dependency on math.
See http://www.pdimov.com/tmp/report-develop-3b5a9f4/module-overview.html
math is really big. In testing bpm, an install of timer (which depends on chrono) slowed noticeably when it got to math. Presumably the build is also much slower, although I haven't tested that yet.
Is the math dependency really needed?
Longer term the better solution would be to repackage Math as several smaller packages. However, Paul and I have worked hard to try to make a wide range of functionality part of a coherent whole, and I would hate to loose that. Which is my plea to not just hack it apart in a hurry ;-)
+1 It isn't a matter of just moving some /include files about. There is the weightier matter of docs to reorganize which is probably an order of magnitude more work, even with the good tools we have. So I can see some logic, but fear the cost. Whatever, it is important to get any reorganization right first time (second try over my dead body!) And be sure that the benefits outweigh the cost. Is the package manager time really important? Once installed, only takes up disk space? For better or for worse, Boost.Math is more than half of Boost by volume/mass/weight/LOC (if not all has general utility). Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830
Beman Dawes wrote:
The latest dependency report shows chrono as having a dependency on math.
chrono's dependency on math is only for common_factor_rt.hpp: http://www.pdimov.com/tmp/report-develop-3b5a9f4/chrono.html#math common_factor_rt.hpp and the accompanying _ct.hpp appear to belong to Integer. This actually seems a low-hanging fruit; these files are even authored by Daryle Walker et al, and Daryle is, I believe, the original author of Integer. Not clear how they ended up in Math.
The latest dependency report shows chrono as having a dependency on math.
chrono's dependency on math is only for common_factor_rt.hpp:
http://www.pdimov.com/tmp/report-develop-3b5a9f4/chrono.html#math
common_factor_rt.hpp and the accompanying _ct.hpp appear to belong to Integer. This actually seems a low-hanging fruit; these files are even authored by Daryle Walker et al, and Daryle is, I believe, the original author of Integer. Not clear how they ended up in Math.
They were there already when Paul and I staged a coup ;) But yes they could be moved into integer relatively easily. John.
On Fri, Jan 9, 2015 at 1:53 PM, John Maddock <boost.regex@virgin.net> wrote:
The latest dependency report shows chrono as having a dependency on math.
chrono's dependency on math is only for common_factor_rt.hpp:
http://www.pdimov.com/tmp/report-develop-3b5a9f4/chrono.html#math
common_factor_rt.hpp and the accompanying _ct.hpp appear to belong to Integer. This actually seems a low-hanging fruit; these files are even authored by Daryle Walker et al, and Daryle is, I believe, the original author of Integer. Not clear how they ended up in Math.
They were there already when Paul and I staged a coup ;)
But yes they could be moved into integer relatively easily.
While I certainly don't want you and Paul to "hack [math] apart in a hurry", moving those two headers to integer would seem to make more sense than asking Vicente to duplicate code in chrono. --Beman
Le 09/01/15 20:03, Beman Dawes a écrit :
On Fri, Jan 9, 2015 at 1:53 PM, John Maddock <boost.regex@virgin.net> wrote:
The latest dependency report shows chrono as having a dependency on math.
chrono's dependency on math is only for common_factor_rt.hpp:
http://www.pdimov.com/tmp/report-develop-3b5a9f4/chrono.html#math
common_factor_rt.hpp and the accompanying _ct.hpp appear to belong to Integer. This actually seems a low-hanging fruit; these files are even authored by Daryle Walker et al, and Daryle is, I believe, the original author of Integer. Not clear how they ended up in Math.
They were there already when Paul and I staged a coup ;)
But yes they could be moved into integer relatively easily. While I certainly don't want you and Paul to "hack [math] apart in a hurry", moving those two headers to integer would seem to make more sense than asking Vicente to duplicate code in chrono.
There is a suplementaty issue. The dependencies of Chrono on Math wouldn't be broken as Chrono -> Ratio -> Rational -> Math I would like to broke the Ratio -> Rational dependency (but this wouldn't be backward compatible). I think I added too much to Ratio, and that the Ratio -> rational conversion (in the same line than integral_constant -> Integral conversion) should be available via a free function (which obviously could not be implicit). In this case I could add the function in a Ratio.RationalConversion submodule (in the sens of a directory). I have this on my todo list, but I had not found yet the time to discuss the possibility of this breaking change. If you think this could be a good thing I could see when I can do it. Best, Vicente
Le 09/01/15 16:13, Beman Dawes a écrit :
The latest dependency report shows chrono as having a dependency on math.
See http://www.pdimov.com/tmp/report-develop-3b5a9f4/module-overview.html
math is really big. In testing bpm, an install of timer (which depends on chrono) slowed noticeably when it got to math. Presumably the build is also much slower, although I haven't tested that yet.
Is the math dependency really needed?
The dependency is on |<boost/math/common_factor_rt.hpp>| * from |<boost/chrono/io/duration_get.hpp>| * from |<boost/chrono/io_v1/chrono_io.hpp>| Of course, I can duplicate this function for my specific case, but I think this is not a good approach. I could suggest to move this file to integer which is at level 2 :) Reducing dependencies could improve compilation times and it could surely be the case here as chrono needs only a specific gcd, but I will not expect too much gain. Best, Vicente
Le 10/01/15 04:32, Peter Dimov a écrit :
Vicente J. Botet Escriba wrote:
The dependency is on
|<boost/math/common_factor_rt.hpp>|
Rational's dependency on Math is also caused by the same file, by the way.
Great. I'm all for moving common_factor_rt.hpp to Integer. This move would change the level of Test/Timer/Chrono/Ratio/Rational from 15/14/13/12/11 to 7/6/5/4/3. :) Vicente
participants (7)
-
alex -
Beman Dawes -
John Maddock -
Paul A. Bristow -
Peter Dimov -
Rob Stewart -
Vicente J. Botet Escriba