On Monday 16 June 2014 11:42:58 Julian Gonggrijp wrote:
Andrey Semashev wrote:
If the tool is able to extract the information from the headers then why do we need the config files? We should minimize the amount of information to be managed by developers - to just the "optional" annotations.
I agree with this sentiment, but we need a way to cache the full dependencies that doesn't require us to change the entire structure of Boost.
Ok, I just don't think that library repos are a good place for this cache.
But this doesn't mean this list has to be stored in git and managed by developers. You can employ the approach taken by most package managers (e.g. in Linux and OS X ports, I think). There are downloadable packages (which would correspond to sub-modules or links to their git repos) and auto-generated metadata to help resolving dependencies _prior_ to downloading anything.
Why emphasise "prior"? The user requests module X so I download X anyway. Then I can look up what X depends on, whether those data are summarized within X or somewhere outside it.
This is a common courtesy of package managers and installers. The tool should state what it's going to download and install (and often this alerts the user enough so that he cancels). This is a useful protection against undesirable situations like "let me install that one library... oh, now it installs half of Boost for no apparent reason."
Where do you store the metadata if not within the module? I like the idea of taking automatically generated data out of the versioning system, but it should be minimally invasive.
I'm specifically not restricting this part, other than that metadata has to be available without downloading the submodule. As a simple example, an ftp server should be enough to set up a Boost distribution repository. The metadata is stored in a (compressed) text file or several files (better in one file though to speedup its downloading). The tool is able to download this file and build the dependency graph upon user's request before installing anything. If you feel git or another VCS suits better for this metadata, you can use it instead, but I don't see much value in version controlling for this data, and VCSs seem to add quite some overhead. Another reason I want to separate the metadata from git repos - and I'm fantasizing now - is that I can see this tool being used without git at all - to download source packages and install Boost on the user's machine. For example, if I want to install a subset of Boost 1.57 on my machine, I'd like to be able to do that easily, without dealing with git submodules and without downloading the whole git history. The tool will just resolve dependencies, download and extract a set of archives for me.
The metadata should be automatically updated when the packages are uploaded (i.e. official snapshots are uploaded or a referred git tag is added).
Do you envision this in the current situation where "packages" are loaded as sub-modules of the boost super-project, or in a new situation where the boost super-project is taken away and "packages" are standalone (but with dependencies)?
What I meant is that there should be some kind of "official" Boost repository which will be used by the packaging tool (let's call it boost-pkg for brevity). That repository will serve the metadata for boost-pkg. The metadata will be updated when a certain new release is published into it, whether that is a new library release through a git tag or the whole Boost release. I'm not sure if updating the metadata upon a tag creation can be automated, but at least for major Boost releases this should be doable. Continuing my fantasy, the repository may contain standalone packages for library and Boost releases, which may be useful for Boost users (in the above example, I would be downloading archives of Boost 1.57 from this repository). It may also contain references to git repositories - tags or branches. boost- pkg would offer a unified interface so that it is possible to use either of them - e.g. download official Boost 1.57 and a newer release of that one library X, which fixes a critical bug for me. Potentially, boost-pkg could replace the superproject and be used to checkout the whole Boost from git on develop or master branch, which would be useful for developers and testers. For testers this would help to checkout the tested library from develop and everything else from master. Although it is also possible to do with plain git, when you have checked out everything.
In a nutshell, both for users and maintainers: 1. Clone the superproject non-recursively. 2. Request specific modules to be installed by Boost.Build; dependencies are tracked by the handler tool which uses git to clone more modules.
That would require at least Boost.Build to be checked out as well.
In addition, for maintainers: 3. Create/update the configuration file by running the handler tool before pushing to the public repo (this could be a git hook).
This would be a blocker, for me at least. I'm sure I will forget doing that and will be very much annoyed. A git hook that scans the headers on every push doesn't sound very good.