On Mon, Jun 16, 2014 at 11:02 PM, Julian Gonggrijp
There is a very obvious value in version control: dependencies may change from one Boost commit to the next. The dependency handler should work not only for end users, but also for maintainers and testers who check out any point in Git history.
Given that the dependency tracking tool is able to reconstruct that information from headers, this doesn't seem like a loss. Remember, that cached metadata is only supposed to speed up things for the most frequent use cases, which are checking out the most recent version (for testers and maintainers) or installing a release (for users). Everything that cannot be reconstructed (the "optional" annotations) is still in git. Thinking about it more, the requirement to build the dependency graph before the the download may require the cache to be available for any given git commit. This is probably a good reason to make the cache version controlled.
Perhaps it could be stored in a git note [1].
I'm not very familiar with git and don't know anything about git notes. Maybe they fit for this purpose. But my request would be that these notes are not required to be added by maintainers. Do git notes affect history? If yes, it would be undesirable if libraries history is spammed with automated commits adding notes with dependency info.
It is friendly to tell end users in advance what dependencies will be installed, but that can be solved by other means. A very simple solution would be to list the dependencies on the Boost website.
That doesn't really work for obvious reasons: (a) the advertised dependencies will get out of sync with reality sooner or later and (b) you can't realistically request users to consult the website when they are about to install a Boost library. The tool should provide that information. It is possible that the tool is not able to do that, if the cache is not available for the given commit to be checked out. The tool should notify the user about this problem but still allow to download the necessary components "blindly", by parsing headers for dependencies.
A slightly more advanced solution would be to have the handler download only the dependency file associated with the release tag using git archive [2], before cloning the entire module (that might not work with git notes, though). For releases, the dependency information could also simply be aggregated in the superproject archive.
Ok. As I said, I specifically did not require any particular means for delivering metadata into the tool. If this is possible with git, provided that usability is satisfactory, I'm all for it. Another alternative is to create a new git submodule to store the cache in.
The advantage of just storing a plain file in the module directory is that it certainly works, even if you download an archive without git history, and without a need to set up a new FTP server or other web service. I would prefer to start there and investigate prettier solutions later.
We're discussing a mechanism that will require mass changes to the libraries and possibly the workflow. I'd say the system should be designed without fundamental flaws from the start. It may not implement everything from the beginning, that I agree.
What I meant is that there should be some kind of "official" Boost repository which will be used by the packaging tool (let's call it boost-pkg for brevity). That repository will serve the metadata for boost-pkg. The metadata will be updated when a certain new release is published into it, whether that is a new library release through a git tag or the whole Boost release. I'm not sure if updating the metadata upon a tag creation can be automated, but at least for major Boost releases this should be doable.
This seems to confirm that you are not interested in dependency handling for maintainers and testers (yet).
What makes you think so?
Continuing my fantasy,...
This gets more and more ambitious. Don't take me wrong, I like what you describe, but I think it will be easier to get there if we take it one step at a time.
I think you have now almost reinvented Ryppl, but I might be mistaken.
Yes, I let myself get carried away a little, and I'm not asking to implement all that. I'm just assessing the possibilities.
In addition, for maintainers: 3. Create/update the configuration file by running the handler tool before pushing to the public repo (this could be a git hook).
This would be a blocker, for me at least. I'm sure I will forget doing that and will be very much annoyed. A git hook that scans the headers on every push doesn't sound very good.
But you do agree that caching is a good idea, right? You seem to believe that caches should only be created for releases.
I agree that the cache is a good idea, as long as it's just a cache. I'm just saying that its role is auxiliary and it should not be managed by developers. I did not say that the cache should only be created for releases (note that I mentioned checking out develop and master with boost-pkg). But it might be more difficult to build the cache in time for heads of branches; there will be some latency between the commit and its metadata.
I think a dependency handler would have at least as much value to maintainers and testers, if it works for any commit.
I'm not arguing with that. And it should work, even if there is no cache whatsoever.