bpm, a tool to install a modular Boost distribution
One month ago, Stephen Kelly wrote:
git affects the developer workflow, but that thread was about creating modular releases and downloads, which is a different goal and has a different set of problems to solve. That's what that thread was about. I didn't see any desire to have that as a goal in the thread.
Git doesn't affect users.
Before and after transition to git, they download all-of-boost.zip from the boost website.
To address this point of his, I've been working at a tool, bpm, that can install a modular Boost distribution. As I've already mentioned in other threads, it works like this: D:\boost-1.60.0>bpm install system bpm: installing module 'system' bpm: installing module 'assert' bpm: installing module 'config' bpm: installing module 'core' bpm: installing module 'predef' bpm: installing module 'build' bpm: the following libraries need to be built: system bpm: (use ./b2 to build) bpm: recreating header links bpm: recreating index That is, when told to install Boost.System, it downloads and installs the 'system' package into ./libs/system, along with all its dependencies, then creates links to the headers (like 'b2 headers') in ./include and an ./index.html file listing the installed libraries. The full syntax is: D:\boost-1.60.0>bpm Usage: bpm [-v] [-q] command [options] [modules] -v: Be verbose -vv: Be more verbose -q: Be quiet bpm install [-n] [+d] [-k] [-a] [-i] [-p] <module> <module>... Installs the specified modules and their dependencies into the current directory. -n: Only output what would be installed +d: Do not install dependencies -k: Do not remove partial installations on error -a: All modules (use instead of a module list) -i: Installed modules -p: Partially installed modules bpm remove [-n] [-f] [-d] [-a] [-p] <package> <package>... Removes the specified packages. (A package, like 'numeric', can contain more than one module.) -n: Only output what would be removed -f: Force removal even when dependents exist -d: Remove dependents as well. Requires -f -a: Remove all packages. Requires -f -p: Remove partially installed packages bpm list [-a] [-i] [-p] [-b] [prefix] Lists modules matching [prefix]. -a: All modules (default when no -b) -i: Installed modules (default when -b) -p: Partially installed modules -b: Modules that require building bpm headers Recreates the header links in the include/ subdirectory of the current directory. bpm index Recreates the file index.html, which lists the installed modules, in the current directory. -- This all works already under Windows. (I haven't finished the POSIX part yet but it should be close.) I could have made a Windows binary available for people to play with, but I'd rather give sources (at first). bpm contains code to do rm -rf <directory>, and if I've made a mistake somewhere, it could cause damage. So, if there is interest and there are no objections, I'd like to put this tool into the Boost repository, in tools/bpm.
On 5 January 2015 at 15:29, Peter Dimov <lists@pdimov.com> wrote:
So, if there is interest and there are no objections, I'd like to put this tool into the Boost repository, in tools/bpm.
IMO we should get out of the habit of adding everything to the Boost tree, new tools should be able to live in their own repo. I think the only reason not to is because Boost.Build is setup to expect that they'll be in the tools directory, which doesn't apply here. Especially since it aims to stop treating Boost as monolithic.
Hi! Am Montag, 5. Januar 2015, 17:12:57 schrieb Daniel James:
On 5 January 2015 at 15:29, Peter Dimov <lists@pdimov.com> wrote:
So, if there is interest and there are no objections, I'd like to put this tool into the Boost repository, in tools/bpm.
+1 for the tools.
IMO we should get out of the habit of adding everything to the Boost tree, new tools should be able to live in their own repo.
+1 from me.
I think the only reason not to is because Boost.Build is setup to expect that they'll be in the tools directory, which doesn't apply here.
I see no reason that tools/bpm can't be a submodule.
Especially since it aims to stop treating Boost as monolithic.
Yes. That should be a goal. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
On 5 January 2015 at 19:39, Jürgen Hunold <jhunold@gmx.eu> wrote:
I think the only reason not to is because Boost.Build is setup to expect that they'll be in the tools directory, which doesn't apply here.
I see no reason that tools/bpm can't be a submodule.
Is there any reason that it should be? The submodule system isn't great, I'd rather not add things to it for no reason. It also means adding the tool to the release, which isn't very appropriate here. If someone has already downloaded the release, they don't need a tool to download the things they've already got. It also probably won't work with the contents of the tarball, because the headers aren't in the same place. A lot of things in tools (e.g. inspect, boostbook, quickbook) aren't needed by users, they mainly need to be there because Boost.Build expects them to be. Otherwise I'd have suggested removing the things I maintain a while ago.
Daniel James wrote:
A lot of things in tools (e.g. inspect, boostbook, quickbook) aren't needed by users, they mainly need to be there because Boost.Build expects them to be.
That is the reason. We could leave the tools in their separate repositories under boostorg without making them submodules, but they won't build unless put into tools/. And if a tool X requires another tool Y for some reason, that other tool would also need to be in tools/Y for tools/X/build/Jamfile to find it. It's certainly more convenient for people using the development source tree for the tools to be there by default, kept up to date by git pull && git submodule update --init, switched to master/develop as appropriate, and so on. Although you are right that putting bpm into a release may not be of much use, except for users who want to look at its (officially released) source. And now that you've brought it up, I see that the release doesn't contain libs/*/include, so boostdep wouldn't work on it as well. I hadn't realized that. :-/
On 5 January 2015 at 20:52, Peter Dimov <lists@pdimov.com> wrote:
Daniel James wrote:
A lot of things in tools (e.g. inspect, boostbook, quickbook) aren't needed by users, they mainly need to be there because Boost.Build expects them to be.
That is the reason. We could leave the tools in their separate repositories under boostorg without making them submodules, but they won't build unless put into tools/.
You can set up a repo to build outside of the boost tree, I've done it for quickbook in the past. The problem is that when you're building documentation, boost.build expects the tools to be at the relevant location. Some of the tests run inspect in a similar manner. This is also an issue for using the tools outside of boost, since everything is in the tree, no one installs anything. The python script for rebuilding the website was meant to be really simple to use, but isn't because there's no standard way to find quickbook.
And if a tool X requires another tool Y for some reason, that other tool would also need to be in tools/Y for tools/X/build/Jamfile to find it.
Is that the case for any of our existing tools?
It's certainly more convenient for people using the development source tree for the tools to be there by default, kept up to date by git pull && git submodule update --init, switched to master/develop as appropriate, and so on.
Ideally the tools shouldn't need to be updated and rebuilt too often. Some of them are now packaged on Debian, and perhaps elsewhere, so I'd like to use that. I'd prefer to use a stable binary, even if it's a bit out of date. Boost.Build is a bit annoying in this regard because you can install a standalone version including the various jamfiles it come with, but when building anything from boost it uses the scripts from boost rather than the installation, which might not match.
On Mon, Jan 5, 2015 at 3:52 PM, Peter Dimov <lists@pdimov.com> wrote:
Daniel James wrote:
A lot of things in tools (e.g. inspect, boostbook, quickbook) aren't
needed by users, they mainly need to be there because Boost.Build expects them to be.
That is the reason. We could leave the tools in their separate repositories under boostorg without making them submodules, but they won't build unless put into tools/. And if a tool X requires another tool Y for some reason, that other tool would also need to be in tools/Y for tools/X/build/Jamfile to find it. It's certainly more convenient for people using the development source tree for the tools to be there by default, kept up to date by git pull && git submodule update --init, switched to master/develop as appropriate, and so on.
Although you are right that putting bpm into a release may not be of much use, except for users who want to look at its (officially released) source.
Putting tools that are not normally needed by users into the boost super project either directly or as sub-modules seems like the monolithic approach we are trying to get away from. Is there any reason Boost.Build can't be set up to build our tools even if they are in separate boostorg repos? --Beman
Beman Dawes wrote:
Putting tools that are not normally needed by users into the boost super project either directly or as sub-modules seems like the monolithic approach we are trying to get away from. Is there any reason Boost.Build can't be set up to build our tools even if they are in separate boostorg repos?
After giving this some thought, I'm actually not quite sure what we'll be trying to accomplish with that. Why is having the tools in $BOOST/tools/ (as submodules) a problem? As a developer, I (obviously) need tools/build to build and test, need tools/quickbook and tools/boostbook (and possibly tools/auto_index as well?) to build the documentation, need tools/inspect to run a local inspection report, need tools/boostdep to check dependencies, so why would I ever want to have to git clone each of those separately? What does this leave us with? tools/bcp, which is useful for users, so it goes into the release, tools/litre, about which I have no idea, tools/regression and tools/server. Hm. So which of these are we going to move outside of the tree, and what specific benefits will we derive from that?
On Mon, Jan 5, 2015 at 9:34 PM, Peter Dimov <lists@pdimov.com> wrote:
Beman Dawes wrote:
Putting tools that are not normally needed by users into the boost super project either directly or as sub-modules seems like the monolithic approach we are trying to get away from. Is there any reason Boost.Build can't be set up to build our tools even if they are in separate boostorg repos?
After giving this some thought, I'm actually not quite sure what we'll be trying to accomplish with that. Why is having the tools in $BOOST/tools/ (as submodules) a problem?
As a developer, I (obviously) need tools/build to build and test, need tools/quickbook and tools/boostbook (and possibly tools/auto_index as well?) to build the documentation, need tools/inspect to run a local inspection report, need tools/boostdep to check dependencies, so why would I ever want to have to git clone each of those separately?
What does this leave us with? tools/bcp, which is useful for users, so it goes into the release, tools/litre, about which I have no idea, tools/regression and tools/server. Hm. So which of these are we going to move outside of the tree, and what specific benefits will we derive from that?
I was thinking of removing tools/release when the switch to it's subrepo is complete. To move towards only having stuff that developers and users need in the main tree. But that's only possible because most of what's in regression is similar in nature to the website. Which has an independent repo. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
On Mon, Jan 5, 2015 at 10:34 PM, Peter Dimov <lists@pdimov.com> wrote:
Beman Dawes wrote:
Putting tools that are not normally needed by users into the boost super project either directly or as sub-modules seems like the monolithic approach we are trying to get away from. Is there any reason Boost.Build can't be set up to build our tools even if they are in separate boostorg repos?
After giving this some thought, I'm actually not quite sure what we'll be trying to accomplish with that. Why is having the tools in $BOOST/tools/ (as submodules) a problem?
As a developer, I (obviously) need tools/build to build and test, need tools/quickbook and tools/boostbook (and possibly tools/auto_index as well?) to build the documentation, need tools/inspect to run a local inspection report, need tools/boostdep to check dependencies, so why would I ever want to have to git clone each of those separately?
What does this leave us with? tools/bcp, which is useful for users, so it goes into the release, tools/litre, about which I have no idea, tools/regression and tools/server. Hm. So which of these are we going to move outside of the tree, and what specific benefits will we derive from that?
At this point, I'm unwilling to slow bpm development while we try to come up with a clear guideline as to what tools get their own separate repo. So it is OK with me if you put it in tools/bpm. --Beman PS: I'm thinking a lot about Unicode issues right now, so my fingers keep typing bmp (i.e. Basic Multilingual Plane) instead of bpm (Boost Package Manager:-).
On Tue, Jan 6, 2015 at 10:40 AM, Beman Dawes <bdawes@acm.org> wrote:
On Mon, Jan 5, 2015 at 10:34 PM, Peter Dimov <lists@pdimov.com> wrote:
Beman Dawes wrote:
Putting tools that are not normally needed by users into the boost super project either directly or as sub-modules seems like the monolithic
approach
we are trying to get away from. Is there any reason Boost.Build can't be set up to build our tools even if they are in separate boostorg repos?
After giving this some thought, I'm actually not quite sure what we'll be trying to accomplish with that. Why is having the tools in $BOOST/tools/ (as submodules) a problem?
As a developer, I (obviously) need tools/build to build and test, need tools/quickbook and tools/boostbook (and possibly tools/auto_index as well?) to build the documentation, need tools/inspect to run a local inspection report, need tools/boostdep to check dependencies, so why would I ever want to have to git clone each of those separately?
What does this leave us with? tools/bcp, which is useful for users, so it goes into the release, tools/litre, about which I have no idea, tools/regression and tools/server. Hm. So which of these are we going to move outside of the tree, and what specific benefits will we derive from that?
At this point, I'm unwilling to slow bpm development while we try to come up with a clear guideline as to what tools get their own separate repo. So it is OK with me if you put it in tools/bpm.
I'm fine with placing it at tools/bpm. But I would highly prefer that it was a submodule. Like everything else is in that directory (or soon will be). -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
participants (5)
-
Beman Dawes -
Daniel James -
Jürgen Hunold -
Peter Dimov -
Rene Rivera