New boost packaging suggestion for windows
Hello, yesterday I tried to download the boostpro binaries for windows and I had to download over a gigabyte to get all configurations for VS 2008. Sadly the version also was outdated and in general not that great ( e.g. Installer quite bad to handle and you have to register to download ) . So I decided to build from source and package it for a few friends who also needed these binaries. I managed to compress boost_1_40 ( no source except headers, no python AFAIK , but everything else you get with a standard build ) from over 900MB down to 37MB by using 7z. I uploaded it here https://obliviononline.com/pub/boost/boost_1_40_0.7z . Should boost use this form of releasing binaries in the future for windows ? Alternatively, you could endorse this form, like boostpro is endorsed right now and I could host it on google code, etc . An easy to set up ( just unzip this file ) binary distribution would make compiling boost-dependent software much easier. I would be willing to distribute future releases compatible with VS 2005 and 2008 in this form. Yours, Julian Bangert __________ Information from ESET NOD32 Antivirus, version of virus signature database 4551 (20091028) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com
on Wed Oct 28 2009, Julian Bangert <jbangert-AT-acm.org> wrote:
Hello,
yesterday I tried to download the boostpro binaries for windows and I had to download over a gigabyte to get all configurations for VS 2008. Sadly the version also was outdated
We've finally released a 1.40 installer today. I don't expect to see a delay like this one again; sorry for the wait.
and in general not that great ( e.g. Installer quite bad to handle
Specifically?
and you have to register to download ) .
That's not a huge problem is it?
So I decided to build from source and package it for a few friends who also needed these binaries. I managed to compress boost_1_40 ( no source except headers, no python AFAIK , but everything else you get with a standard build ) from over 900MB down to 37MB by using 7z.
We ship .zip files over the wire. Everything that the installer can install, including all variants of all libraries for 3 different compilers, tools, and source, for 1.40.0 comes to 360M. I know 7zip does better, but it's not an order of magnitude.
I uploaded it here https://obliviononline.com/pub/boost/boost_1_40_0.7z . Should boost use this form of releasing binaries in the future for windows ?
That's not a bad idea. Of course, if we could integrate 7(un-)zip into our installer, that too would keep the download size way down.
Alternatively, you could endorse this form, like boostpro is endorsed right now and I could host it on google code, etc . An easy to set up ( just unzip this file ) binary distribution would make compiling boost-dependent software much easier. I would be willing to distribute future releases compatible with VS 2005 and 2008 in this form.
I'm willing to reference anything in the Boost getting started guide that will be provided *reliably*. -- Dave Abrahams Meet me at BoostCon: http://www.boostcon.com BoostPro Computing http://www.boostpro.com
on Wed Oct 28 2009, Olaf van der Spek <olafvdspek-AT-gmail.com> wrote:
On Wed, Oct 28, 2009 at 6:30 PM, David Abrahams <dave@boostpro.com> wrote:
and you have to register to download ) .
That's not a huge problem is it?
It's still a problem.
In what way? -- Dave Abrahams Meet me at BoostCon: http://www.boostcon.com BoostPro Computing http://www.boostpro.com
David Abrahams a écrit :
on Wed Oct 28 2009, Olaf van der Spek <olafvdspek-AT-gmail.com> wrote:
On Wed, Oct 28, 2009 at 6:30 PM, David Abrahams <dave@boostpro.com> wrote:
and you have to register to download ) . That's not a huge problem is it? It's still a problem.
In what way?
As a rule, the need to register to access something is always an annoyance.
on Wed Oct 28 2009, Mathias Gaunard <mathias.gaunard-AT-ens-lyon.org> wrote:
David Abrahams a écrit :
on Wed Oct 28 2009, Olaf van der Spek <olafvdspek-AT-gmail.com> wrote:
On Wed, Oct 28, 2009 at 6:30 PM, David Abrahams <dave@boostpro.com> wrote:
and you have to register to download ) . That's not a huge problem is it? It's still a problem.
In what way?
As a rule, the need to register to access something is always an annoyance.
Understood. -- Dave Abrahams Meet me at BoostCon: http://www.boostcon.com BoostPro Computing http://www.boostpro.com
On Wed, Oct 28, 2009 at 7:58 PM, David Abrahams <dave@boostpro.com> wrote:
on Wed Oct 28 2009, Mathias Gaunard <mathias.gaunard-AT-ens-lyon.org> wrote:
David Abrahams a écrit :
on Wed Oct 28 2009, Olaf van der Spek <olafvdspek-AT-gmail.com> wrote:
On Wed, Oct 28, 2009 at 6:30 PM, David Abrahams <dave@boostpro.com> wrote:
and you have to register to download ) . That's not a huge problem is it? It's still a problem.
In what way?
As a rule, the need to register to access something is always an annoyance.
Understood.
What is the reason to require registration? Olaf
On Wed, Oct 28, 2009 at 7:31 PM, David Abrahams <dave@boostpro.com> wrote:
on Wed Oct 28 2009, Olaf van der Spek <olafvdspek-AT-gmail.com> wrote:
On Wed, Oct 28, 2009 at 6:30 PM, David Abrahams <dave@boostpro.com> wrote:
and you have to register to download ) .
That's not a huge problem is it?
It's still a problem.
In what way?
Unnecessary steps that take time and require an email address. Olaf
Hello, On 10/28/2009 11:34 AM, Julian Bangert wrote:
Hello,
yesterday I tried to download the boostpro binaries for windows and I had to download over a gigabyte to get all configurations for VS 2008.
Sadly, the only point being discussed so far is the online registration. What I find to be at least as annoying is the size of the package. And by that I'm not (only) referring to the memory or bandwidth. The most important issue is the burden of having to deal with such an amount of data, even if I'm only interested into a relatively small part of boost. I wonder whether there is still any thought or even effort spent on breaking boost up into individual components that can be built, tested, and ultimately also packaged separately. Any news on that ? Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin...
on Wed Oct 28 2009, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
Hello,
On 10/28/2009 11:34 AM, Julian Bangert wrote:
Hello,
yesterday I tried to download the boostpro binaries for windows and I had to download over a gigabyte to get all configurations for VS 2008.
Sadly, the only point being discussed so far is the online registration. What I find to be at least as annoying is the size of the package. And by that I'm not (only) referring to the memory or bandwidth. The most important issue is the burden of having to deal with such an amount of data, even if I'm only interested into a relatively small part of boost.
Huh? Admittedly the headers are all-or-nothing, but the Boostpro installer lets you choose exactly the binaries you're interested in.
I wonder whether there is still any thought or even effort spent on breaking boost up into individual components that can be built, tested, and ultimately also packaged separately. Any news on that ?
I'm hoping Troy can give us a public report on the modularization effort. Troy? -- Dave Abrahams Meet me at BoostCon: http://www.boostcon.com BoostPro Computing http://www.boostpro.com
On Wed, Nov 4, 2009 at 9:08 PM, David Abrahams <dave@boostpro.com> wrote:
on Wed Oct 28 2009, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
Hello,
On 10/28/2009 11:34 AM, Julian Bangert wrote:
Hello,
yesterday I tried to download the boostpro binaries for windows and I had to download over a gigabyte to get all configurations for VS 2008.
Sadly, the only point being discussed so far is the online registration. What I find to be at least as annoying is the size of the package. And by that I'm not (only) referring to the memory or bandwidth. The most important issue is the burden of having to deal with such an amount of data, even if I'm only interested into a relatively small part of boost.
Huh? Admittedly the headers are all-or-nothing, but the Boostpro installer lets you choose exactly the binaries you're interested in.
I wonder whether there is still any thought or even effort spent on breaking boost up into individual components that can be built, tested, and ultimately also packaged separately. Any news on that ?
I'm hoping Troy can give us a public report on the modularization effort. Troy?
Since we're talking about the boostpro installers, it would be great if: a) they included the option of 64-bit binaries b) There were filters of some kind so that I could easily target a specific compiler / platform / etc. So maybe a set of checkboxes for platform (x86/x64), a set of checkboxes for configuration (Debug/Release), and a set of checkboxes for compiler version (vc8, vc9, vc10). Then two additional buttons "enable all" and "disable all". So if I check x86/x64/Debug/Release/vc9, then click enable all, it would automatically enable or disable the appropriate items. As it is, it takes forever to go through the tree manually expanding every single library, then debug/release, then the compiler version which I would think is >90% of the use cases.
On Wed, Nov 4, 2009 at 9:08 PM, David Abrahams <dave@boostpro.com>
I'm hoping Troy can give us a public report on the modularization effort. Troy?
I suppose a picture is worth a thousand words: http://sodium.resophonic.com/boost/boost-dependencies.jpg Please stop reading now and look at that. Look for cycles and dependencies that don't seem to make sense. Realize that if you took the dependencies of test binaries into account you'd just add more edges. I don't know how many more. This graph was generated by graphviz from dependency information encoded in Boost.CMake's 'module.cmake' files, located in each library's source directory. Boost.CMake has the ability to reorganize headers so that instead of being held in one directory, they are held in ~70 (IIRC). First I'll explain my take on the tough bits about reorganizing boost's directory structure, then I'll explain why I think this is a bad idea. As I recently wrote offlist, regarding what happens to include paths when each of ~100 libraries has its own include/ directory:
Here's my back-of-the-envelope sketch of the discussion as I last recall it.
The hard use case is the library that is dependent on all the rest of boost. Here, adding a bunch of -I flags to the compile line and calculating some dependencies isn't the problem, as no sane person puts 100 header directories on one compile line, it is hard on the eyes and just doesn't scale. Eventually you're going to run out of commandline buffer space somewhere.
CMake's 'modularize' target moves headers out of toplevel boost/ as follows, this was uncontroversial:
libs/ python/ src/ ... doc/ ... test/ ... examples/ ... include/ boost/ <---- moved files python.hpp python/
and for each library p_i in a project's P's dependencies, -I$BOOST_ROOT/libs/p_i/include was added to P's compile line.
So with 100 dependencies, how do you present all of these things to the compiler such that you've got a reasonable compile line? .rsp files may do it on windows. Elsewhere, you're going to need to somehow construct a single header directory on disk. Maybe the mechanism is build in to source control, maybe it isn't. Possibilities:
- symlinks (not on windows you don't) - hardlinks created on checkout by version control - hardlinks generated by script - one generated directory full of forwarding headers (Qt has done this for years. They check these files in. I implemented a python script to do this for boost at some point.). - svn:externals (I used to advocate this, not any more) - git submodules (I don't advocate this either)
My view: I would prefer to follow Qt's method as it is simple, does not rely on version control tricks, and can do sanity checking for duplicate files and conventions about where one is allowed to put things. I believe I would distribute the generated header directory in release tarballs but not check it in. Developers would have to learn to regenerate the main header directory when adding/removing headers.
But since then my view has changed. Now why this is a bad idea: Shuffling headers around just makes the problem worse. It won't remove any edges from that graph. Take the parameter -> python dependency, for instance. Parameter is dependent on python because it uses python's internal version of referent_storage (basically just aligned_storage) in only one place. Python's version of boost/aligned_storage.hpp dates from 2002, presumably before boost/aligned_storage.hpp came along. This is easy to fix: just point them both at the toplevel aligned_storage, and you're done, and an edge disappears from that graph. There are probably hundreds more cases like this. Remove those edges. Then you would start seeing what modularity might look like. As when detangling a large knot: gently tug at the loose bits and see if you can unravel it, being careful not to make it worse. Shuffling headers around and making source control more complicated, on the other hand, won't remove any edges from that graph. So I would: - Write a script to generate that graph from the header files themselves. - Declare a moratorium on new libraries. - Start removing edges. -t
troy d. straszheim wrote:
So I would:
- Write a script to generate that graph from the header files themselves. - Declare a moratorium on new libraries. - Start removing edges.
Yep... And the really entertaining part is that removing edges might involve violating the moratorium. So what we are talking about is serious refactoring of code and libraries... Something that might be called Boost 2.0. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org (msn) - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail
Rene Rivera wrote:
troy d. straszheim wrote:
So I would:
- Write a script to generate that graph from the header files themselves. - Declare a moratorium on new libraries. - Start removing edges.
Yep... And the really entertaining part is that removing edges might involve violating the moratorium. So what we are talking about is serious refactoring of code and libraries... Something that might be called Boost 2.0.
+1 -t
Rene Rivera wrote:
- Write a script to generate that graph from the header files themselves. - Declare a moratorium on new libraries. - Start removing edges.
Yep... And the really entertaining part is that removing edges might involve violating the moratorium. So what we are talking about is serious refactoring of code and libraries... Something that might be called Boost 2.0.
I suggested that at some point. Havign a parallel boost branches in which this kind of restructuration is done.
AMDG troy d. straszheim wrote:
On Wed, Nov 4, 2009 at 9:08 PM, David Abrahams <dave@boostpro.com>
I'm hoping Troy can give us a public report on the modularization effort. Troy?
I suppose a picture is worth a thousand words:
http://sodium.resophonic.com/boost/boost-dependencies.jpg
Please stop reading now and look at that. Look for cycles and dependencies that don't seem to make sense.
I actually don't see /very/ much there that doesn't make sense. I really don't see what the problem is. Maybe some refactoring is in order, but I think that major refactoring is probably a waste of time.
Realize that if you took the dependencies of test binaries into account you'd just add more edges. I don't know how many more.
In Christ, Steven Watanabe
Steven Watanabe wrote:
troy d. straszheim wrote:
On Wed, Nov 4, 2009 at 9:08 PM, David Abrahams <dave@boostpro.com>
I'm hoping Troy can give us a public report on the modularization effort. Troy?
I suppose a picture is worth a thousand words:
http://sodium.resophonic.com/boost/boost-dependencies.jpg
Please stop reading now and look at that. Look for cycles and dependencies that don't seem to make sense.
I actually don't see /very/ much there that doesn't make sense.
I do. Why does spirit depend on xpressive? Why does xpressive depend on intrusive? And why *doesn't* proto depend on mpl? These are just the first three things I checked. :-P -- Eric Niebler BoostPro Computing http://www.boostpro.com
Eric Niebler wrote:
Steven Watanabe wrote:
troy d. straszheim wrote:
On Wed, Nov 4, 2009 at 9:08 PM, David Abrahams <dave@boostpro.com>
I'm hoping Troy can give us a public report on the modularization effort. Troy?
I suppose a picture is worth a thousand words:
http://sodium.resophonic.com/boost/boost-dependencies.jpg
Please stop reading now and look at that. Look for cycles and dependencies that don't seem to make sense.
I actually don't see /very/ much there that doesn't make sense.
I do. Why does spirit depend on xpressive? Why does xpressive depend on intrusive? And why *doesn't* proto depend on mpl? These are just the first three things I checked. :-P
I removed the modularization targets from cmake a while ago as they were causing confusion, trashing people's source directories, and I'd given up on solving this problem alone. You may see bugs. So use wave to write a program that generates this graph directly from the header files instead of from handcoded module.cmake files. -t
AMDG Eric Niebler wrote:
Steven Watanabe wrote:
troy d. straszheim wrote:
On Wed, Nov 4, 2009 at 9:08 PM, David Abrahams <dave@boostpro.com>
I'm hoping Troy can give us a public report on the modularization effort. Troy?
I suppose a picture is worth a thousand words:
http://sodium.resophonic.com/boost/boost-dependencies.jpg
Please stop reading now and look at that. Look for cycles and dependencies that don't seem to make sense.
I actually don't see /very/ much there that doesn't make sense.
I do. Why does spirit depend on xpressive?
AFAICT it doesn't now. This may be an artifact from when proto lived under xpressive?
Why does xpressive depend on intrusive?
Again it doesn't directly. I have no idea why this dependency appears.
And why *doesn't* proto depend on mpl?
The graph displayed is incomplete. proto depends on fusion which depends on function_type which depends on mpl.
These are just the first three things I checked. :-P
In Christ, Steven Watanabe
Steven Watanabe wrote:
AMDG
Eric Niebler wrote:
Steven Watanabe wrote:
troy d. straszheim wrote:
On Wed, Nov 4, 2009 at 9:08 PM, David Abrahams <dave@boostpro.com>
I'm hoping Troy can give us a public report on the modularization effort. Troy?
I suppose a picture is worth a thousand words:
http://sodium.resophonic.com/boost/boost-dependencies.jpg
Please stop reading now and look at that. Look for cycles and dependencies that don't seem to make sense.
I actually don't see /very/ much there that doesn't make sense.
I do. Why does spirit depend on xpressive?
AFAICT it doesn't now. This may be an artifact from when proto lived under xpressive?
Why does xpressive depend on intrusive?
Again it doesn't directly. I have no idea why this dependency appears.
And why *doesn't* proto depend on mpl?
The graph displayed is incomplete. proto depends on fusion which depends on function_type which depends on mpl.
These are just the first three things I checked. :-P
Right you are. Links to new versions and tabular format included. Feeling shamed into redoing it, I wrote a script that assumes that if file boost/X/F.hpp includes file boost/Y/f.hpp, then project X depends on Y. For toplevel files, if a file boost/K.hpp exists as well as a directory boost/K, then an include of boost/K.hpp is a dependency on K. for files boost/R.hpp where no subdirectory R exists, the dependency is either determined via an incomplete hand-constructed list of mappings from file to project, or the dependency is on special project UNCATEGORIZED. If a file is found that includes a file that does not exist, the dependency is on project MISSING. Script is here: http://sodium.resophonic.com/boost/depgraph.py output in tabular form: http://sodium.resophonic.com/boost/dependencies.txt and in pdf form: http://sodium.resophonic.com/boost/dependencies.pdf Enjoy, -t
On Thu, Nov 5, 2009 at 3:36 PM, troy d. straszheim <troy@resophonic.com> wrote:
Feeling shamed into redoing it, I wrote a script that assumes that if file boost/X/F.hpp includes file boost/Y/f.hpp, then project X depends on Y. For toplevel files, if a file boost/K.hpp exists as well as a directory boost/K, then an include of boost/K.hpp is a dependency on K. for files boost/R.hpp where no subdirectory R exists, the dependency is either determined via an incomplete hand-constructed list of mappings from file to project, or the dependency is on special project UNCATEGORIZED. If a file is found that includes a file that does not exist, the dependency is on project MISSING.
That's pretty useful. I'd like X -> R.hpp instead of X -> UNCATEGORIZED same with MISSING. ie no loss of information in that case. At least for the text version. Tony
participants (12)
-
David Abrahams -
Eric Niebler -
Gottlob Frege -
Joel Falcou -
Julian Bangert -
Mathias Gaunard -
Olaf van der Spek -
Rene Rivera -
Stefan Seefeld -
Steven Watanabe -
troy d. straszheim -
Zachary Turner