On 18 Sep 2014 at 9:25, Stephen Kelly wrote:

...

...

And I get all of it. There are no separate 'modular Boost' tarballs so that I can download what I want and not what I don't.

Is something similar to this a goal or a non-goal for 'Boost modularization'?

I have begun weekend work as I said I would on my clang AST parser based C++11-STL-to-Boost bindings generator.

It's an interesting announcement but this thread seems a strange place for it, as you didn't directly address the thread topic.

No point in announcing something which isn't finished. But it was germane to your point, so I mentioned it. I'm annoying with having to download and build all of Boost every single bloody time I want some small bit of header only Boost for some task, and it wastes enough of my productivity I am willing to personally invest in making it go away permanently.

But, I guess you're saying 'for me it's a non-goal'.

No, rather it's a goal for me for the best maintained C++ 11/14 capable header-only Boost libraries - the ones with maintainers active enough that I suspect they will reuse my tooling to achieve standalone C++ 11 only usefulness on their own, I just need to put the tooling there and they'll do the rest themselves as I am sure my Boost pain is not unique. The rest of libraries and the other legacy cruft, as everyone knows, in my opinion need to be put out to pasture and gracefully deprecated. Growth ~= creative destruction. Properly modularising all of Boost is an enormous undertaking, even with a full time developer exclusively working on it it would take at least a year and lots of library maintainers would see breakage, so it would be unpopular too. Personally, I think it's a non-starter, and the lack of decisive leadership on this issue merely delays the inevitable to the detriment of all. So I'm pressing ahead with my own solution - I specifically want single git submodule dependencies, so when I throw together a Boost using project I merely need to submodule a single Boost git submodule and it just works with no arsing around with bootstrapping or config or staging. When it's usable, I'm sure you'll hear from me here. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 18 Sep 2014 at 11:01, Stephen Kelly wrote:

...

...

But, I guess you're saying 'for me it's a non-goal'.

No,

Let me rephrase. The download link on boost.org takes me to a page where I can download all-of-boost.zip. My question was whether it should be a goal for it to offer boost-static_assert.zip.

Your response was along the lines of 'it doesn't matter to me what the download link on boost.org does. Here is my solution...', or with a different paraphrase 'what you describe is not a goal for me. My goal is...'.

Or you're saying you want the download link on boost.org to point to your tool?

Oh now I see what you're saying. You're saying that the git submodule boost.thread should be separately downloadable as boost-thread.zip right? What I was very badly explaining was that can't be useful until each submodule is capable of standing alone - or, at the very minimum, each submodule indicates which other submodules it depends on, and significant effort is invested in tidying up the sloppy inadvertent dependencies which are in there. But then you especially know all this already. I then outlined how I am going about breaking off the libraries I listed so they can work without the rest of Boost by using the C++ 11 STL instead. They could be very easily distributed as standalone zip archives, indeed I was going to get github to do that for me.

...

and the lack of decisive leadership on this issue merely delays the inevitable to the detriment of all.

A lack of leadership or cohesion on this issue is indeed disappointing.

I have personally recently invested some time into creating a donations channel for Boost, so lack of funding at least will not be an issue. I have also invested some time into arranging for GSoC students to stay on after GSoC at Boost expense to work on Boost, so by next year hopefully lack of grunt workers will also not be an issue.

...

So I'm pressing ahead with my own solution - I specifically want single git submodule dependencies, so when I throw together a Boost using project I merely need to submodule a single Boost git submodule and it just works with no arsing around with bootstrapping or config or staging.

And the user will be responsible for adding the individual -Ifoo/include paths?

I won't be supporting non-header-only libraries, so no linking needed apart from system libraries, and yes you're on your own for those. This only really affects GCC anyway, clang and MSVC auto-link system libraries. Include paths are straightforward. My tool replicates the boost header structure into a faked boost/* directory include structure, so you simply use a different -Ipath and you're good to go. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 18 Sep 2014 at 12:05, Stephen Kelly wrote:

...

Oh now I see what you're saying. You're saying that the git submodule boost.thread should be separately downloadable as boost-thread.zip right?

I am asking if that is a goal or a non-goal.

For the whole Boost org, I don't think anybody knows, and even if they did, it ain't happening without a huge investment of time. There are vague statements of "wouldn't it be nice", but that isn't "boots on the ground" as it were. Individual library maintainers have much stronger opinions. For example, ASIO has been capable of standalone for a long time now. I predict many more Boost libraries will become capable of standalone too in the next few years, because no C++ 11/14 library wants to force all of Boost onto its users if it expects to get any users. Especially as STLs are pretty good now, and that makes using Boost for the same facilities redundant.

...

Include paths are straightforward. My tool replicates the boost header structure into a faked boost/* directory include structure, so you simply use a different -Ipath and you're good to go.

Let's say I want to use boostfoo and boostbar which you make available as standalone libraries in boostfoo.git and boostbar.git and I want to use both of them in myproj, which also uses git.

So, you say the steps are:

1) I add boostfoo.git and boostbar.git as submodules to myproj.git 2) I run some tool to create a faked boost/ directory 3) I add -Ipath to my buildsystem for that faked boost directory.

So, I need the tool at step 2, right? And when I also need boostflub.git I need to run it again? Or are you instead saying that the steps are

1) I add boostfoo.git and boostbar.git as submodules to myproj.git 2) I add -Ipath/foo and -Ipath/bar to my buildsystem

In other words "the user will be responsible for adding the individual - Ifoo/include paths?" as I wrote before?

Yes, the latter. Or, seeing as we know they are submodules, you could simply include the headers explicitly in your source code. I am hoping that because no build system is needed, we need to neither force nor require any build system on the users. Or unit test system for that matter. By standalone, I really mean standalone. Each standalone library has its own faked boost headers suitable for it, but these shouldn't spill outwards thanks to namespacing. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 09/18/2014 11:31 AM, Stephen Kelly wrote:

Vladimir Prus wrote:

...

...

Is something similar to this a goal or a non-goal for 'Boost modularization'?

It looks that unless you, or somebody else, writes npm-for-c++, then the huge amount of modules in Boost will make any such modular tarballs be a disservice to users.

A disservice to users? How so?

You have current situation - where users know to go a known place, download tarball, and use it. What you're proposing is...

There are several such npm-like things. The problem isn't choosing or writing one, but adoption. A prerequisite to it being operational is modular releases, so all you're doing is designing catch-22 situations for me to answer.

... to introduce yet another option, namely to pick some subset of 100 tarballs to download, where a useful subset is often big, and where you don't appear to plan any way to automate this right away. So that seems like offering a users a fairly inconvenient, if not useless, way to obtain Boost components, and so such choice seems more like trouble to me. You can argue that an ultimate goal of running some tool with the name of Boost components you want, and having those downloaded along with dependencies, is a good goal (and there's a lot of reason to that) and that an infrastructure to create per-component tarballs is important implementation part, but in itself, having a bunch of tarballs somewhere on the web seems like a non-goal to me.

...

It's must easier to just close everything and remove a couple of components that take up 90% of source size.

Users have always had that option.

What does 'boost modularization' introduce that's new for users? Why would anyone possibly get excited about the fact that static_assert is in a library of its own? How does that fact affect users of boost in any way?

I don't follow the logic of your questions. I'm not excited about static_assert being a separate git repository, nor am I excited about prospect of having a separate 'release' thereof. And by saying "why would anyone possibly be excited" above you appear to have the same view - while your earlier comments appear to suggest that individual releases is a good thing. Could you spell out the final destination you're after?

...

(Preemptive snarky comment: Qt does not need npm, since the number of modules is relatively small, and many of them are truly optional)

Tell me how the 'truly optional' phrase applies or does not apply to boost modules. It's not clear what comparison you're drawing here.

It applies in that many users can download Qt Core and Qt GUI, and don't care about the rest. In case of Boost, pretty much any useful library will pull a dozen others, and therefore asking the users to download component tarballs becomes much less reasonable approach. - Volodya

On 09/18/2014 02:29 PM, Stephen Kelly wrote:

...

So that seems like offering a users a fairly inconvenient, if not useless, way to obtain Boost components, and so such choice seems more like trouble to me.

Whether it's useless depends on how many split tarballs there are and what they contain. There are many ways to split. The 'many small core libraries' could all be together for example. But don't get attached to details: I'm saying there are many ways to split.

Fair enough.

...

...

What does 'boost modularization' introduce that's new for users? Why would anyone possibly get excited about the fact that static_assert is in a library of its own? How does that fact affect users of boost in any way?

I don't follow the logic of your questions.

Do you follow logic of the question "What does 'boost modularization' introduce that's new for users?" ?

Putting boost-static_assert in a library of it's own is part of modularizing boost, and I'm told it's a good thing that it's in a library of it's own. How would any 'good thing' from 'modularized boost' translate to 'a good thing' for users of boost?

In this particular case, I don't believe it's a good thing at all, but as you said above, there are many ways to split, so let's move on to your next point.

What is the 'user story' of modularized boost? Most users use the download link in boost.org, right? They don't use the git repos. What changes?

Modularization of the git repos changes the 'developer story', but how does the 'user story' change?

If there's a consensus on that, I haven't heard it. I heard vague suggestions that modulization would permit individual libraries to have their own release schedule, but it was never spelled out how that would work for users - like how the users would obtain everything they need, make sure it's supposed to work together and verify that it does, and so forth. What Niall say in this thread (i.e. "JQuery model") is closest to a workable proposal that I've heard - although the idea to run a separate site for that rings a few bells.

...

Could you spell out the final destination you're after?

I'm trying to find out whether this thing is a goal or a non-goal, and if it is a non-goal, then is there a different 'user story' to aim for.

It seems that some way for users to get a desirable subset of Boost will be valuable, but I don't know whether it is "official" goal in any sense, and implementation is just as important as general goal, so it's hard to agree on a goal without somebody sketching how it will work. - Volodya

Daniel James wrote:

On 18 September 2014 08:31, Stephen Kelly wrote:

...

What does 'boost modularization' introduce that's new for users? Why would anyone possibly get excited about the fact that static_assert is in a library of its own?

Maybe you should have asked such questions before aggressively pushing for changes.

Maybe you could tell me, in your words, why Boost migrated from one svn repo to a hundred interdependent git repos? Was that done with any purpose or goal in mind, in your words? Thanks, Steve.

Thijs (M.A.) van den Berg wrote:

Now it has grows into a large set of libraries for specific niches with varying level quality (code, docs, maintenance support)

Really? Is that what it is? Are you sure? Or is that a goal you have in mind? Is that what boost was while it was in svn? Or did boost become that by migrating to 100 *interdependent* git repos? Did migrating to 100 *interdependent* git repos help the above statement in any way?

, and perhaps varying levels of language support (C++14 only libs?) and modularization makes sense.

Some examples of environment that have solved the issues of having large sets of libraries with dependencies and various levels of quality: Debian Linux, R project, python pip.

Are you saying you want similarity to those projects from boost? Boost is released as one tarball, as I linked in the original post, and that is not going to change. Or it is not a goal to change that, apparently. In that monolithic light, please walk me through the analogy how current boost (or a boost you have as a goal) relates to Debian Linux, because I don't get it.

Maybe we need an independent "apt-get" like tool for C++ libraries? Not just for boost libraries, but various other C++ libraries as well?

Are you saying this should be a goal? Is a modular release of boost (a tarball per library) a prerequisite for that or not? Tell me how things are now in your words and tell me what your goal is. Thanks, Steve.

Thijs van den Berg wrote:

A goal I have in mind is to give individual libraries clear personal identities and allow for fragmentation of boost.

What do you mean by fragmentation? Will such fragmentation affect users in any way? How so?

...

apt-get install boost-math and that that would result in my machine downloading some boost libraries so that I can use boost.math with my specific compiler on my specific

It would be nice if I could do: platform.

Boost could help debian packagers make that possible by creating modular releases. However, even without modular tarball releases, debian packagers can create split packages. However, that would require that packages can be split in a non-cyclicly dependent way. Anyway, the above niceness does not seem to be a goal of boost modularization currently, as you can see from reading this thread.

Part of the it is about a dependency tree,minimal requirements, but also about configuration / versions. I would not like it to pull in *all* of the boost respository. I imagine that at some point we could have e.g. a Boost.Math for C++14 fork and If I said I wanted that, then I would *not* get parts like <random> that have already been added to the standard. Boost could also solve that by staying monolithic and providing switches or include wrappers, but those solutions would be less intuitive.

Yes, boost could do these things. However, you will notice by reading this thread that these things (modularity from the users perspective) are non- goals currently.

...

Tell me how things are now in your words and tell me what your goal is.

My goals would be change boost from a single entity "the boost libraries bundle" to a platform hosting multiple identies "the platform where i can find e.g.Boost.Math" because I think it will make it easier to contribute, and easier to use. The most important change would be to change the single identity of the whole of boost (identity = the thing you install as a user, the thing you need to learn to use, they think you asses the quality of) to multiple indentities -one for each library of boost-.

This thread shows that these things are non-goals.

In my opinion these are some direction we *can* take now that we have focus on modularisation and dependency graphs

Yes, but apparently they are non-goals.

Thanks for responding and asking for clarification,

Thanks for your response too. I agree what you want would be nice, but what you want seems to be non-goals for boost. That's why I would like to hear from others why boost migrated to git. What goals or purpose did that serve? Did it only serve some goals for the developer perspective (ie, not any goals for the users of boost)? Thanks, Steve.

On Mon, Sep 22, 2014 at 1:53 AM, Stephen Kelly wrote:

Thijs (M.A.) van den Berg wrote:

...

Now it has grows into a large set of libraries for specific niches with varying level quality (code, docs, maintenance support)

Really? Is that what it is? Are you sure? Or is that a goal you have in mind? Is that what boost was while it was in svn? Or did boost become that by migrating to 100 *interdependent* git repos? Did migrating to 100 *interdependent* git repos help the above statement in any way?

Reducing physical coupling is a good thing even if you use a single repo. Modularizing Boost does make it practical to break it down into separate repos, which allows users to git only what they need. I don't see how this interferes with users who want to get everything Boost if the official distribution remains monolithic. -- Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

Daniel James wrote:

On 22 September 2014 09:07, Stephen Kelly wrote:

...

Daniel James wrote:

...

On 18 September 2014 08:31, Stephen Kelly wrote:

...

What does 'boost modularization' introduce that's new for users? Why would anyone possibly get excited about the fact that static_assert is in a library of its own?

Maybe you should have asked such questions before aggressively pushing for changes.

Maybe you could tell me, in your words, why Boost migrated from one svn repo to a hundred interdependent git repos? Was that done with any purpose or goal in mind, in your words?

I was against splitting up the repo in the first place, so I'm not really the person to ask. But IMO the main (perhaps only) benefit is that it makes version control easier.

Well, that's seems to be a good reason to push for it. In this thread I asked if there are any additional user-oriented goals for modularization. It is conclusive that there are not. Thanks, Steve.

On Tue, Sep 23, 2014 at 2:51 AM, Daniel James wrote:

I was against splitting up the repo in the first place, so I'm not really the person to ask. But IMO the main (perhaps only) benefit is that it makes version control easier. Merging under subversion was a pain because we didn't have a coherent merge history and a monolothic git repo would certainly have had the same problem. By splitting the code into modules we can track what has been merged, and do things like using the develop branch of a single library with the release branch of other libraries.

I don't really see any direct benefit to users. Maybe if we had some sort of package management system it could, but we don't. I suspect that packages shouldn't have a one-to-one relationship with git modules. The reorganisation of the MPL module might be a model for how multiple packages could be stored in a single repo.

Physical coupling is difficult to break, easy to introduce. That is to say users can not possibly suffer from the many repos because it is trivial to make a release that bundles everything together. As for the benefits of breaking up dependencies and splitting libraries into separate modules, they aren't limited to version control. Reducing coupling increases the efficiency of development and maintenance work in general, but that is only visible in very large projects. Which is why it gets neglected. -- Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

On Tue, Sep 23, 2014 at 1:27 PM, Daniel James wrote:

I was writing about splitting up repos. That's a completely separate issue to reducing dependencies.

How is it different? To me it looks like simply a higher level of commitment. -- Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Daniel James Sent: 23 September 2014 10:52 To: boost@lists.boost.org Subject: Re: [boost] [modularization] Are modular releases a goal or a non-goal?

...

Maybe you could tell me, in your words, why Boost migrated from one svn repo to a hundred interdependent git repos? Was that done with any purpose or goal in mind, in your words?

I suspect that it was partly because of a desire to use a better version control system than SVN? (There were long discussions comparing Mercurial and GIT etc). And a wish for a better build system - using Cmake? And Modularity sounded a Good Thing too. Sadly, I think that users haven't seen any benefits - yet. Without some sort of magic apt-get, the only downside of a big Boost package is download time and disk space - neither really a big issue. What it *has* bought us is a much more powerful version control system, but one that is also *much, much, much* more complex. This may mean that some libraries will never be updated as their maintainers never master GIT. I sense a continuing tension between just-a-Standard-library-and-language-extensions Sandpit/Testbed, and a very much large number of larger application-ish packages. The testing and document challenges of these types of libraries are often rather different. Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830

Sadly, I think that users haven't seen any benefits - yet.

I can see some potential benefits though: - Some users make a distinction between good-Boost and bad-Boost: they allow the use of some parts of Boost, but not others. I would think these users are helped by a degree of modularity that makes hem confident that by getting good-Boost they are not pulling in bad-Boost. See for instance the Google Style Guide: http://google-styleguide.googlecode.com/svn/trunk/cppguide.html#Boost - Well defined modularity might also make them less suspicious of parts that are currently considered bad-Boost. - As a Boost user you might find it easier to convince your organization to allow the use of specified parts of Boost rather than Boost-whole. - As a non-Boost library developer you might be more happy to tell your users to get specific parts of Boost, rather than Boost-whole.