Core libraries should separated from experimental libraries
Continued from "Shouldn't both logging proposals be reviewed in the same formal review" Vladimir Batov said:
People will eventually start asking to distribute boost in parts, where the "core" libraries are separated from the "experimental" libraries.
Isn't it already the case? Just put yourself into the shoes of a project manager facing overruns, deadlines, etc. I -- an excited s/w eng. -- come over and say -- there is a library that does what we want. Your first questions will be 1) quality? 2) will the lib. around long enough? 3) will the interface be stable enough? 4) etc.
In that setting I suspect you'll steer clear from anything "inspirational-only".
For boost to remain popular and successful, this issue will eventually have to be addressed. Fortunately, it probably does not need to be addressed for a few years.
I'll probably have to disagree. I'd be more likely leaning towards "addressed now".
I'm actually sympothetic with your point of view and you've certainly made some strong points. My experience has been similar. Nevertheless, no matter how much you wish boost was something different, Boost's mission statement is pretty clear. I dont see that changing anytime soon. You should know that my favorite development library is Gnome (GLIB). http://library.gnome.org/devel/glib/2.22/ Its a very different library, its mission statement clearly states different priorities. Granted its a 'C' library, but its the library that my teams turns to for most of our programming needs. Its not community based like boost is. Not much original thinking is done their, its all about being practical, minimilistic and writing tight code. Boost is different. The libraries here reach for the moon -- some succeed. Boost is where I come for inspiration and there has been some absolutly brillant successes. However, for practical day to day developer needs, I discourage doing too much advanced C++. Neverthless, boost needs to keep on pushing the envelope. Functional programming is the future and boost is where you come to see whats possible in that domain. I would hate to see that change. If boost's focus is switched to addressing the needs of day to day "average" programmers, I would loose interest. I want boost developers to continue to reach for the stars. If "average" developers happen to find boost useful, of course they can use it, but they need to appreciate where boost priorities have been and use it appropriately.
Tom Brinkman wrote:
Neverthless, boost needs to keep on pushing the envelope. Functional programming is the future and boost is where you come to see whats possible in that domain. I would hate to see that change.
If boost's focus is switched to addressing the needs of day to day "average" programmers, I would loose interest. I want boost developers to continue to reach for the stars.
If "average" developers happen to find boost useful, of course they can use it, but they need to appreciate where boost priorities have been and use it appropriately.
+1 Patrick
On 20 Nov 2009, at 18:30, Patrick Horgan wrote:
Tom Brinkman wrote:
Neverthless, boost needs to keep on pushing the envelope. Functional programming is the future and boost is where you come to see whats possible in that domain. I would hate to see that change.
If boost's focus is switched to addressing the needs of day to day "average" programmers, I would loose interest. I want boost developers to continue to reach for the stars.
If "average" developers happen to find boost useful, of course they can use it, but they need to appreciate where boost priorities have been and use it appropriately.
+1
The second line on the boost website is: We emphasize libraries that work well with the C++ Standard Library. Boost libraries are intended to be widely useful, and usable across a broad spectrum of applications. The Boost license encourages both commercial and non-commercial use. The users page says: Why should an organization use Boost? In a word, Productivity. Use of high-quality libraries like Boost speeds initial development, results in fewer bugs, reduces reinvention-of-the-wheel, and cuts long-term maintenance costs. To me, that reads as encouraging average developers to make use of boost in long-term projects. If boost doesn't want to help day-to-day programmers, that's fine, but it shouldn't advertise it self as a library which helps long-term maintenance. Chris
Tom, Sorry for jumping into this thread, but since it has taken a different twist now, and since I have some rather strong feelings with respect to the points you are making here, let me try to address them. In particular, I find the view that you express here very narrow-minded. You construct a false dichotomy: On 11/20/2009 12:48 AM, Tom Brinkman wrote:
Neverthless, boost needs to keep on pushing the envelope. Functional programming is the future and boost is where you come to see whats possible in that domain. I would hate to see that change.
There is definitely more than functional programming in the future. The trend in hardware design certainly raises a lot of challenges to software programming as a whole, and it isn't clear what programming paradigms will yield best results.
If boost's focus is switched to addressing the needs of day to day "average" programmers, I would loose interest. I want boost developers to continue to reach for the stars.
This statement is what I disagree most with. You seem to suggest that the only reason not using boost is because users aren't qualified enough for it. Don't forget that boost's (original) mission statement explicitly is (was) to be a platform for library standardization, which, by definition, addresses 'day to day' needs.
If "average" developers happen to find boost useful, of course they can use it, but they need to appreciate where boost priorities have been and use it appropriately.
And this view I find particularly narrow-minded and condescending. The issues I see with boost have (almost) nothing to do with 'average' vs. 'star' developers, and how much expertise is required to get the most out of boost (though it certainly plays its role). A much bigger issue I see with industry acceptance is code stability (not in terms of bugs, but in terms of backwards compatibility), scalability (boost is growing bigger and bigger, and very few people seem to care about how sustainable this growth is. It's just not true that boost users only pay for what they use), or maintainability (what if I need a bugfix from boost-1.(x+1), but without loosing compatibility with boost-1.x ? What about packaging issues ? I build software that runs on a wide variety of platforms, so the nightmare of figuring out the platform-specific boost dependencies and build issues for all of them is all mine.) Finally, the point triggering your comments was about *core* libraries. Those have been in use for quite a while, and have thus stabilized (if not even being integrated into and or obsoleted by the upcoming standard). Thus, I think the dichotomy is fake: Releasing core components of boost stand-alone (which all that implies: independent release cycle, independent packaging, etc.) does not prevent other "bleeding edge" components to continue to evolve. Quite the opposite, actually. Stefan -- ...ich hab' noch einen Koffer in Berlin...
On Nov 20, 2009, at 1:55 PM, Stefan Seefeld wrote:
On 11/20/2009 12:48 AM, Tom Brinkman wrote:
If "average" developers happen to find boost useful, of course they can use it, but they need to appreciate where boost priorities have been and use it appropriately.
And this view I find particularly narrow-minded and condescending.
You think all developers are at an equal footing? In my limited experience, only having worked in this field for 30 years and led a few hundred developers in total, there is a huge difference between Good and Regular developers. As big as between the a junior nurse and an experienced brain surgeon. If you agree with that, I have another question for you: do you think the exact same tools work equally well for people at these different levels? If not, it is at least fair enough to reason about the intended target of Boost. I fail to see how (i) point out that huge difference in skills (and cognitive performance) between developers, and (ii) suggest that the tools provided by Boost should be for the upper layers, is condescending. So let us get back to the basic question: who the heck is Boost for and for what use? Honestly, real C++ is not for junior developers. Period. Part of the language can be used by a junior developer, and even part of Boost (shared_ptr + some collection types and traits), but most (of C++ - C) cannot.
The issues I see with boost have (almost) nothing to do with 'average' vs. 'star' developers,
You think that distinction is not real?
and how much expertise is required to get the most out of boost (though it certainly plays its role).
Yes, agreed. It does play a huge role, to get outside of shared_ptr and a few collection goodies.
A much bigger issue I see with industry acceptance is code stability (not in terms of bugs, but in terms of backwards compatibility), scalability (boost is growing bigger and bigger, and very few people seem to care about how sustainable this growth is. It's just not true that boost users only pay for what they use), or maintainability (what if I need a bugfix from boost-1.(x+1), but without loosing compatibility with boost-1.x ? What about packaging issues ? I build software that runs on a wide variety of platforms, so the nightmare of figuring out the platform-specific boost dependencies and build issues for all of them is all mine.)
Finally, the point triggering your comments was about *core* libraries. Those have been in use for quite a while, and have thus stabilized (if not even being integrated into and or obsoleted by the upcoming standard). Thus, I think the dichotomy is fake: Releasing core components of boost stand-alone (which all that implies: independent release cycle, independent packaging, etc.) does not prevent other "bleeding edge" components to continue to evolve. Quite the opposite, actually.
I might agree here. The danger I see, although I have been a game developer in a few early lives of mine, is the domain-specific proliferation in 3D and numerics. I have posted about this before. These do not solve day-to-day needs for most developers, not even in a particular skill layer. If the truly domain-agnostic would be considered Core, I would agree with Tom. No matter how complex the notions in those libraries are. /David
2009/11/20 David Bergman <David.Bergman@bergmangupta.com>
You think all developers are at an equal footing? In my limited experience, only having worked in this field for 30 years and led a few hundred developers in total, there is a huge difference between Good and Regular developers.
The difference as I see it is that the good developers are the ones helping everyone, including themselves, to become better developers.
I fail to see how (i) point out that huge difference in skills (and cognitive performance) between developers, and (ii) suggest that the tools provided by Boost should be for the upper layers, is condescending.
Unless you are putting yourself in the lower category, it is condescending. It encourages "appealing to authority" arguments instead of teaching, learning and mentoring.
Honestly, real C++ is not for junior developers. Period. Part of the language can be used by a junior developer, and even part of Boost (shared_ptr + some collection types and traits), but most (of C++ - C) cannot.
Really? What parts can't be used? RAII? Exceptions? Templates? Boost.Bind? Boost.Lambda? Boost.Function? Etc., etc. We use all that stuff (and more), and we (junior and senior developers) are making better designs and writing better code now than when those things weren't available. The real problem is that most developers, junior and senior alike, don't learn how to write good, modern C++. And Boost is a part of modern C++.
The issues I see with boost have (almost) nothing to do with 'average' vs. 'star' developers,
You think that distinction is not real?
I find that more of the issues tend to be because of experience, not talent. The junior people may not know, for instance, the tradeoffs between using an algorithm and a bind expression instead of a hand coded loop. They may not even know enough to consider using the algorithm and bind expression. Junior people might not know why things like Separation of Concerns, or One Class, One Responsibility is important. Etc., etc.
and how much expertise is required to get the most out of boost (though it certainly plays its role).
Yes, agreed. It does play a huge role, to get outside of shared_ptr and a few collection goodies.
An organization that categorizes between "Good" developers and the rest is in danger of topping out at the level of their "Good" developers expertise and biases, because the rest of the developers never get to use what isn't blessed by the "Good" developers. How does an organization like that ever improve? -- Nevin Liber <mailto:nevin@eviloverlord.com> (847) 691-1404 Sent from Chicago, IL, United States
Hi, I am on the Boost packaging team for Debian linux. On Fri, Nov 20, 2009 at 01:55:59PM -0500, Stefan Seefeld wrote:
On 11/20/2009 12:48 AM, Tom Brinkman wrote:
[ ... ]
If "average" developers happen to find boost useful, of course they can use it, but they need to appreciate where boost priorities have been and use it appropriately.
[ ... ]
A much bigger issue I see with industry acceptance is code stability (not in terms of bugs, but in terms of backwards compatibility), scalability (boost is growing bigger and bigger, and very few people seem to care about how sustainable this growth is. It's just not true that boost users only pay for what they use), or maintainability (what if I need a bugfix from boost-1.(x+1), but without loosing compatibility with boost-1.x ? What about packaging issues ? I build software that runs on a wide variety of platforms, so the nightmare of figuring out the platform-specific boost dependencies and build issues for all of them is all mine.)
Indeed, "What about packaging issues?" From my point of view, as part of the Boost packaging team for Debian linux, 4 releases/year is killing me. Debian releases on a 1.5 - 2 year cycle, so we spin through 6-8 Boost releases between stable Debian releases. Other linux distributions, although they release faster, experience similar churn. At this point, let me be perfectly clear: I'm not asking Boost to slow down the release cycle. But let's understand what this rapid release does to the various distributions (linux, BSD, whatever). With no guidance from the Boost maintainers, we will take whatever Boost release is current at the time we freeze. This is the version that is in the field for the next 2 years or so. Ubuntu will do the same, but because they freeze at a different time, often it is a different version of Boost. Same for Red Hat, & etc. This contributes to the nightmare that Stefan describes. What can be done? Part of the issue is the disparate release cycles between the various distributions. This has been described at length by Mark Shuttleworth [1] and there seems to be some movement in this direction between Debian and Ubuntu [2]. The action here is on the distributions. Assuming this can be solved, what can Boost do? One thought is that, if Boost knows that a particular version will be in the field for 2 years, that there could be some maintenance releases done to fix issues with updated compilers or what have you. A lot of such things seem to get fixed in trunk and not propagated to release for some time. Another thought is that mature "core" boost libraries that don't change rapidly could be separated and released on a slower cycle. I'd like to hear the thoughts of the Boost release managers to this kind of thinking. Thanks, -Steve [1] http://lists.debian.org/debian-project/2009/08/msg00092.html [2] http://lists.debian.org/debian-project/2009/11/msg00027.html https://blueprints.edge.launchpad.net/ubuntu/+spec/foundations-lucid-release...
Indeed, "What about packaging issues?"
From my point of view, as part of the Boost packaging team for Debian linux, 4 releases/year is killing me. Debian releases on a 1.5 - 2 year cycle, so we spin through 6-8 Boost releases between stable Debian releases. Other linux distributions, although they release faster, experience similar churn.
At this point, let me be perfectly clear: I'm not asking Boost to slow down the release cycle.
Why not? 2 releases/year doesn't sound that bad to me. However, it would reopen the question of maintenance releases. But at least in theory, it should be easy to create a separate branch for each major release in svn, after it has been released and before the changes for the next major release start to be merged to the release branch from trunk. The argument against this were test resources for the maintenance releases. I have no idea how this test resources issue could be addressed. Regards, Thomas
On 11/21/2009 10:10 AM, Thomas Klimpel wrote:
Indeed, "What about packaging issues?"
At this point, let me be perfectly clear: I'm not asking Boost to slow down the release cycle.
Why not? 2 releases/year doesn't sound that bad to me.
I think the fundamental issue (as far as boost.org is concerned) is not the number of releases, but their relationship (or in fact, the lack of relationship, as it stands) to each other. Without any kind of API or ABI compatibility guarantees (or even promises), I have to consider two distinct boost releases as two entirely different products. Try to look at it that way, and you may understand the concerns I'm talking about. Again, the context of this discussion is "core libraries". I can definitely see the lack of appeal in any sort of compatibility guarantee for highly experimental code. If boost.org started to use a modular approach (even if only as a mental model), may be at least the most widely used boost components could become "stable" in this sense. Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin...
Stefan Seefeld wrote:
Thomas Klimpel wrote:
Why not? 2 releases/year doesn't sound that bad to me.
I think the fundamental issue (as far as boost.org is concerned) is not the number of releases, but their relationship (or in fact, the lack of relationship, as it stands) to each other. Without any kind of API or ABI compatibility guarantees (or even promises), I have to consider two distinct boost releases as two entirely different products.
I have different reasons why I think reducing the frequency of major releases could save some work for different parties. There are some libraries in the sandbox already in active real world use, even so they still have a long way to go before they are ready for a formal review request. In my case it is the numeric_bindings library, but there are many more libraries of this sort. Among others, numeric_bindings depends on Boost.uBlas, which is probably not a "core" library in your view of the world. It's regression tests also depend on Boost.Build. Having to run regression tests against a number of released boost versions in addition to running regression tests against a number of blas/lapack implementations compiled with different fortran compilers is less funny and more time consuming than it looks from the distance. And yes, there were issues, Boost.Build changed incompatible from 1.33 to 1.34, there was a minor bug in 1.35 in a boost library numeric_bindings depended on (sorry, I don't remember exactly which, but I could look it up if it is important to you), there were changes in uBlas that forced some updates to numeric_bindings library, even so it was possible to update it without breaking compatibility with older boost releases. The situation for Glas (Generic Linear Algebra Software) was probably still worse, because it depends in parts on Boost.Numeric.Bindings. It ended up directly including Boost.Numeric.Bindings in its repository, in order to not completely confuse its users. I don't know about the situation of mtl4 (matrix template library), but it also depends on Boost in some way. And I don't think that these numeric libraries are the exception. For example, Boost.Asio was already widely used before acceptance into boost, and the dependence on Boost.Serialization at least caused some nontrivial issues, even if they were no showstoppers in the end. Boost.Cmake decided to mirror the complete release branch to resolve its issues, but this approach doesn't scale to more than one external project.
Try to look at it that way, and you may understand the concerns I'm talking about.
I don't know whether I understand your concerns. I'm just not sure that it will be easy to agree on a set of core libraries. If it were just the official boost libraries themselves, it would at least be easy to figure out which library must necessarily be considered as "core". But you will never know exactly which boost libraries are used by closely boost related projects, like the libraries in the sandbox. For them, the official release boost libraries present the "core" libraries, and the other sandbox libraries are the experimental ones (and as Glas shows, it is quite possible that there is even a dependence to another experimental library).
Again, the context of this discussion is "core libraries". I can definitely see the lack of appeal in any sort of compatibility guarantee for highly experimental code. If boost.org started to use a modular approach (even if only as a mental model), may be at least the most widely used boost components could become "stable" in this sense.
But I think the need for testing with the different releases would not go away. It would even grow, because there would be more combinations to be considered during testing. Regards, Thomas
On 11/22/2009 07:13 AM, Thomas Klimpel wrote:
There are some libraries in the sandbox already in active real world use, even so they still have a long way to go before they are ready for a formal review request. In my case it is the numeric_bindings library, but there are many more libraries of this sort. Among others, numeric_bindings depends on Boost.uBlas, which is probably not a "core" library in your view of the world. It's regression tests also depend on Boost.Build. Having to run regression tests against a number of released boost versions in addition to running regression tests against a number of blas/lapack implementations compiled with different fortran compilers is less funny and more time consuming than it looks from the distance. And yes, there were issues, Boost.Build changed incompatible from 1.33 to 1.34, there was a minor bug in 1.35 in a boost library numeric_bindings depended on (sorry, I don't remember exactly which, but I could look it up if it is important to you), there were changes in uBlas that forced some updates to numeric_bindings library, even so it was possible to update it without breaking compatibility with older boost releases.
Actually, I'm not very fond of the idea of such a simple 'core' / 'non-core' split. What would certainly help is to consider inter-component (library) dependencies, to see which ones are prerequisites for others. 'core' is then merely an alias for those that don't depend on 'non-core', are themselves depended on, and are stable.
I'm just not sure that it will be easy to agree on a set of core libraries.
Probably not, but I'm not sure that such difficulties are a reason not even to start. Spinning off stable components into their own (build-,package-,release-)procedure is an incremental process, and doesn't require everybody to agree upfront what components should go through this in the long run. Stefan -- ...ich hab' noch einen Koffer in Berlin...
Stefan Seefeld wrote:
I'm just not sure that it will be easy to agree on a set of core libraries.
Probably not, but I'm not sure that such difficulties are a reason not even to start. Spinning off stable components into their own (build-,package-,release-)procedure is an incremental process, and doesn't require everybody to agree upfront what components should go through this in the long run.
The only point I definitively don't like about this is the own release procedure for the stable components (test, build and package seem OK). Even if there should be a "core" package and a "complete" package, both should still be strictly released together. And I'm also strictly against dreaming that the "core" package will offer significantly more guarantees than the current boost releases. And keep in mind that relaxing the guarantees for the "complete" package might make it easier to add a library like numeric_bindings to it, which is difficult to test on all potential target systems. Currently, libraries like Boost.MPI or Boost.Phyton have the full responsibility to ensure that they can be tested with reasonable effort on all potential target systems. So dreaming about relaxing the guarantees for the "complete" package is probably not a good idea either. Regards, Thomas
On 11/22/2009 02:18 PM, Thomas Klimpel wrote:
The only point I definitively don't like about this is the own release procedure for the stable components (test, build and package seem OK). Even if there should be a "core" package and a "complete" package, both should still be strictly released together. And I'm also strictly against dreaming that the "core" package will offer significantly more guarantees than the current boost releases.
Wait. You'd like to have releases bundle both, and you don't want to provide distinct guarantees. What's the point in this split, then ? Stefan -- ...ich hab' noch einen Koffer in Berlin...
Tom Brinkman wrote:
However, for practical day to day developer needs, I discourage doing too much advanced C++.
What is considered advanced? Is it shared_ptr/weak_ptr? Is it Boost.MultiIndex or Boost.Spirit? What would you suggest to use instead of these tools that I employ on almost a daily basis? Would my code be better in any way had I written all these tools in C again and again? Sorry, I don't think so.
Neverthless, boost needs to keep on pushing the envelope. Functional programming is the future and boost is where you come to see whats possible in that domain. I would hate to see that change.
C++ is good because it allows to use different programming concepts, and functional programming is only one of them. Limiting yourself to only one paradigm in C++ seems unwise to me.
If boost's focus is switched to addressing the needs of day to day "average" programmers, I would loose interest. I want boost developers to continue to reach for the stars.
If "average" developers happen to find boost useful, of course they can use it, but they need to appreciate where boost priorities have been and use it appropriately.
Well, programming is a practical art. Boost is not a playground for a bunch of C++ gurus that are being bored. There are no libraries in Boost that "reach the stars" without a pure boring average task to solve behind it. Every library is intended to address practical needs of developers, otherwise it is no more than a useless junk. Boost is good because it addresses a diverse spectrum of these tasks, and often does it in the most elegant way. Yes, in many ways it's experimental and innovative, as it often introduces nonstandard solutions and pushes compilers to their limits. But that being said, it _is_ intended and _should_ be used in the wild, including commercial software, because it's the only way to verify whether these solutions are actually successful or not. That is why I see Boost support (including documentation, fixing bugs, simplifying deployment, maintaining API stability, etc., etc.) to be one of the top priorities.
On Fri, Nov 20, 2009 at 5:26 PM, Andrey Semashev <andrey.semashev@gmail.com>wrote:
Well, programming is a practical art. Boost is not a playground for a bunch of C++ gurus that are being bored. There are no libraries in Boost that "reach the stars" without a pure boring average task to solve behind it. Every library is intended to address practical needs of developers, otherwise it is no more than a useless junk.
Boost is good because it addresses a diverse spectrum of these tasks, and often does it in the most elegant way. Yes, in many ways it's experimental and innovative, as it often introduces nonstandard solutions and pushes compilers to their limits. But that being said, it _is_ intended and _should_ be used in the wild, including commercial software, because it's the only way to verify whether these solutions are actually successful or not. That is why I see Boost support (including documentation, fixing bugs, simplifying deployment, maintaining API stability, etc., etc.) to be one of the top priorities.
Well said! --Beman
participants (10)
-
Andrey Semashev -
Beman Dawes -
Christopher Jefferson -
David Bergman -
Nevin Liber -
Patrick Horgan -
Stefan Seefeld -
Steve M. Robbins -
Thomas Klimpel -
Tom Brinkman