[lib maintainers] Please identify headers in boost-root/boost
For the headers that appear in boost-root/boost, rather than a library specific subdirectory, it is very helpful if they self-identify what library they belong to. This allows scripts to associate the header with a library. The usual way to do this is to include a comment. A typical example: // See http://www.boost.org/libs/config for documentation The exact format doesn't matter. What does matter is that the library name be found somewhere in the file, immediately following a string which contains "/libs/". Below is the result of doing: grep -L "/libs/" *.h* I'd appreciate maintainers adding a library identifying comment to these files. Otherwise I'll do it, which carries a slight risk that I'll get it wrong. Thanks, --Beman aligned_storage.hpp asio.hpp bimap.hpp blank.hpp blank_fwd.hpp circular_buffer.hpp circular_buffer_fwd.hpp date_time.hpp dynamic_property_map.hpp foreach.hpp function_equal.hpp function_output_iterator.hpp get_pointer.hpp implicit_cast.hpp is_placeholder.hpp iterator_adaptors.hpp lexical_cast.hpp mpi.hpp non_type.hpp none.hpp none_t.hpp optional.hpp parameter.hpp pfto.hpp pointer_cast.hpp program_options.hpp smart_cast.hpp spirit.hpp state_saver.hpp strong_typedef.hpp thread.hpp type.hpp variant.hpp wave.hpp
For the headers that appear in boost-root/boost, rather than a library specific subdirectory, it is very helpful if they self-identify what library they belong to. This allows scripts to associate the header with a library.
wave.hpp
Done. Regards Hartmut
Hartmut Kaiser wrote:
For the headers that appear in boost-root/boost, rather than a library specific subdirectory, it is very helpful if they self-identify what library they belong to. This allows scripts to associate the header with a library.
wave.hpp
Done.
Thanks! --Beman
Beman Dawes wrote:
For the headers that appear in boost-root/boost, rather than a library specific subdirectory, it is very helpful if they self-identify what library they belong to. This allows scripts to associate the header with a library.
The usual way to do this is to include a comment. A typical example:
// See http://www.boost.org/libs/config for documentation
The exact format doesn't matter. What does matter is that the library name be found somewhere in the file, immediately following a string which contains "/libs/".
Below is the result of doing:
grep -L "/libs/" *.h*
I'd appreciate maintainers adding a library identifying comment to these files. Otherwise I'll do it, which carries a slight risk that I'll get it wrong.
I've now updated many of the headers that didn't have such a comment. There were a bunch (see below) where it wasn't obvious to me where they were documented. I'd appreciate it if the authors would add a comment identifying the library where the docs live. If there aren't any docs, write something! Use libs/utility as the home for anything not associated with a particular library. Thanks, --Beman aligned_storage.hpp blank.hpp blank_fwd.hpp dynamic_property_map.hpp function_equal.hpp function_output_iterator.hpp get_pointer.hpp implicit_cast.hpp is_placeholder.hpp non_type.hpp pfto.hpp pointer_cast.hpp smart_cast.hpp state_saver.hpp strong_typedef.hpp type.hpp
Beman Dawes wrote:
Robert Ramey wrote:
smart_cast.hpp state_saver.hpp strong_typedef.hpp pfto.hpp The above are documented in the serialization library. Each one has its own page.
Thanks, Robert.
SVN updated.
This is somewhat discomforting. Isn't there a policy for placing headers in the root boost directory? Can anyone simply place whetever he/she wants there? The files above look like small utilities. As such, they are not fast track reviewed, AFAICT. That goes the same for the boost namespace. IMO, we need a clear policy on what goes there. Lest, we'll have it filled up and polluted with lotsa stuff before we know it. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
on Sat Nov 17 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
Beman Dawes wrote:
Robert Ramey wrote:
smart_cast.hpp state_saver.hpp strong_typedef.hpp pfto.hpp The above are documented in the serialization library. Each one has its own page.
Thanks, Robert.
SVN updated.
This is somewhat discomforting. Isn't there a policy for placing headers in the root boost directory?
There is a standard practice that was established by tradition and consensus, but no written policy.
Can anyone simply place whetever he/she wants there? The files above look like small utilities. As such, they are not fast track reviewed, AFAICT.
Unfortunately, the standard practice was not followed by everyone, and although moderators have made explicit requests that these headers be brought into conformance with that practice, our requests have been ignored.
That goes the same for the boost namespace. IMO, we need a clear policy on what goes there. Lest, we'll have it filled up and polluted with lotsa stuff before we know it.
Yes, well, now we have a precedent that violates what is, IMO, the only reasonable policy. That makes it a little harder to make the policy official. But anyway, I have a strong suspicion that you and I substantially agree on that policy, so I would be very happy if you'd write something up. Then we have something concrete to argue about ;-) -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
A little greping reveals the following headers which would seem to violate "standard practice". I've tried to exclude those which are "convenience headers" which do nothing but call a bunch of includes for some library. boost/aligned_storage.hpp boost/any.hpp boost/array.hpp boost/assert.hpp boost/bind.hpp boost/blank.hpp boost/blank_fwd.hpp boost/call_traits.hpp boost/cast.hpp boost/checked_delete.hpp boost/concept_archetype.hpp boost/concept_check.hpp boost/config.hpp boost/cstdint.hpp boost/current_function.hpp boost/dynamic_bitset_fwd.hpp boost/dynamic_property_map.hpp boost/enable_shared_from_this.hpp boost/function.hpp boost/function_equal.hpp boost/functional.hpp boost/get_pointer.hpp boost/implicit_cast.hpp boost/indirect_reference.hpp boost/int_iterator.hpp boost/integer.hpp boost/integer_fwd.hpp boost/integer_traits.hpp boost/intrusive_ptr.hpp boost/io_fwd.hpp boost/iterator.hpp boost/iterator_adaptors.hpp boost/last_value.hpp boost/lexical_cast.hpp boost/limits.hpp boost/math_fwd.hpp boost/mem_fn.hpp boost/next_prior.hpp boost/non_type.hpp boost/noncopyable.hpp boost/nondet_random.hpp boost/none.hpp boost/operators.hpp boost/optional.hpp boost/pfto.hpp boost/pointee.hpp boost/preprocessor.hpp boost/property_map.hpp boost/property_map_iterator.hpp boost/random.hpp boost/ref.hpp boost/regex.hpp boost/scoped_ptr.hpp boost/shared_array.hpp boost/shared_ptr.hpp boost/smart_cast.hpp boost/smart_ptr.hpp boost/static_assert.hpp boost/static_warning.hpp boost/strong_typedef.hpp boost/throw_exception.hpp boost/tokenizer.hpp boost/type.hpp boost/type_traits.hpp boost/utility.hpp boost/variant.hpp boost/version.hpp boost/visit_each.hpp boost/weak_ptr.hpp Lots of these have been there for years. So I'm not sure where the idea that there exists a "standard practice" comes from. In my particular case, I had a need for some things like static_warning and strong_typedef which didn't exist. I didn't put them where they are currently found by accident. I knew they didn't fit in the serialization library and it seemed they were very similar to other files in the above list. I made a manual page for each of them. There wasn't a convenient place to put the documenation so I left it as "misc" in the serialization documentation. The serialization library was reviewed TWICE and no one ever mentioned this. So rather than saying that authors have refused to follow "standard practice", it would be more accurate to say that there has been no definition of "standard practice" and many authors have interpreted it in the most logical way given the current situation. Now if one want's to define a "standard practice" that's fine. And clearly boost is a lot bigger than it used to be so the best practice may well be different than it used to be. But it has to be recognized that this is a whole new thing. And its not clear what that new thing should be? While we're at it, there is also the question of boost/utility. Clearly there is useful stuff in there. I've used it myself. But non of it has any documentation nor separate tests as far as I can tell. (ucs code cvt facet is tested and documented in the serialization library). I think the only practical thing to do at this point is a) debate and establish a policy for using boost/ and boost:: b) enforce it for new additions c) hope that over time the current contents migrate out Robert Ramey David Abrahams wrote:
on Sat Nov 17 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
Beman Dawes wrote:
Robert Ramey wrote:
smart_cast.hpp state_saver.hpp strong_typedef.hpp pfto.hpp The above are documented in the serialization library. Each one has its own page.
Thanks, Robert.
SVN updated.
This is somewhat discomforting. Isn't there a policy for placing headers in the root boost directory?
There is a standard practice that was established by tradition and consensus, but no written policy.
Can anyone simply place whetever he/she wants there? The files above look like small utilities. As such, they are not fast track reviewed, AFAICT.
Unfortunately, the standard practice was not followed by everyone, and although moderators have made explicit requests that these headers be brought into conformance with that practice, our requests have been ignored.
That goes the same for the boost namespace. IMO, we need a clear policy on what goes there. Lest, we'll have it filled up and polluted with lotsa stuff before we know it.
Yes, well, now we have a precedent that violates what is, IMO, the only reasonable policy. That makes it a little harder to make the policy official. But anyway, I have a strong suspicion that you and I substantially agree on that policy, so I would be very happy if you'd write something up. Then we have something concrete to argue about ;-)
Robert Ramey wrote:
A little greping reveals the following headers which would seem to violate "standard practice". I've tried to exclude those which are "convenience headers" which do nothing but call a bunch of includes for some library.
[...]
Lots of these have been there for years. So I'm not sure where the idea that there exists a "standard practice" comes from.
First, define "standard practice". I can't say that all the files you listed violates the "standard practice" if we haven't defined it yet. I am not saying that your files are violations of the practice too. I just said that it is somewhat discomforting. I am not pinpointing serialization.
In my particular case, I had a need for some things like static_warning and strong_typedef which didn't exist. I didn't put them where they are currently found by accident. I knew they didn't fit in the serialization library and it seemed they were very similar to other files in the above list. I made
So, if I decide that some parts of spirit (a lot! e.g. multi_pass iterator) is useful outside spirit and does not really belong in spirit, I'm free to put that outside the spirit sphere and into boost? That does not make sense. So what's the need for fast-track reviews if I can easily put anything into the root directory and into the main boost namespace?
a manual page for each of them. There wasn't a convenient place to put the documenation so I left it as "misc" in the serialization documentation.
Why not put it in serialization "misc" then? In my libraries, we have such utilities in "support".
The serialization library was reviewed TWICE and no one ever mentioned this.
It's unfortunate that this has gone unnoticed from 2 reviews. Perhaps it didn't matter much at the time, or simply it slipped from the radar screen. Nevertheless, that does not make it right.
So rather than saying that authors have refused to follow "standard practice", it would be more accurate to say that there has been no definition of "standard practice" and many authors have interpreted it in the most logical way given the current situation.
Agreed. I'm not quite sure about the "most logical" way though. For me, it's pretty clear that free-for-all addition of just about anything in the boost root directory and the boost namespace is not good. There are namespaces and subdirectories.
Now if one want's to define a "standard practice" that's fine. And clearly boost is a lot bigger than it used to be so the best practice may well be different than it used to be. But it has to be recognized that this is a whole new thing. And its not clear what that new thing should be?
While we're at it, there is also the question of boost/utility. Clearly there is useful stuff in there. I've used it myself. But non of it has any documentation nor separate tests as far as I can tell. (ucs code cvt facet is tested and documented in the serialization library).
Really? enable_if, addressof, result_of, at least, have docs and tests. I do not see ucs code cvt facet there. What am I missing?
I think the only practical thing to do at this point is
a) debate and establish a policy for using boost/ and boost:: b) enforce it for new additions
Encourage developers of libraries in violation of the policy to do the right thing.
c) hope that over time the current contents migrate out
Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman wrote:
Robert Ramey wrote:
A little greping reveals the following headers which would seem to violate "standard practice". I've tried to exclude those which are "convenience headers" which do nothing but call a bunch of includes for some library.
[...]
Lots of these have been there for years. So I'm not sure where the idea that there exists a "standard practice" comes from.
First, define "standard practice". I can't say that all the files you listed violates the "standard practice" if we haven't defined it yet.
aaaaaaaa - that's my point. There has never been any "standard practice defined"
In my particular case, I had a need for some things like static_warning and strong_typedef which didn't exist. I didn't put them where they are currently found by accident. I knew they didn't fit in the serialization library and it seemed they were very similar to other files in the above list. I made
So, if I decide that some parts of spirit (a lot! e.g. multi_pass iterator) is useful outside spirit and does not really belong in spirit, I'm free to put that outside the spirit sphere and into boost?
Up until now that's been the case.
Why not put it in serialization "misc" then? In my libraries, we have such utilities in "support".
To make such a move at this point it would be an interface breaking change. How much code would break if boost/shared_ptr (and boost::shared_ptr etc) is changed at this point? Do you really think that all these changes should be made now?
The serialization library was reviewed TWICE and no one ever mentioned this.
It's unfortunate that this has gone unnoticed from 2 reviews. Perhaps it didn't matter much at the time, or simply it slipped from the radar screen. Nevertheless, that does not make it right.
I doubt it went unnoticed. I suspect that it was percieved as the "standard practice" at the time.
So rather than saying that authors have refused to follow "standard practice", it would be more accurate to say that there has been no definition of "standard practice" and many authors have interpreted it in the most logical way given the current situation.
Agreed. I'm not quite sure about the "most logical" way though. For me, it's pretty clear that free-for-all addition of just about anything in the boost root directory and the boost namespace is not good. There are namespaces and subdirectories.
As I said, I don't object to establishing a "standard practice". But it's really annoying to be characterised as being unwilling to respect a "standard practice" when no such thing has been established.
While we're at it, there is also the question of boost/utility. Clearly there is useful stuff in there. I've used it myself. But non of it has any documentation nor separate tests as far as I can tell. (ucs code cvt facet is tested and documented in the serialization library).
My mistake - I meant boost/detail.
Really? enable_if, addressof, result_of, at least, have docs and tests.
LOL - yeah but the directory structure doesn't respect the practice used by other libraries. And that's only 3 modules about of
I do not see ucs code cvt facet there. What am I missing?
the header is in boost/detail. It should have gone into boost/utility but there was a hassle because it wasn't reviewed. Before that the same code was in multiple libraries and there was an objection to that. No matter what - we can't get a break. So it went into boost/detail and guess what - there is no place to put documentation or tests for anything in boost/detail. Now I see that boost/utility/multi-pass has a documentation page but no way to get to by following the links in the documentation index. Then there is utility.hpp which is a convenience header for a group of otherwise separate library - not including mult-index. So any kind of "standard practice" should also normalize all of this as well. Fine by me.
I think the only practical thing to do at this point is
a) debate and establish a policy for using boost/ and boost::
and boost/detail and boost/utility
b) enforce it for new additions
Encourage developers of libraries in violation of the policy to do the right thing.
c) hope that over time the current contents migrate out
So I guess we're in agreement. Robert Ramey
Robert Ramey wrote:
Joel de Guzman wrote:
In my particular case, I had a need for some things like static_warning and strong_typedef which didn't exist. I didn't put them where they are currently found by accident. I knew they didn't fit in the serialization library and it seemed they were very similar to other files in the above list. I made
So, if I decide that some parts of spirit (a lot! e.g. multi_pass iterator) is useful outside spirit and does not really belong in spirit, I'm free to put that outside the spirit sphere and into boost?
Up until now that's been the case.
Well, this IMO has got to stop.
Why not put it in serialization "misc" then? In my libraries, we have such utilities in "support".
To make such a move at this point it would be an interface breaking change. How much code would break if boost/shared_ptr (and boost::shared_ptr etc) is changed at this point? Do you really think that all these changes should be made now?
No, of course not. But was has boost/shared_ptr got to do with this?
The serialization library was reviewed TWICE and no one ever mentioned this.
It's unfortunate that this has gone unnoticed from 2 reviews. Perhaps it didn't matter much at the time, or simply it slipped from the radar screen. Nevertheless, that does not make it right.
I doubt it went unnoticed. I suspect that it was percieved as the "standard practice" at the time.
It went unnoticed at least for me. I would have objected otherwise.
So rather than saying that authors have refused to follow "standard practice", it would be more accurate to say that there has been no definition of "standard practice" and many authors have interpreted it in the most logical way given the current situation.
Agreed. I'm not quite sure about the "most logical" way though. For me, it's pretty clear that free-for-all addition of just about anything in the boost root directory and the boost namespace is not good. There are namespaces and subdirectories.
As I said, I don't object to establishing a "standard practice". But it's really annoying to be characterised as being unwilling to respect a "standard practice" when no such thing has been established.
I don't think anyone is pinpointing you as unwilling to respect "standard practice".
While we're at it, there is also the question of boost/utility. Clearly there is useful stuff in there. I've used it myself. But non of it has any documentation nor separate tests as far as I can tell. (ucs code cvt facet is tested and documented in the serialization library).
My mistake - I meant boost/detail.
Really? enable_if, addressof, result_of, at least, have docs and tests.
LOL - yeah but the directory structure doesn't respect the practice used by other libraries. And that's only 3 modules about of
I do not see ucs code cvt facet there. What am I missing?
the header is in boost/detail. It should have gone into boost/utility but there was a hassle because it wasn't reviewed. Before that the same code was in multiple libraries and there was an objection to that. No matter what - we can't get a break. So it went into boost/detail and guess what - there is no place to put documentation or tests for anything in boost/detail.
Yes, because boost/detail is not public API and the stuff there is implementation detail.
Now I see that boost/utility/multi-pass has a documentation page but no way to get to by following the links in the documentation
I'm lost again. boost/utility/multi-pass is non-existent.
index. Then there is utility.hpp which is a convenience header for a group of otherwise separate library - not including mult-index.
If a particular utility is useful for many libraries, then one of the libraries can host it prior to it getting reviewed independently and becoming a full-fledged boost citizen. That's the policy that we've been following with the synergy between xpressive and spirit, for example. Case in point: fusion, proto, multi_pass, and many more smaller utilities and stuff. fusion was hosted by spirit before it was reviewed and was used by xpressive even before spirit did. Now that fusion passed the boost review, it has become a full-fledged boost citizen.
So any kind of "standard practice" should also normalize all of this as well. Fine by me.
I think the only practical thing to do at this point is
a) debate and establish a policy for using boost/ and boost::
and boost/detail and boost/utility
b) enforce it for new additions Encourage developers of libraries in violation of the policy to do the right thing.
c) hope that over time the current contents migrate out
So I guess we're in agreement.
That's a good start. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman wrote:
No, of course not. But was has boost/shared_ptr got to do with this?
wow - this is the iconic example of a violation of "standard practice" The file is implemented in boost/shared_ptr.hpp and shared_ptr class is to be found in namespace boost. This is not what one would expect to find given that the documentation is smart_pointers ...
The serialization library was reviewed TWICE and no one ever mentioned this.
It's unfortunate that this has gone unnoticed from 2 reviews. Perhaps it didn't matter much at the time, or simply it slipped from the radar screen. Nevertheless, that does not make it right.
I doubt it went unnoticed. I suspect that it was percieved as the "standard practice" at the time.
It went unnoticed at least for me. I would have objected otherwise.
So rather than saying that authors have refused to follow "standard practice", it would be more accurate to say that there has been no definition of "standard practice" and many authors have interpreted it in the most logical way given the current situation.
Agreed. I'm not quite sure about the "most logical" way though. For me, it's pretty clear that free-for-all addition of just about anything in the boost root directory and the boost namespace is not good. There are namespaces and subdirectories.
As I said, I don't object to establishing a "standard practice". But it's really annoying to be characterised as being unwilling to respect a "standard practice" when no such thing has been established.
I don't think anyone is pinpointing you as unwilling to respect "standard practice".
Yes, because boost/detail is not public API and the stuff there is implementation detail.
Hmm - and implementation details shouldn't be documented? shouldn't be tested? If they're in a common area it seems that they are meant to be used by more than one library - and indeed they are. lightweight test others come to mind. Its even worse. Someone uses an "implementation detail" from this public area. and the author decides to make an interface breaking change. This breaks a lot of stuff and the author says - what's the problem? its and implemenation detail. Another case of ill defined "standard practice" creating problems.
Now I see that boost/utility/multi-pass has a documentation page but no way to get to by following the links in the documentation
I'm lost again. boost/utility/multi-pass is non-existent.
There is libs/utility/Mult-PassIterator.html that isn't linked to anything. I jumped to the conclusion that corresponded to a corresponding module in boost/utility - sorry. I'm not sure what "standard practice" would indicate should be done with this - if anything.
index. Then there is utility.hpp which is a convenience header for a group of otherwise separate library - not including mult-index.
If a particular utility is useful for many libraries, then one of the libraries can host it prior to it getting reviewed independently and becoming a full-fledged boost citizen. That's the policy that we've been following with the synergy between xpressive and spirit,
I faced exactly that same issue with headers such as static_warning.hpp There was already a header in boost/static_assert.hpp. I thought (and still think) that logical place for static_warning.hpp was in that same directory. I considered the possibility of placing it in boost/serialization/static_warning.hpp but at the time I thought I might have to change it later to boost/static_warning.hpp sometime in the near future and I wanted to avoid an interface breaking change.
for example. Case in point: fusion, proto, multi_pass, and many more smaller utilities and stuff. fusion was hosted by spirit before it was reviewed and was used by xpressive even before spirit did. Now that fusion passed the boost review, it has become a full-fledged boost citizen.
hmmm - I looked for it in boost on the trunk and 1.34 and didn't find either documentation for it nor code. I don't doubt its in there somewhere but I can't find it. Wouldn't "standard practice" suggest that I look in boost/utility or perhaps boost/iterators ? Hmmm has is been separately reviewed. All these examples demonstrate that there really is no defined "standard practice" and never has been. Robert Ramey
Robert Ramey wrote:
Joel de Guzman wrote:
Hmm - and implementation details shouldn't be documented? shouldn't be tested? If they're in a common area it seems that they are meant to be used by more than one library - and indeed they are. lightweight test others come to mind.
If they are public or protected they should be documented for end-users and other developers respectively. What should be tested is the public interface, which is normally enough to test whether the protected and private implementation is working properly.
Robert Ramey wrote:
Joel de Guzman wrote:
No, of course not. But was has boost/shared_ptr got to do with this?
wow - this is the iconic example of a violation of "standard practice"
Why are you saying that while insisting that there's no "standard practice"?
The file is implemented in boost/shared_ptr.hpp and shared_ptr class is to be found in namespace boost. This is not what one would expect to find given that the documentation is smart_pointers ...
Yes, because boost/detail is not public API and the stuff there is implementation detail.
Hmm - and implementation details shouldn't be documented? shouldn't be tested? If they're in a common area it seems that they are meant to be used by more than one library - and indeed they are. lightweight test others come to mind.
Its even worse. Someone uses an "implementation detail" from this public area. and the author decides to make an interface breaking change. This breaks a lot of stuff and the author says - what's the problem? its and implemenation detail.
I am not saying that. But this is irrelevant to the matter at hand.
Another case of ill defined "standard practice" creating problems.
Perhaps, but complain/object if you will in another thread. This is irrelevant to what I am objecting to.
If a particular utility is useful for many libraries, then one of the libraries can host it prior to it getting reviewed independently and becoming a full-fledged boost citizen. That's the policy that we've been following with the synergy between xpressive and spirit,
I faced exactly that same issue with headers such as static_warning.hpp There was already a header in boost/static_assert.hpp. I thought (and still think) that logical place for static_warning.hpp was in that same directory. I considered the possibility of placing it in
If everyone thinks the way you do, then we're in trouble. Imagine hundreds of header files in the root directory and thousands of components in the boost namespace. Project that with the rate boost is growing! Man!
boost/serialization/static_warning.hpp but at the time I thought I might have to change it later to boost/static_warning.hpp sometime in the near future and I wanted to avoid an interface breaking change.
Wrong decision! It would be the other way around.
for example. Case in point: fusion, proto, multi_pass, and many more smaller utilities and stuff. fusion was hosted by spirit before it was reviewed and was used by xpressive even before spirit did. Now that fusion passed the boost review, it has become a full-fledged boost citizen.
hmmm - I looked for it in boost on the trunk and 1.34 and didn't find either documentation for it nor code. I don't doubt its in there somewhere but I can't find it. Wouldn't "standard practice" suggest that I look in boost/utility or perhaps boost/iterators ? Hmmm has is been separately reviewed.
All these examples demonstrate that there really is no defined "standard practice" and never has been.
What documentation are you looking for? Fusion: it's in the main libraries.html, proto: it's in xpressive (its host) multi_pass: it's in spirit (its host). Can't see the pattern? It's obvious. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman wrote:
Robert Ramey wrote:
Joel de Guzman wrote:
No, of course not. But was has boost/shared_ptr got to do with this?
wow - this is the iconic example of a violation of "standard practice"
Why are you saying that while insisting that there's no "standard practice"?
There is none - that's why I put it in quotes. "standard practice" is a fluid definition in this context. The latest version cited now seems to depend on something referred to as "core libraries" whatever that means.
Another case of ill defined "standard practice" creating problems.
Perhaps, but complain/object if you will in another thread. This is irrelevant to what I am objecting to.
What I'm objecting to is that an author works hard to follow the stated standards for documentation and tries to discern the "standard practice" from examples, and submits his library for review, addresses all issues raised, and after years, has to deal with strident objections which could have and should have been raised during the review and could easily have been addressed at that time. Its unprofessional and boost has lots of examples of this. So for me, its not a question so much of this or that directory structure but the whole idea that its ok to change the rules in the middle of the game. It wastes huge amounts of time.
boost/serialization/static_warning.hpp but at the time I thought I might have to change it later to boost/static_warning.hpp sometime in the near future and I wanted to avoid an interface breaking change.
Wrong decision! It would be the other way around.
Hmmm - well, maybe we'll just submit static_warning.hpp for a fast track review and be done with it.
Robert Ramey wrote:
Joel de Guzman wrote:
Joel de Guzman wrote:
No, of course not. But was has boost/shared_ptr got to do with this? wow - this is the iconic example of a violation of "standard practice" Why are you saying that while insisting that there's no "standard
Robert Ramey wrote: practice"?
There is none - that's why I put it in quotes. "standard practice" is a fluid definition in this context. The latest version cited now seems to depend on something referred to as "core libraries" whatever that means.
We can't argue that way. Give me your definition of "standard practice" first. Seems I'm looking at Mars and you might be looking at Jupiter. To me, it's clear that there's standard practice. The boost community has been here a long time and the conventions are pretty much clear by just looking at the libraries before (i.e. I learned from Dan Nuffer who did the initial boost-ification of Spirit prior to its review). To me, the idea of "core" libraries has been very clear right from the start. So, I'm curious. Is there a precedent that prompted you to place static_warning in the root boost directory and in namespace boost?
Another case of ill defined "standard practice" creating problems.
Perhaps, but complain/object if you will in another thread. This is irrelevant to what I am objecting to.
What I'm objecting to is that an author works hard to follow the stated standards for documentation and tries to discern the "standard practice" from examples, and submits his library for review, addresses all issues raised, and after years, has to deal with strident objections which could have and should have been raised during the review and could easily have been addressed at that time. Its unprofessional and boost has lots of examples of this.
It's not perfect, for sure. Discussions like this, hopefully, should get the issues straightened. Again, I'm curious, it seems (you say) that you based your judgement from discerning standard practice from existing libraries. So which is it that you based placing static_warning in the root boost directory and in namespace boost on? Please understand that I want to look at this constructively and objectively.
So for me, its not a question so much of this or that directory structure but the whole idea that its ok to change the rules in the middle of the game. It wastes huge amounts of time.
I disagree. IMO, no-one is changing the rules in the middle of the game. It's just that you did something that slipped the radar screen. Our task now is to make sure that this does not happen. And, I agree with you that we should have some kind of policy written.
boost/serialization/static_warning.hpp but at the time I thought I might have to change it later to boost/static_warning.hpp sometime in the near future and I wanted to avoid an interface breaking change. Wrong decision! It would be the other way around.
Hmmm - well, maybe we'll just submit static_warning.hpp for a fast track review and be done with it.
Cool! Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman wrote:
No, of course not. But was has boost/shared_ptr got to do with this? wow - this is the iconic example of a violation of "standard practice" Why are you saying that while insisting that there's no "standard practice"?
There is none - that's why I put it in quotes. "standard practice" is a fluid definition in this context. The latest version cited now seems to depend on something referred to as "core libraries" whatever that means.
We can't argue that way. Give me your definition of "standard practice" first. Seems I'm looking at Mars and you might be looking at Jupiter.
I'm not the one claiming there there exists an unambiguous definition of "standard practice" in this context so that's why I haven't claimed I followed it. My claim is that its undefined and being stretched and distorted to fit with every post on this thread. I don't know what it is. I don't know where its defined. That's why I had to try to extract from the example of the way boost was organized at the time. Given that is the way I had to do it, its no wonder at least some people think I got it wrong. Given the lack of definition - its inevitable that there is going to be disagreement on some decision which is supposedly based upon it. And BTW - boost is riddled with this problem. The same sorts of issues plague documentation standards, usage of boost/detail, release and testing procedures.
So, I'm curious. Is there a precedent that prompted you to place static_warning in the root boost directory and in namespace boost?
I just followed the example of static_assert.hpp. I was new to boost so I wasn't aware of any precedent. I followed what seemed to me to be an inescapably obvious pattern. Also the WHOLE serialization library was subject to review. Everything was open to question. The first review produced a rejection and a very long list of things that were unacceptable. The next version addressed all the issues raised and the library was accepted with very little change. It was my understanding that the review was to cover all aspects of the library. I could just as well argue that those components were reviewed. I'm not going to do that however. The serialization library is a large library and really needed some things that boost didn't have like strongtypedef, extended_typeinfo, static_warning, and a couple of others I forgot. They really arn't part of the library - just things boost was missing. So I put them where other similar components were found and no one objected neither then nor for years afterwards. And these things weren't hidden. They were prominantly featured in the documentation as separate components with links to the true path.
It's not perfect, for sure. Discussions like this, hopefully, should get the issues straightened. Again, I'm curious, it seems (you say) that you based your judgement from discerning standard practice from existing libraries. So which is it that you based placing static_warning in the root boost directory and in namespace boost on? Please understand that I want to look at this constructively and objectively.
I appreciate that. See above. Also consider the context. Source code, headers, test programs and documentation of the serialization library I believe is on the order of 30,000 lines of text. (we'll exclude the email discussions, reviews etc.) A header like static_warning is maybe 100 lines. I saw a very close analogy, leveraged on it, and moved on. Without clearer more explicity policy, one doesn't have much other alternative. Actually, considering the scale of what was involved, the possible misplacement of a small number of headers (8?) is very small potatoes and has been blown way out of proportion. And considering boost's other problems, release procedures, documentation build, out of date documentation standards, components used but not formally tested, testing on just a subset of build combinations, bjam, etc. It's microscopic potatoes.
So for me, its not a question so much of this or that directory structure but the whole idea that its ok to change the rules in the middle of the game. It wastes huge amounts of time.
I disagree. IMO, no-one is changing the rules in the middle of the game. It's just that you did something that slipped the radar screen.
LOL - we could disagree on this, but as a practical matter the effect is the same.
Our task now is to make sure that this does not happen. And, I agree with you that we should have some kind of policy written.
Halleluhuh How about this as a policy: a) No new headers in boost/... Boost is too big for this b) small libraries can be placed in boost/utility and ancillary files such as tests, source code, documentation, etc be placed in the same spots they are as other libraries c) No concept of "core libraries" vs "non-core" libraries d) exception to a) one convenience header per library e) documentation in boost/libs/utility/doc/... same as other libraries f) library authors encourged to move components over time out of boost. This would include ALL the ones I posted before (except for those which are convenience headers).
Hmmm - well, maybe we'll just submit static_warning.hpp for a fast track review and be done with it.
I haven't done this up until now for a couple of reasons. I thought it mostly redundant. It takes a lot of time on my part. It would take time from other things I thought were alot more urgent and important. these would be state_saver.hpp, static_warning.hpp, strongtypedef.hpp. The other ones of mine are pfto.hpp, smart_cast.hpp, and I would be inclined to migrate them out at a convenient time in the future. So you would only have about 100 others to deal with. Robert Ramey
Cool!
Regards,
Robert Ramey wrote:
I'm not the one claiming there there exists an unambiguous definition of "standard practice" in this context so that's why I haven't claimed I followed it. My claim is that its undefined and being stretched and distorted to fit with every post on this thread.
The concept of "core libraries" is not new. Search the mailing list and you'll see what I mean. I don't see it being stretched to fit the posts. I wasn't even aware of the exchanges between you and Dave a year ago. I noticed the misplaced header and raised my concern.
I don't know what it is. I don't know where its defined. That's why I had to try to extract from the example of the way boost was organized at the time. Given that is the
You mention below that you based that action by following static_assert.hpp. You see now why it is wrong? There's a big difference -- static_assert was reviewed. Yours is not. To be considered a boost library, it must be reviewed independently outside of serialization.
way I had to do it, its no wonder at least some people think I got it wrong.
Who? Which libraries? If there are any, I'll raise the same objection.
Given the lack of definition - its inevitable that there is going to be disagreement on some decision which is supposedly based upon it.
And BTW - boost is riddled with this problem. The same sorts of issues plague documentation standards, usage of boost/detail, release and testing procedures.
So, let's work together to fix them. Resistance to change something that is broken does not help. You do agree that free for all pollution of the root directory and namespace is not good, right?
So, I'm curious. Is there a precedent that prompted you to place static_warning in the root boost directory and in namespace boost?
I just followed the example of static_assert.hpp. I was new to boost so I wasn't aware of any precedent. I followed what seemed to me to be an inescapably obvious pattern.
Ok. Let's not fight on this anymore. At least we are in agreement that something must be done to correct this. That's good enough.
Also the WHOLE serialization library was subject to review. Everything was open to question. The first review produced a rejection and a very long list of things that were unacceptable. The next version addressed all the issues raised and the library was accepted with very little change. It was my understanding that the review was to cover all aspects of the library. I could just as well argue that those components were reviewed. I'm not going to do that however. The serialization library is a large library and really needed some things that boost didn't have like strongtypedef, extended_typeinfo, static_warning, and a couple of others I forgot. They really arn't part of the library - just things boost was missing. So I put them where other similar components were found and no one objected neither then nor for years afterwards. And these things weren't hidden. They were prominantly featured in the documentation as separate components with links to the true path.
Again, that's very unfortunate. We are humans and glitches like this do happen. There's never a perfect library. The important thing is to be able to accept mistakes like this. It is a mistake. We can argue as much as we want, it is still a mistake. Let's not focus on why the mistake was committed but rather on how to correct it.
It's not perfect, for sure. Discussions like this, hopefully, should get the issues straightened. Again, I'm curious, it seems (you say) that you based your judgement from discerning standard practice from existing libraries. So which is it that you based placing static_warning in the root boost directory and in namespace boost on? Please understand that I want to look at this constructively and objectively.
I appreciate that. See above. Also consider the context. Source code, headers, test programs and documentation of the serialization library I believe is on the order of 30,000 lines of text. (we'll exclude the email discussions, reviews etc.) A header like static_warning is maybe 100 lines. I saw a very close analogy, leveraged on it, and moved on. Without clearer more explicity policy, one doesn't have much other alternative. Actually, considering the scale of what was involved, the possible misplacement of a small number of headers (8?) is very small potatoes and has been blown way out of proportion. And considering boost's other problems, release procedures, documentation build, out of date documentation standards, components used but not formally tested, testing on just a subset of build combinations, bjam, etc. It's microscopic potatoes.
It's not microscopic for me. It sets a bad precedent. If anyone can do as you did, then there'll be chaos, if not now, then surely in the future. Anyway, at least you now say it's misplaced. That's good enough. Let's work together to put them in the proper place.
So for me, its not a question so much of this or that directory structure but the whole idea that its ok to change the rules in the middle of the game. It wastes huge amounts of time. I disagree. IMO, no-one is changing the rules in the middle of the game. It's just that you did something that slipped the radar screen.
LOL - we could disagree on this, but as a practical matter the effect is the same.
Our task now is to make sure that this does not happen. And, I agree with you that we should have some kind of policy written.
Halleluhuh
How about this as a policy:
a) No new headers in boost/... Boost is too big for this
Forward headers are ok. I also don't see a problem with reviewed libraries which have a small single header to be placed there. Ideally, the header would follow the pattern: lib.hpp where "lib" is the name of the reviewed library.
b) small libraries can be placed in boost/utility and ancillary files such as tests, source code, documentation, etc be placed in the same spots they are as other libraries
As long as they passed the review process.
c) No concept of "core libraries" vs "non-core" libraries
You'll have a hard time defending that. The concept of "core library" has been with us since time immemorial. Just search the lists. Why are you so against this?
d) exception to a) one convenience header per library
Ah, ok.
e) documentation in boost/libs/utility/doc/... same as other libraries
again, as long as those are reviewed.
f) library authors encourged to move components over time out of boost. This would include ALL the ones I posted before (except for those which are convenience headers).
Tweak: encourged to move *unreviewed* components over time out of boost.
Hmmm - well, maybe we'll just submit static_warning.hpp for a fast track review and be done with it.
I haven't done this up until now for a couple of reasons. I thought it mostly redundant. It takes a lot of time on my part. It would take time from other things I thought were alot more urgent and important.
these would be state_saver.hpp, static_warning.hpp, strongtypedef.hpp. The other ones of mine are pfto.hpp, smart_cast.hpp, and I would be inclined to migrate them out at a convenient time in the future. So you would only have about 100 others to deal with.
Why not ask for a review for all these in one shot. Dunno if that's possible though. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
On Mon, 19 Nov 2007 16:05:25 +0800, Joel de Guzman wrote:
It's not microscopic for me. It sets a bad precedent. If anyone can do as you did, then there'll be chaos, if not now, then surely in the future. Anyway, at least you now say it's misplaced. That's good enough. Let's work together to put them in the proper place.
Hi friend :-) In my experience, it is very easy to get worked up after the "crime" and fix the problem while pointing a finger. But it isn't very fair. I'm sure Robert didn't intentionally mean to violate any conventions, even if he felt they were undocumented. It might be a good opportunity to document and agree upon these conventions. -- Sohail Somani http://uint32t.blogspot.com
Sohail Somani wrote:
On Mon, 19 Nov 2007 16:05:25 +0800, Joel de Guzman wrote:
It's not microscopic for me. It sets a bad precedent. If anyone can do as you did, then there'll be chaos, if not now, then surely in the future. Anyway, at least you now say it's misplaced. That's good enough. Let's work together to put them in the proper place.
Hi friend :-)
In my experience, it is very easy to get worked up after the "crime" and fix the problem while pointing a finger. But it isn't very fair. I'm sure Robert didn't intentionally mean to violate any conventions, even if he felt they were undocumented. It might be a good opportunity to document and agree upon these conventions.
Yeah, it's not fair to say that Robert committed a crime when there's no law to begin with. I too am sure that Robert didn't intentionally mean to violate anything. All I am saying is that what occurred is a mistake and should be corrected. I am sure everyone agrees on that. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman wrote:
Robert Ramey wrote:
You mention below that you based that action by following static_assert.hpp. You see now why it is wrong? There's a big difference -- static_assert was reviewed. Yours is not. To be considered a boost library, it must be reviewed independently outside of serialization.
So its not the placement itself that's wrong? The error is is only that it reviewed as part of something else rather than independently? So the correctness of the placement of such a module is defined in terms of the outcome of a formal review? Does that seem like a practical system to you? Suppose static_warning.hpp were reviewed and for some reason it was rejected and it were moved into the serialization library. Then we would have the situation where two very similar libraries would be in two entirely different places. Would this be logical from the standpoint of someone examining boost and trying to understand where stuff is? Should review results be included as permanent part of every library? Note that these are rhetorical questions meant to illustrate my view that defining the "expected placement" or "standard whatever" which depends upon the result of a formal review (especially when examination of such a review) has some problems. What is better is a stated policy and a review acceptance process which enforces conformance to such a policy. Perhaps had such a policy existed at the time and the formal review process stated that comformance to such a policy was one of the things to be checked, this particular instance might be been addressed at an opportune time.
way I had to do it, its no wonder at least some people think I got it wrong.
Who? Which libraries? If there are any, I'll raise the same objection.
I previously posted the following list. I'm not sure which ones are convenience headers and which are not. But somehow I don't think they've all been independently reviewed. given that there in the boost directory I don't know where to look for the documentation without out doing some sort of search. (Hmmm - now I'm curious what something like "none.hpp" does. ahhh - its parto of optional.hpp.) boost/aligned_storage.hpp boost/any.hpp boost/array.hpp boost/assert.hpp boost/bind.hpp boost/blank.hpp boost/blank_fwd.hpp boost/call_traits.hpp boost/cast.hpp boost/checked_delete.hpp boost/concept_archetype.hpp boost/concept_check.hpp boost/config.hpp boost/cstdint.hpp boost/current_function.hpp boost/dynamic_bitset_fwd.hpp boost/dynamic_property_map.hpp boost/enable_shared_from_this.hpp boost/function.hpp boost/function_equal.hpp boost/functional.hpp boost/get_pointer.hpp boost/implicit_cast.hpp boost/indirect_reference.hpp boost/int_iterator.hpp boost/integer.hpp boost/integer_fwd.hpp boost/integer_traits.hpp boost/intrusive_ptr.hpp boost/io_fwd.hpp boost/iterator.hpp boost/iterator_adaptors.hpp boost/last_value.hpp boost/lexical_cast.hpp boost/limits.hpp boost/math_fwd.hpp boost/mem_fn.hpp boost/next_prior.hpp boost/non_type.hpp boost/noncopyable.hpp boost/nondet_random.hpp boost/none.hpp boost/operators.hpp boost/optional.hpp boost/pfto.hpp boost/pointee.hpp boost/preprocessor.hpp boost/property_map.hpp boost/property_map_iterator.hpp boost/random.hpp boost/ref.hpp boost/regex.hpp boost/scoped_ptr.hpp boost/shared_array.hpp boost/shared_ptr.hpp boost/smart_cast.hpp boost/smart_ptr.hpp boost/static_assert.hpp boost/static_warning.hpp boost/strong_typedef.hpp boost/throw_exception.hpp boost/tokenizer.hpp boost/type.hpp boost/type_traits.hpp boost/utility.hpp boost/variant.hpp boost/version.hpp boost/visit_each.hpp boost/weak_ptr.hpp
Given the lack of definition - its inevitable that there is going to be disagreement on some decision which is supposedly based upon it.
And BTW - boost is riddled with this problem. The same sorts of issues plague documentation standards, usage of boost/detail, release and testing procedures.
So, let's work together to fix them.
well, I'm trying to put my 2 cents in.
Resistance to change something that is broken does not help. You do agree that free for all pollution of the root directory and namespace is not good, right?
I do - I was just trying to follow established patterns. But now you raise the real issue. Given the size that boost has grown to and the problems that this has engendered I don't think that there should be anything in the boost directory/namespace other than convenience headers. (personally I don't even like convenience headers but I'm think I'm in the minory on this point.)
Again, that's very unfortunate. We are humans and glitches like this do happen. There's never a perfect library. The important thing is to be able to accept mistakes like this. It is a mistake. We can argue as much as we want, it is still a mistake. Let's not focus on why the mistake was committed but rather on how to correct it.
Hmmm - maybe
It's not perfect, for sure. Discussions like this, hopefully, should get the issues straightened. Again, I'm curious, it seems (you say) that you based your judgement from discerning standard practice from existing libraries. So which is it that you based placing static_warning in the root boost directory and in namespace boost on? Please understand that I want to look at this constructively and objectively.
I appreciate that. See above. Also consider the context. Source code, headers, test programs and documentation of the serialization library I believe is on the order of 30,000 lines of text. (we'll exclude the email discussions, reviews etc.) A header like static_warning is maybe 100 lines. I saw a very close analogy, leveraged on it, and moved on. Without clearer more explicity policy, one doesn't have much other alternative. Actually, considering the scale of what was involved, the possible misplacement of a small number of headers (8?) is very small potatoes and has been blown way out of proportion. And considering boost's other problems, release procedures, documentation build, out of date documentation standards, components used but not formally tested, testing on just a subset of build combinations, bjam, etc. It's microscopic potatoes.
It's not microscopic for me. It sets a bad precedent.
I didn't set precedent, I followed it
If anyone can do as you did, then there'll be chaos,
lots of people did and there is.
if not now, then > surely in the future. Anyway, at least you now say it's misplaced. That's good enough. Let's work together to put them in the proper place.
Define a policy that is implementable. Then you won't have the problem.
How about this as a policy:
a) No new headers in boost/... Boost is too big for this
Forward headers are ok. I also don't see a problem with reviewed libraries which have a small single header to be placed there. Ideally, the header would follow the pattern:
lib.hpp
where "lib" is the name of the reviewed library.
I see a big problem. Defining the directory placement in terms of a review means that developers can't really make the library until after its review. Its like sending every court case to the legislature. It means extra work for developers. It means that one doesn't really know where to look for something.
b) small libraries can be placed in boost/utility and ancillary files such as tests, source code, documentation, etc be placed in the same spots they are as other libraries
As long as they passed the review process.
Actually everything in boost should be subject to some sort of review process. And it is even though its sometimes sort of informal for smaller changes and additions.
c) No concept of "core libraries" vs "non-core" libraries
You'll have a hard time defending that. The concept of "core library" has been with us since time immemorial. Just search the lists. Why are you so against this?
For a couple of reasons: a) it's subjective and arbitrary - (one more think to be decided by commitee) b) it's unnecessary - (the concept doesn't add anything) c) it's confusiing - one looks for something in different places depending on whether one thinks its a "core library" or not. d) once I do #include boost/none.hpp its not obvious anymore what other libraries my own code depends upon. e) it's not scalable. This is the big one. As boost gets bigger the number of "core libraries" (regardless of how its defined) will have to grow. This will make all the problems it generates that much worse. f) To me its inevitable that boost will have to move more toward the spirit model of development. That is, larger libraries, and maybe all of them, being an asyncronous process which depends upon the latest release of other libraries. I know there is resistence to this, but I think that it is inevitable regardless. having stuff in boost/... make this process more painful than it already its. g) I'm already having problems with namespace clashes. I used Multi_Index and it used aligned_storage. Some sort of namespace issue which seems in this case to be precipated by a compiler bug. So maybe its not the best example but it took a fair amount of time to track down. Its very confusing to have assert.hpp in boost/... as well as in some other libraries - and can lead to hard to find bugs.
d) exception to a) one convenience header per library
Ah, ok.
LOL - and I don't even like convenience headers myself.
e) documentation in boost/libs/utility/doc/... same as other libraries
again, as long as those are reviewed.
well everything is reviewed. The questions at hand would be a) If a new library has a component which is of such a nature that it really stands apart from the library itself, is it OK that it be placed in a more central spot like boost/utility. b) Is it OK for review of such a component to be handled as part of the review for the whole library. c) Does acceptance of a library imply acceptance of such a component or not. a) would be a question of policy. As these things will vary from case to case, b) and c) things should be added to the review manager's checklist.
f) library authors encourged to move components over time out of boost. This would include ALL the ones I posted before (except for those which are convenience headers).
Tweak: encourged to move *unreviewed* components over time out of boost.
ReTweak: *ALL*
Why not ask for a review for all these in one shot. Dunno if that's possible though.
LOL - that's what I thought I was getting when I went through the TWO orginal reviews. Robert Ramey
on Mon Nov 19 2007, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
So its not the placement itself that's wrong? The error is is only that it reviewed as part of something else rather than independently? So the correctness of the placement of such a module is defined in terms of the outcome of a formal review? Does that seem like a practical system to you?
Robert, the system has worked very well until recently.
Suppose static_warning.hpp were reviewed and for some reason it was rejected and it were moved into the serialization library. Then we would have the situation where two very similar libraries would be in two entirely different places. Would this be logical from the standpoint of someone examining boost and trying to understand where stuff is?
It may in fact be suboptimally logical from that point of view. In my opinion there are advantages that outweigh those disadvantages, but that's not really important right now. We can debate the system as it has existed all you like, but until there is consensus that it needs to change, we ask that you try to follow the previous practices as they have been laid out for you. The Boost policy gives library authors a huge amount of freedom to organize and evolve their libraries as they see fit. Once his library has passed review, a library author can add huge new functionality at any time, or decide to break backward compatibility when necessary, or restructure the library's headers without a review. We only ask that libraries stay within the boundaries of a boost/ subdirectory and boost:: namespace named after the library, with the exception of consolidated forwarding headers and small core libraries. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
on Mon Nov 19 2007, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
Joel de Guzman wrote:
Robert Ramey wrote:
I previously posted the following list. I'm not sure which ones are convenience headers and which are not. But somehow I don't think they've all been independently reviewed.
As I mentioned in http://lists.boost.org/Archives/boost/2006/05/105195.php, there are *very few* exceptions to the set of rules I laid out there. Some of those exceptions predate the use of subnamespaces and subdirectories in Boost. If you don't believe me, rather than speculating, do the search for review remarks and you will see. And yes, a few exceptions do exist. Some mistakes were made before you came along.
I didn't set precedent, I followed it
You misinterpreted the existing pattern and thus broke with precedent.
If anyone can do as you did, then there'll be chaos,
lots of people did and there is.
No, lots of people did not. There was a small amount of disorder before serialization was added; it looks nothing like chaos once you understand the pattern that existed (which I laid out in the message cited above). In repeatedly citing what you consider to be precedents for what you have done, lack of documentation, and what you consider to be flaws in the current system, you seem to hope to avoid dealing with the request to begin bringing your library into line with the usual Boost practice. Please stop; it's a big waste of everyone's time. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
Robert Ramey wrote:
Joel de Guzman wrote:
Robert Ramey wrote:
You mention below that you based that action by following static_assert.hpp. You see now why it is wrong? There's a big difference -- static_assert was reviewed. Yours is not. To be considered a boost library, it must be reviewed independently outside of serialization.
So its not the placement itself that's wrong?
I think so, yes. static_warning can very well be a "core" library like static_assert IFF it has undergone a boost review and is accepted.
The error is is only that it reviewed as part of something else rather than independently?
Yes. That's the crucial misinterpretation of the process and the (unwritten) standard practice. The libraries I authored, for instance, has tremendous amount of support code that ideally could exist outside the framework as stand alone components. These parts have undergone the same review process as serialization components did. If I, or all libraries that has been accepted thus far, were to place them outside the sphere of their host into root, imagine the result.
So the correctness of the placement of such a module is defined in terms of the outcome of a formal review?
Correctness of a lot of things depend *not only* on the formal review. We can never have a perfect system.
Does that seem like a practical system to you?
Well, unless you have a better suggestion, the current review process works. It's up to us to uncover the glitches and propose remedies and workarounds or, if you will, better processes.
Suppose static_warning.hpp were reviewed and for some reason it was rejected and it were moved into the serialization library. Then we would have the situation where two very similar libraries would be in two entirely different places. Would this be logical from the standpoint of someone examining boost and trying to understand where stuff is?
No. And that is unfortunate.
Should review results be included as permanent part of every library?
Not sure about that. At least an immediately accessible record of all the reviews would be helpful, I think.
Note that these are rhetorical questions meant to illustrate my view that defining the "expected placement" or "standard whatever" which depends upon the result of a formal review (especially when examination of such a review) has some problems. What is better is a stated policy and a review acceptance process which enforces conformance to such a policy. Perhaps had such a policy existed at the time and the formal review process stated that comformance to such a policy was one of the things to be checked, this particular instance might be been addressed at an opportune time.
Agreed. Still, even after we settle this issue for now, there will still be things that we will not catch and even more things that will be uncovered in the future that we've missed yet may be detrimental to the health of boost in the future to come. There's no silver bullet. We can only try to do our best, as we always have. What's equally important is that we remain agile to future changes and accept that we do make mistakes, we do uncover unforeseen bugs. And it is part of the ongoing process to correct them when we find them.
I previously posted the following list. I'm not sure which ones are convenience headers and which are not. But somehow I don't think they've all been independently reviewed. given that there in the boost directory I don't know where to look for the documentation without out doing some sort of search. (Hmmm - now I'm curious what something like "none.hpp" does. ahhh - its parto of optional.hpp.)
[snip list] Once we settle this, we can run through the list and inform the author of the violation, if there is one. I notice though that most of the items in your list have been reviewed. Can you please trim your list to only those like static_warning? none.hpp is probably one of them, AFAIK.
Resistance to change something that is broken does not help. You do agree that free for all pollution of the root directory and namespace is not good, right?
I do - I was just trying to follow established patterns. But now you raise the real issue. Given the size that boost has grown to and the problems that this has engendered I don't think that there should be anything in the boost directory/namespace other than convenience headers. (personally I don't even like convenience headers but I'm think I'm in the minory on this point.)
In my mind, that would be ideal, yes. But I'm not sure it's practical as this would cause massive breaking for existing libraries.
lots of people did and there is.
I don't think it's chaos at this point. There's still chance to correct the trend. I'll reply to the rest of your post in a new thread later. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Robert Ramey wrote:
a) No new headers in boost/... Boost is too big for this Forward headers are ok. I also don't see a problem with reviewed libraries which have a small single header to be placed there. Ideally, the header would follow the pattern:
lib.hpp
where "lib" is the name of the reviewed library.
I see a big problem. Defining the directory placement in terms of a review means that developers can't really make the library until after its review. Its like sending every court case to the legislature. It means extra work for developers. It means that one doesn't really know where to look for something.
I don't quite understand what you mean here. Please elaborate.
b) small libraries can be placed in boost/utility and ancillary files such as tests, source code, documentation, etc be placed in the same spots they are as other libraries As long as they passed the review process.
Actually everything in boost should be subject to some sort of review process. And it is even though its sometimes sort of informal for smaller changes and additions.
No, that would be too slow and bureaucratic. Libraries can add their own modules and upgrade as needed without having to undergo another review. Individual libraries, can host such utilities or even sub-libraries until they are reviewed separately. Only then can they be promoted outside its host and into boost.
c) No concept of "core libraries" vs "non-core" libraries You'll have a hard time defending that. The concept of "core library" has been with us since time immemorial. Just search the lists. Why are you so against this?
For a couple of reasons:
a) it's subjective and arbitrary - (one more think to be decided by commitee)
It can be part of the review. If the author of a library intends it to be a "core" library, he can explicitly say so and be subject for the review.
b) it's unnecessary - (the concept doesn't add anything)
It does. The shared_ptr is a core library. I do not want to spell it out as boost::smart_ptr::shared_ptr. It's pretty much common to all libraries. Also, it gives us structure. Typically "core" libraries have the highest number of dependency in terms of structure. If you look at the dependency diagram of boost, the core libraries will be at the bottommost with lots of other modules pointing acyclically at it. It's a must to avoid having them depend on other libraries in other layers above the core. It's good that smart folks like Peter Dimov knows what they are doing by keeping the strict acyclic relationship. More emphasis and constraints must be given to these "core" libraries as they form the backbone of boost as a whole. I do not want to touch on that, however, to not digress from the subject.
c) it's confusiing - one looks for something in different places depending on whether one thinks its a "core library" or not.
Subjective.
d) once I do #include boost/none.hpp its not obvious anymore what other libraries my own code depends upon.
Is none.hpp a "core" library? Was it individually reviewed? If not, then it's probably in violation too.
e) it's not scalable. This is the big one. As boost gets bigger the number of "core libraries" (regardless of how its defined) will have to grow. This will make all the problems it generates that much worse.
It is scalable. Surely it will grow, but its size will be a fraction of the whole of boost. Not a lot of libraries can be considered core. Only a few.
f) To me its inevitable that boost will have to move more toward the spirit model of development. That is, larger libraries, and maybe all of them, being an asyncronous process which depends upon the latest release of other libraries. I know there is resistence to this, but I think that it is inevitable regardless. having stuff in boost/... make this process more painful than it already its.
Spirit itself has a "core" and such structuring makes a lot of sense in terms of organization. All libraries I authored have the concept of a "core". Ok, this may be besides the point, but I just want to emphasize that I am all for structuring and layering. Actually, I am for having more than the core. Right now, We have utility and core. That's a good start. But to give you an idea, Spirit has this organizational structure: http://tinyurl.com/ywr2fo Such a layering structure is very scalable. It is also very intuitive to just look at it, reason out, etc. in a high level, if you have such a layering structure in place.
g) I'm already having problems with namespace clashes. I used Multi_Index and it used aligned_storage. Some sort of namespace issue which seems in this case to be precipated by a compiler bug. So maybe its not the best example but it took a fair amount of time to track down. Its very confusing to have assert.hpp in boost/... as well as in some other libraries - and can lead to hard to find bugs.
These symptoms are not caused by having a "core". Irrelevant.
d) exception to a) one convenience header per library Ah, ok.
LOL - and I don't even like convenience headers myself.
e) documentation in boost/libs/utility/doc/... same as other libraries
again, as long as those are reviewed.
well everything is reviewed. The questions at hand would be
a) If a new library has a component which is of such a nature that it really stands apart from the library itself, is it OK that it be placed in a more central spot like boost/utility.
That question should be part of the review. The author should state her wish that the library be part of 'core" or "utility". Ultimately, the review manager decides.
b) Is it OK for review of such a component to be handled as part of the review for the whole library.
Perhaps not. That's a question best addressed by the review folks.
c) Does acceptance of a library imply acceptance of such a component or not.
No.
a) would be a question of policy. As these things will vary from case to case, b) and c) things should be added to the review manager's checklist.
f) library authors encourged to move components over time out of boost. This would include ALL the ones I posted before (except for those which are convenience headers). Tweak: encourged to move *unreviewed* components over time out of boost.
ReTweak: *ALL*
Why not ask for a review for all these in one shot. Dunno if that's possible though.
LOL - that's what I thought I was getting when I went through the TWO orginal reviews.
Yes. Given the considerations above. I see it's not a good idea. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Robert, Please try to read the response below neutrally, if at all possible. It is not intended as a retort or an admonishment. My intent is only to bring this argument to a successful conclusion for everyone. on Mon Nov 19 2007, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
I'm not the one claiming there there exists an unambiguous definition of "standard practice" in this context so that's why I haven't claimed I followed it.
Nobody is claiming there has been an unambiguous definition.
My claim is that its undefined and being stretched and distorted to fit with every post on this thread.
I gave you a summary of the Boost header placement customs in http://lists.boost.org/Archives/boost/2006/05/105195.php and before that, more than a year earlier, in http://lists.boost.org/Archives/boost/2006/05/105195.php I don't think anyone has made claims that contradict these summaries. If they have, I'd like to try to clarify, so please point to the specific "definition stretching" if you really think there has been. Otherwise, I'm happy to drop this issue.
I don't know what it is. I don't know where its defined.
Are the above summaries reasonably usable as definitions? Do they leave you with any questions?
That's why I had to try to extract from the example of the way boost was organized at the time. Given that is the way I had to do it, its no wonder at least some people think I got it wrong. Given the lack of definition - its inevitable that there is going to be disagreement on some decision which is supposedly based upon it.
Absolutely. So it happened, and now we are going to correct the lack-of-documentation side of the problem. Are you willing to correct your side of the problem? This is not a question of whether your past actions are condemnable (they are not) but whether you now have the information needed to understand the usual Boost practices. Thanks, -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
on Sun Nov 18 2007, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
What I'm objecting to is that an author works hard to follow the stated standards for documentation and tries to discern the "standard practice" from examples, and submits his library for review, addresses all issues raised, and after years, has to deal with strident objections which could have and should have been raised during the review and could easily have been addressed at that time. Its unprofessional and boost has lots of examples of this.
In the case of your library review, the people participating simply didn't notice the problem. I don't see how you can claim there's anything unprofessional about that, especially in an open-source project where nobody's getting paid for his work. I'm truly sorry if you felt anyone's objections were strident. I know that I in particular sometimes have a way of putting people on the defensive, and I regret that. However, I do believe all the requests made were reasonable and there's no reason we shouldn't be able to make progress on them. As I wrote in http://lists.boost.org/Archives/boost/2006/05/105195.php, I realized "that these problems can't be repaired all at once, but they should be fixed. I'd start by pushing boost/pfto.hpp into boost/detail, for example." -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
Robert Ramey wrote:
Yes, because boost/detail is not public API and the stuff there is implementation detail.
Hmm - and implementation details shouldn't be documented? shouldn't be tested? If they're in a common area it seems that they are meant to be used by more than one library - and indeed they are. lightweight test others come to mind.
The way I approached this in the Boost.Math update was to place "documented details" in a sub-namespace of boost::math. Not sure if it's ideal, but it does keep everything nice and tidy :-)
If a particular utility is useful for many libraries, then one of the libraries can host it prior to it getting reviewed independently and becoming a full-fledged boost citizen. That's the policy that we've been following with the synergy between xpressive and spirit,
I faced exactly that same issue with headers such as static_warning.hpp There was already a header in boost/static_assert.hpp. I thought (and still think) that logical place for static_warning.hpp was in that same directory. I considered the possibility of placing it in boost/serialization/static_warning.hpp but at the time I thought I might have to change it later to boost/static_warning.hpp sometime in the near future and I wanted to avoid an interface breaking change.
Sure, but static_assert has been through a review and is documented as a library in it's own right. static_warning should probably have gone down the same route: probably a fast-track review for a small component like this would suffice.
All these examples demonstrate that there really is no defined "standard practice" and never has been.
Please give us credit for trying though: yes we have legacy headers in boost/ that maybe shouldn't be there, but that doesn't mean that new stuff should be a free for all ! :-) Regards, John Maddock.
on Sun Nov 18 2007, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
In my particular case, I had a need for some things like static_warning and strong_typedef which didn't exist. I didn't put them where they are currently found by accident. I knew they didn't fit in the serialization library and it seemed they were very similar to other files in the above list. I made
So, if I decide that some parts of spirit (a lot! e.g. multi_pass iterator) is useful outside spirit and does not really belong in spirit, I'm free to put that outside the spirit sphere and into boost?
Up until now that's been the case.
In general, no, it has not been the case. This may not be a perfect arrangement, but with very few exceptions: * Core libraries whose documented public interface fits in just a few headers and a few pages of documentation have been able to put all their public headers in boost/ and their components in boost::. * Larger libraries have sometimes placed a single consolidated convenience header in boost/, forwarding to files of the form boost/<libraryname>/<header>.hpp. Regardless, their documented public components are not in boost/ or namespace boost:: * Un-reviewed implementation details of libraries have been placed in boost/detail if they need to be used by other libraries and a subdirectory of boost/<libraryname>/ otherwise I explained most of this in http://lists.boost.org/Archives/boost/2006/05/105195.php
So rather than saying that authors have refused to follow "standard practice", it would be more accurate to say that...
Nobody said that. What I said was that some people have ignored direct (public and private) appeals from moderators to bring their headers into conformance with the standard practice, and that is exactly correct. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
In general, no, it has not been the case. This may not be a perfect arrangement, but with very few exceptions:
* Core libraries whose documented public interface fits in just a few headers and a few pages of documentation have been able to put all their public headers in boost/ and their components in boost::.
Ahh - "Core libraries" - perhaps that's the missing link. So static_assert.hpp would be a "core library" but static_warning.hpp would be a ..... non-core library?
* Un-reviewed implementation details of libraries have been placed in boost/detail if they need to be used by other libraries and a subdirectory of boost/<libraryname>/ otherwise
Which is another problem in my view. If they're meant to be used by more than one library they should have a documented and tested interface. Otherwise, they should be in the particular library even at the cost of code repetition.
I explained most of this in http://lists.boost.org/Archives/boost/2006/05/105195.php
and I responded at the time. well, as far as pfto.hpp is concerned, I concede an error in foresight on my part. I envisioned that as a general fix for those compilers which failed to correctly implement partial function template ordering and thought it orthogonal to the serialization library as a whole. As the whole serialization library was subject to review - including this point and no objection was raised at the time - i left it there. BTW it IS documented and its NOT really part a part of serialization per se. Had I known that boost would relatively soon start restricting itself to compilers which don't manifest this problem, I would have taken a different decision.
So rather than saying that authors have refused to follow "standard practice", it would be more accurate to say that...
Nobody said that.
from http://lists.boost.org/Archives/boost/2006/05/105200.php
These unconventional moves are at best confusing for users and other maintainers. The Boost source base is hard enough to control without library authors inventing their own new rules for organizing things.
What I said was that some people have ignored direct (public and private) appeals from moderators to bring their headers into conformance with the standard practice, and that is exactly correct.
a) There is no stated "standard practice" so it cannot be correct. b) There is no obvious "convention" in many areas c) Authors are left to try to decipher intent from the current hodgepodge. I believe that all authors have tried to do this d) One purpose of the review process is to raise these issues at a proper time e) Authors (including myself) have been very willing to address all issues raised during a review. The can't be faulted years later for failing to adhere to some undefined "standard practive" or "convention" which has never been stated nor is obvious. Robert Ramey
on Sun Nov 18 2007, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
David Abrahams wrote:
In general, no, it has not been the case. This may not be a perfect arrangement, but with very few exceptions:
* Core libraries whose documented public interface fits in just a few headers and a few pages of documentation have been able to put all their public headers in boost/ and their components in boost::.
Ahh - "Core libraries" - perhaps that's the missing link. So static_assert.hpp would be a "core library"
Yes.
but static_warning.hpp would be a ..... non-core library?
No, it's not a Boost library at all unless it has ever passed review.
* Un-reviewed implementation details of libraries have been placed in boost/detail if they need to be used by other libraries and a subdirectory of boost/<libraryname>/ otherwise
Which is another problem in my view. If they're meant to be used by more than one library they should have a documented and tested interface. Otherwise, they should be in the particular library even at the cost of code repetition.
Irrelevant; we can talk about that policy idea in another thread.
I explained most of this in http://lists.boost.org/Archives/boost/2006/05/105195.php
and I responded at the time.
In approximately the same unhelpful way as you are responding now.
well, as far as pfto.hpp is concerned, I concede an error in foresight on my part. I envisioned that as a general fix for those compilers which failed to correctly implement partial function template ordering and thought it orthogonal to the serialization library as a whole. As the whole serialization library was subject to review - including this point and no objection was raised at the time - i left it there.
I am neither criticizing your foresight, nor whatever honest mistakes you may have made in the past. I _am_ criticizing the way you react to polite requests to change what you've done.
BTW it IS documented and its NOT really part a part of serialization per se.
Until it is reviewed as a separate library, it is part of serialization.
So rather than saying that authors have refused to follow "standard practice", it would be more accurate to say that...
Nobody said that.
from http://lists.boost.org/Archives/boost/2006/05/105200.php
That link does not seem in the least to refute my claim.
These unconventional moves are at best confusing for users and other maintainers. The Boost source base is hard enough to control without library authors inventing their own new rules for organizing things.
What I said was that some people have ignored direct (public and private) appeals from moderators to bring their headers into conformance with the standard practice, and that is exactly correct.
a) There is no stated "standard practice" so it cannot be correct.
Look up "practice" in your dictionary; it means "custom." A "standard practice" does not have to be documented. To wit http://www.answers.com/topic/practice says, among others: A habitual or customary action or act. Often used in the plural: That company engages in questionable business practices. Facial tattooing is a standard practice among certain peoples.
b) There is no obvious "convention" in many areas
You've been told what the convention is on several occasions.
c) Authors are left to try to decipher intent from the current hodgepodge. I believe that all authors have tried to do this
Anyone who is curious knows where to ask. It would certainly be better to document it. Lack of documentation doesn't make it alright for library authors to disregard requests to conform, especially when the requests come from (multiple) moderators. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
on Sat Nov 17 2007, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
A little greping reveals the following headers which would seem to violate "standard practice". I've tried to exclude those which are "convenience headers" which do nothing but call a bunch of includes for some library.
That's been a standard practice for many years. What we have never done intentionally is to place components that are not a part of a library's documented public interface in namespace boost, nor their headers in boost/. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
That goes the same for the boost namespace. IMO, we need a clear policy on what goes there. Lest, we'll have it filled up and polluted with lotsa stuff before we know it.
Yes, well, now we have a precedent that violates what is, IMO, the only reasonable policy. That makes it a little harder to make the policy official. But anyway, I have a strong suspicion that you and I substantially agree on that policy, so I would be very happy if you'd write something up. Then we have something concrete to argue about ;-)
Ok, I'll take this one. I see you've got the "standard practice" defined in another post. I'll try to collect all the relevant information from the current discussions and collect them into a document. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman wrote:
David Abrahams wrote:
That goes the same for the boost namespace. IMO, we need a clear policy on what goes there. Lest, we'll have it filled up and polluted with lotsa stuff before we know it. Yes, well, now we have a precedent that violates what is, IMO, the only reasonable policy. That makes it a little harder to make the policy official. But anyway, I have a strong suspicion that you and I substantially agree on that policy, so I would be very happy if you'd write something up. Then we have something concrete to argue about ;-)
Ok, I'll take this one. I see you've got the "standard practice" defined in another post. I'll try to collect all the relevant information from the current discussions and collect them into a document.
I've collected all the relevant information from the discussions so far. If there are more counter-arguments (Robert?) please post them now. I'll post my recommendation (subject to discussion) in the coming days. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Hi, As promised, here's a draft of the proposed "Boost Header And Namespace Policy" (attached) Comments, suggestions, etc. very welcome. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net Boost Header And Namespace Policy There has been an unwritten standard practice on header placement and namespaces that has been developed over the years. The Boost policy gives library authors a huge amount of freedom to organize and evolve their libraries as they see fit. Authors of newer libraries, however, had to try and figure out from the way existing libraries are structured or ask help from fellow boosters with more boost experience in "boostifying" their code. With very few exceptions, this worked well in the past. Library authors stayed within the bounds of the standard practice. Yet, with the rate Boost is growing, it would be crucial to put the Boost standard practice in writing to avoid confusion in the future. This short document will attempt to formalize the Boost standard practice on header placement, subdirectory and library organization, and namespace conventions. One purpose of the review process is to raise these issues at a proper time. It would be best to add adherence to the standard practice as part of the review. * Core libraries whose documented public interface fits in just a few headers and a few pages of documentation can put all their public headers in boost/ and their components in boost::. For a core library named foobar, the convention is as follows: - One or more (preferably one) header file(s). Example: boost/foobar.hpp - foobar components (classes, types, constants etc.) in namespace boost. Example: namespace boost { class foo {/*...*/}; class bar {/*...*/}; } The concept of "core library" is not new. boost::shared_ptr, for example, is a core library. It's pretty much common to all libraries. Having a core library gives us structure. Typically, "core" libraries have the highest number of dependencies. Looking at the dependency diagram of boost, the core libraries will be at the bottommost with lots of other libraries pointing acyclically at it. It's a must to avoid having them depend on other libraries in other layers above the core. More emphasis and constraints must be given to these "core" libraries as they form the backbone of boost as a whole. For instance, a broken core library will have disastrous effects on the whole Boost library --core libraries should be very stable. Determining wether a library is core can be part of the review. If the author of a library intends it to be a "core" library, he can explicitly say so and be subject for the review. A core library will have to accept more stringent requirements such as stability and non-dependence on other non-core libraries. * Utility libraries whose documented public interface fits in just a few headers and a few pages of documentation can put all their public headers in boost/utility and their components in boost::. For a utility library named foobar, the convention is as follows: - One or more (preferably one) header file(s). Example: boost/utility/foobar.hpp - foobar components (classes, types, constants etc.) in namespace boost. Example: namespace boost { class foo {/*...*/}; class bar {/*...*/}; } All utility libraries pass through the same Boost review process. In certain occasions, a small library that are common to one or more Boost libraries and has already been in extensive use may undergo a "Fast track" review for inclusion as a boost utility. * Non-core libraries may place a single consolidated convenience header in boost/, forwarding to files of the form boost/<libraryname>/<header>.hpp. Regardless, their documented public components are not in boost/ or namespace boost:: For a library named foobar, the convention is as follows: - A single boost/foobar.hpp that forwards to all the headers of foobar: boost/foobar.hpp - foobar components (classes, types, constants etc.) in namespace boost::foobar. Example: namespace boost { namespace foobar { class foo {/*...*/}; class bar {/*...*/}; }} - A subdirectory boost/foobar where all foobar headers are placed. * Un-reviewed implementation details of libraries have been placed in boost/detail if they need to be used by other libraries and a subdirectory of boost/<libraryname>/ otherwise. Their documented public components are placed in boost/detail and namespace boost::detail. * Libraries may host sub libraries that may be used by other boost libraries. Such sub libraries are typically meant for future boost review. Their documented public components are placed in boost/<host-libraryname>/<libraryname>/<header>.hpp and namespace boost::<libraryname>. One such example is the proto library that is hosted by xpressive. Another is fusion. The fusion library was once hosted by spirit. After passing the boost formal review the fusion library is now a full fledged boost library.
Joel de Guzman wrote:
* Un-reviewed implementation details of libraries have been placed in boost/detail if they need to be used by other libraries and a subdirectory of boost/<libraryname>/ otherwise. Their documented public components are placed in boost/detail and namespace boost::detail.
I think the current situation is a little problematic. If there in boos/detail, presumable its because they might be useful accross more than one library. However, There is no place for documentation of these things. So there is no guarenteed interface. And of course no separate tests. No guarentee that the interface won't change - after all its an implementation detail. So it can change without warning an break other libraries. So, one has a lot of reservations about depending upon these modules. On the other hand, they have proved very useful so for the sake of expediency they're going to get used - leading to surprise breakages. So I think those things that have been going to boost/detail should just go into boost / utility. Approval for this would be part of the review process for the library which needed them. I realise some might find this bothersome, but its much better the the current situation. Robert Ramey
Robert Ramey wrote:
Joel de Guzman wrote:
* Un-reviewed implementation details of libraries have been placed in boost/detail if they need to be used by other libraries and a subdirectory of boost/<libraryname>/ otherwise. Their documented public components are placed in boost/detail and namespace boost::detail.
I think the current situation is a little problematic.
If there in boos/detail, presumable its because they might be useful accross more than one library. However,
There is no place for documentation of these things. So there is no guarenteed interface.
And of course no separate tests.
No guarentee that the interface won't change - after all its an implementation detail. So it can change without warning an break other libraries.
So, one has a lot of reservations about depending upon these modules. On the other hand, they have proved very useful so for the sake of expediency they're going to get used - leading to surprise breakages.
So I think those things that have been going to boost/detail should just go into boost / utility. Approval for this would be part of the review process for the library which needed them. I realise some might find this bothersome, but its much better the the current situation.
Good points. I for one have not placed anything in boost detail. I just based the text on current standard practice, but to be honest, I too don't like free for all additions into boost detail. It too has a big chance of pollution related problems as Boost grows even bigger. My opinion on this is to keep the boost/detail *solely* for core libraries use. For non-core libraries, it is always better to use boost/<library>/detail and boost::<library>::detail as many libraries are already doing. If they need to be used by other libraries, then my thinking is that they ought not to be in detail, but rather, hosted by a parent library, just like that suggested in the last item in the list of current "practices". Then, the natural place for its documentation is the host library (e.g. The proto docs are currently in xpressive, the pre-review of fusion was once in spirit). Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
on Sat Dec 01 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
My opinion on this is to keep the boost/detail *solely* for core libraries use. For non-core libraries, it is always better to use boost/<library>/detail and boost::<library>::detail as many libraries are already doing. If they need to be used by other libraries, then my thinking is that they ought not to be in detail, but rather, hosted by a parent library, just like that suggested in the last item in the list of current "practices". Then, the natural place for its documentation is the host library (e.g. The proto docs are currently in xpressive, the pre-review of fusion was once in spirit).
That doesn't always work. For example, I have a number of tiny utilities in Boost.Python that I needed to re-use in Boost.Iterator. Having Boost.Iterator depend on Boost Python would be totally inappropriate, so after I got things working I moved these components into boost/detail. That's the way boost/detail ought to be used. These components are tested and commented, but they are pretty obscure, and making them into public Boost components would entail writing rationales, tutorials, and formal interface documentation. I didn't have time for that, and almost nobody would have benefited from it: adding more oddball components into the list at http://boost.org/libs will only make it harder to find the useful stuff. Sometimes a really general and useful component will end up in boost/detail (or <libraryname>/detail) because the author just doesn't have time to do what it takes to make it a public part of the Boost interface. That's legitimate, too; this is a volunteer effort, after all. If someone wants it in the public interface badly enough, he or she can take responsibility for getting it there. That's what ha -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
on Sat Dec 01 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
My opinion on this is to keep the boost/detail *solely* for core libraries use. For non-core libraries, it is always better to use boost/<library>/detail and boost::<library>::detail as many libraries are already doing. If they need to be used by other libraries, then my thinking is that they ought not to be in detail, but rather, hosted by a parent library, just like that suggested in the last item in the list of current "practices". Then, the natural place for its documentation is the host library (e.g. The proto docs are currently in xpressive, the pre-review of fusion was once in spirit).
That doesn't always work. For example, I have a number of tiny utilities in Boost.Python that I needed to re-use in Boost.Iterator. Having Boost.Iterator depend on Boost Python would be totally inappropriate, so after I got things working I moved these components into boost/detail. That's the way boost/detail ought to be used.
These components are tested and commented, but they are pretty obscure, and making them into public Boost components would entail writing rationales, tutorials, and formal interface documentation. I didn't have time for that, and almost nobody would have benefited from it: adding more oddball components into the list at http://boost.org/libs will only make it harder to find the useful stuff.
My concern is that this policy does not seem to be scalable. What's to prevent someone from adding hundreds of small things (typedefs, enums, small classes, etc.) in boost detail? You may say common sense will prevent people from doing so, but with a free for all addition into boost::detail, this is not a paranoid scenario in the future when Boost grows even bigger and a lot more things can go wrong, slipping from the common sense radar screen. Now, multiply that to hundreds of developers. To me, it will not be a pretty situation. I do not see Boost.Iterator having dependence on Boost.Python if the common components are segregated sufficiently in, say, boost/python/support. Fusion was once hosted by Spirit and xpressive used Fusion. That does not mean that xpressive had a dependency on spirit. It's just the location. Anyway, if it's used by both Boost.Python and Boost.Iterator, I'd say that Boost.Iterator should be the host since it is a core library.
Sometimes a really general and useful component will end up in boost/detail (or <libraryname>/detail) because the author just doesn't have time to do what it takes to make it a public part of the Boost interface. That's legitimate, too; this is a volunteer effort, after all. If someone wants it in the public interface badly enough, he or she can take responsibility for getting it there. That's what ha
This paragraph seems truncated? Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman wrote:
David Abrahams wrote:
These components are tested and commented, but they are pretty obscure, and making them into public Boost components would entail writing rationales, tutorials, and formal interface documentation. I didn't have time for that, and almost nobody would have benefited from it: adding more oddball components into the list at http://boost.org/libs will only make it harder to find the useful stuff.
My concern is that this policy does not seem to be scalable. What's to prevent someone from adding hundreds of small things (typedefs, enums, small classes, etc.) in boost detail? You may say common sense will prevent people from doing so, but with a free for all addition into boost::detail, this is not a paranoid scenario in the future when Boost grows even bigger and a lot more things can go wrong, slipping from the common sense radar screen. Now, multiply that to hundreds of developers. To me, it will not be a pretty situation.
I do not see Boost.Iterator having dependence on Boost.Python if the common components are segregated sufficiently in, say, boost/python/support. Fusion was once hosted by Spirit and xpressive used Fusion. That does not mean that xpressive had a dependency on spirit. It's just the location.
Anyway, if it's used by both Boost.Python and Boost.Iterator, I'd say that Boost.Iterator should be the host since it is a core library.
To be clear, I meant to say that it's ok to have a common utility for Boost.Python and Boost.Iterator to be in boost::detail since Boost.Iterator is a core library. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
on Sun Dec 02 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
To be clear, I meant to say that it's ok to have a common utility for Boost.Python and Boost.Iterator to be in boost::detail since Boost.Iterator is a core library.
That appears to be a new rule that didn't appear in your original writeup (?) -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
on Sun Dec 02 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
To be clear, I meant to say that it's ok to have a common utility for Boost.Python and Boost.Iterator to be in boost::detail since Boost.Iterator is a core library.
That appears to be a new rule that didn't appear in your original writeup (?)
Nah, that stems from my suggestion to restrict boost detail to only core libraries. Anyway, I'm withdrawing that suggestion. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
on Sun Dec 02 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
David Abrahams wrote:
on Sat Dec 01 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
My opinion on this is to keep the boost/detail *solely* for core libraries use. For non-core libraries, it is always better to use boost/<library>/detail and boost::<library>::detail as many libraries are already doing. If they need to be used by other libraries, then my thinking is that they ought not to be in detail, but rather, hosted by a parent library, just like that suggested in the last item in the list of current "practices". Then, the natural place for its documentation is the host library (e.g. The proto docs are currently in xpressive, the pre-review of fusion was once in spirit).
That doesn't always work. For example, I have a number of tiny utilities in Boost.Python that I needed to re-use in Boost.Iterator. Having Boost.Iterator depend on Boost Python would be totally inappropriate, so after I got things working I moved these components into boost/detail. That's the way boost/detail ought to be used.
These components are tested and commented, but they are pretty obscure, and making them into public Boost components would entail writing rationales, tutorials, and formal interface documentation. I didn't have time for that, and almost nobody would have benefited from it: adding more oddball components into the list at http://boost.org/libs will only make it harder to find the useful stuff.
My concern is that this policy does not seem to be scalable. What's to prevent someone from adding hundreds of small things (typedefs, enums, small classes, etc.) in boost detail?
The policy does. It says you only do that when they are in fact needed by multiple libraries. And anyway, what's wrong with having hundreds of small things in boost detail?
You may say common sense will prevent people from doing so,
I would say that too.
but with a free for all addition into boost::detail,
Who ever suggested a free-for-all?
this is not a paranoid scenario in the future when Boost grows even bigger and a lot more things can go wrong, slipping from the common sense radar screen. Now, multiply that to hundreds of developers. To me, it will not be a pretty situation.
I do not see Boost.Iterator having dependence on Boost.Python if the common components are segregated sufficiently in, say, boost/python/support.
You may not see it that way, but I do. And I know most other people will see it that way until they understand the very interesting concept that libx/support isn't really part of libx. http://lists.boost.org/Archives/boost/2004/05/65592.php http://lists.boost.org/Archives/boost/2005/08/91483.php
Fusion was once hosted by Spirit and xpressive used Fusion. That does not mean that xpressive had a dependency on spirit. It's just the location.
Was Fusion a component of Spirit, or was it not? If it was, and xpressive depended on it, then it depended on (a part of) Spirit. One library rarely depends on another library entirely, so this isn't any different than depending on boost/type_traits/is_same.hpp.
Anyway, if it's used by both Boost.Python and Boost.Iterator, I'd say that Boost.Iterator should be the host since it is a core library.
Now to play the devil's advocate: core as defined by whom?
Sometimes a really general and useful component will end up in boost/detail (or <libraryname>/detail) because the author just doesn't have time to do what it takes to make it a public part of the Boost interface. That's legitimate, too; this is a volunteer effort, after all. If someone wants it in the public interface badly enough, he or she can take responsibility for getting it there. That's what ha
Yes. Well IIRC that's what happened with boost/detail/value_init.hpp but I could be mistaken. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
My concern is that this policy does not seem to be scalable. What's to prevent someone from adding hundreds of small things (typedefs, enums, small classes, etc.) in boost detail?
The policy does. It says you only do that when they are in fact needed by multiple libraries. And anyway, what's wrong with having hundreds of small things in boost detail?
--The same reasons why we use sub-namespaces and sub-directories. Multiply that with the number of Boost developers past and present. The single boost::detail namespace can become utterly crowded. OK, you have good points. I won't argue this any more. My only concern is a crowded boost::detail namespace in the future. Perhaps what we can do is subdivide boost::detail in the future when the need arises. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman wrote:
David Abrahams wrote:
My concern is that this policy does not seem to be scalable. What's to prevent someone from adding hundreds of small things (typedefs, enums, small classes, etc.) in boost detail? The policy does. It says you only do that when they are in fact needed by multiple libraries. And anyway, what's wrong with having hundreds of small things in boost detail?
--The same reasons why we use sub-namespaces and sub-directories. Multiply that with the number of Boost developers past and present. The single boost::detail namespace can become utterly crowded.
OK, you have good points. I won't argue this any more. My only concern is a crowded boost::detail namespace in the future. Perhaps what we can do is subdivide boost::detail in the future when the need arises.
Hmmm.. which might not be a bad idea. We can name the sub-namespaces based on generic categories like that in http://boost.org/libs/libraries.htm categories. Cheers, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
on Mon Dec 03 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
David Abrahams wrote:
My concern is that this policy does not seem to be scalable. What's to prevent someone from adding hundreds of small things (typedefs, enums, small classes, etc.) in boost detail?
The policy does. It says you only do that when they are in fact needed by multiple libraries. And anyway, what's wrong with having hundreds of small things in boost detail?
--The same reasons why we use sub-namespaces and sub-directories. Multiply that with the number of Boost developers past and present. The single boost::detail namespace can become utterly crowded.
OK, you have good points. I won't argue this any more. My only concern is a crowded boost::detail namespace in the future. Perhaps what we can do is subdivide boost::detail in the future when the need arises.
That sounds entirely sensible. Unlike public interfaces, we can refactor boost/detail as much as we want without breaking any code because we have access to all the code that depends on boost/detail. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
Joel de Guzman wrote: For non-core libraries, it is always better
to use boost/<library>/detail and boost::<library>::detail as many libraries are already doing. If they need to be used by other libraries, then my thinking is that they ought not to be in detail, but rather, hosted by a parent library, just like that suggested in the last item in the list of current "practices". Then, the natural place for its documentation is the host library (e.g. The proto docs are currently in xpressive, the pre-review of fusion was once in spirit).
The case of codecvt_utf8 is interesting in his regard. In the course of development, we ended up with one copy in the serialization library and another identical copy in the program options library. Strenuous objections were raised about this. So it was "moved" to boost/detail - only because of these objections. It was never reviewed as a separate entity. It had its own tests and documentation page - but there was no place to put this in boost/detail. Perhaps boost/utility would be better - but then it would have had to suffer a review and no one wanted to take ownership. And recently I discovered that I forgot to take it out of the serialization library in any case. So a strict adherence to the proposed policy will result in duplicated code - not the end of the world - but some are going to object to that. As an aside, I noticed a talk titled something like "hidden boost gems" where codecvt_utf8 was specifically mentioned. So some things end up being quasi-official by default. Robert Ramey
Robert Ramey wrote:
Joel de Guzman wrote: For non-core libraries, it is always better
to use boost/<library>/detail and boost::<library>::detail as many libraries are already doing. If they need to be used by other libraries, then my thinking is that they ought not to be in detail, but rather, hosted by a parent library, just like that suggested in the last item in the list of current "practices". Then, the natural place for its documentation is the host library (e.g. The proto docs are currently in xpressive, the pre-review of fusion was once in spirit).
The case of codecvt_utf8 is interesting in his regard.
In the course of development, we ended up with one copy in the serialization library and another identical copy in the program options library. Strenuous objections were raised about this. So it was "moved" to boost/detail - only because of these objections. It was never reviewed as a separate entity. It had its own tests and documentation page - but there was no place to put this in boost/detail. Perhaps boost/utility would be better - but then it would have had to suffer a review and no one wanted to take ownership. And recently I discovered that I forgot to take it out of the serialization library in any case. So a strict adherence to the proposed policy will result in duplicated code - not the end of the world - but some are going to object to that.
As an aside, I noticed a talk titled something like "hidden boost gems" where codecvt_utf8 was specifically mentioned. So some things end up being quasi-official by default.
What's wrong with having one of the interested parties host it? What's wrong with having it in, say, boost/serialization/support and have it included by the other library using it? Sure, that is not ideal, but that practice ultimately avoids having boost detail crowded. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman wrote:
Robert Ramey wrote:
What's wrong with having one of the interested parties host it? What's wrong with having it in, say, boost/serialization/support and have it included by the other library using it? Sure, that is not ideal, but that practice ultimately avoids having boost detail crowded.
I never found anything wrong with it. I think what happend was that the author of another library - program options - felt at the time that depending on something which was sort of an implementation detail in another library (that was new at the time) was a risky idea. So he made a local copy. I think this was very reasonable in this situation. But objections were raised about having duplicated code so that's how we got here. I don't think you cat get around the fact that either something is shared and public, or its private - it can't be both. if its the former it has to have the things that shared/public things have, documentation, rationale, stable interface etc. Robert Ramey
Regards,
on Mon Dec 03 2007, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
I don't think you cat get around the fact that either something is shared and public, or its private - it can't be both.
Sure it can. Something can be public to Boost developers but not to Boost users. Any company that develops code with an external API deals with this kind of thing all the time. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
on Mon Dec 03 2007, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
I don't think you cat get around the fact that either something is shared and public, or its private - it can't be both.
Sure it can. Something can be public to Boost developers but not to Boost users. Any company that develops code with an external API deals with this kind of thing all the time.
So obviously we call that "protected". :) -- Jon Biggar Levanta jon@levanta.com 650-403-7252
on Sat Dec 01 2007, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
Joel de Guzman wrote:
* Un-reviewed implementation details of libraries have been placed in boost/detail if they need to be used by other libraries and a subdirectory of boost/<libraryname>/ otherwise. Their documented public components are placed in boost/detail and namespace boost::detail.
I think the current situation is a little problematic.
If there in boos/detail, presumable its because they might be useful accross more than one library.
Not usually. Usually it's because they *have* been useful across more than one library. In general, nobody sticks stuff in boost/detail just because they think it might have a use elsewhere. Components migrate there.
However,
There is no place for documentation of these things.
Sure there is. Comments work.
So there is no guarenteed interface.
And of course no separate tests.
Not necessarily. I always write tests for my library's implementation detail components.
No guarentee that the interface won't change - after all its an implementation detail. So it can change without warning an break other libraries.
So, one has a lot of reservations about depending upon these modules. On the other hand, they have proved very useful so for the sake of expediency they're going to get used - leading to surprise breakages.
That's more of a theoretical problem than a real one. It has never been an issue in the past. Most people realize that once a component moves into boost/detail, other people may be depending on it, so they make an effort to coordinate if there are any breaking changes (which, to my knowledge, there have never been in a boost/detail component).
So I think those things that have been going to boost/detail should just go into boost / utility. Approval for this would be part of the review process for the library which needed them.
Implementation details get invented and shared as part of a library's evolution, so that would never work
I realise some might find this bothersome, but its much better the the current situation.
I don't see why. A little common sense is all it takes to make this work out, and it has worked very well so far. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
on Sat Dec 01 2007, "Robert Ramey" <ramey-AT-rrsd.com> wrote:
Joel de Guzman wrote:
* Un-reviewed implementation details of libraries have been placed in boost/detail if they need to be used by other libraries and a subdirectory of boost/<libraryname>/ otherwise. Their documented public components are placed in boost/detail and namespace boost::detail.
I think the current situation is a little problematic.
If there in boos/detail, presumable its because they might be useful accross more than one library.
Not usually. Usually it's because they *have* been useful across more than one library. In general, nobody sticks stuff in boost/detail just because they think it might have a use elsewhere. Components migrate there.
However,
There is no place for documentation of these things.
Sure there is. Comments work.
So there is no guarenteed interface.
And of course no separate tests.
Not necessarily. I always write tests for my library's implementation detail components.
No guarentee that the interface won't change - after all its an implementation detail. So it can change without warning an break other libraries.
So, one has a lot of reservations about depending upon these modules. On the other hand, they have proved very useful so for the sake of expediency they're going to get used - leading to surprise breakages.
That's more of a theoretical problem than a real one. It has never been an issue in the past. Most people realize that once a component moves into boost/detail, other people may be depending on it, so they make an effort to coordinate if there are any breaking changes (which, to my knowledge, there have never been in a boost/detail component).
So I think those things that have been going to boost/detail should just go into boost / utility. Approval for this would be part of the review process for the library which needed them.
Implementation details get invented and shared as part of a library's evolution, so that would never work
I realise some might find this bothersome, but its much better the the current situation.
I don't see why. A little common sense is all it takes to make this work out, and it has worked very well so far. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
On Nov 30, 2007 8:47 PM, Joel de Guzman <joel@boost-consulting.com> wrote:
Hi,
As promised, here's a draft of the proposed "Boost Header And Namespace Policy" (attached) Comments, suggestions, etc. very welcome.
[snip]
The concept of "core library" is not new. boost::shared_ptr, for example, is a core library. It's pretty much common to all libraries. Having a core library gives us structure. Typically, "core" libraries have the highest number of dependencies. Looking at the dependency diagram of boost, the core libraries will be at the bottommost with lots of other libraries pointing acyclically at it. It's a must to avoid having them depend on other libraries in other layers above the core. More emphasis and constraints must be given to these "core" libraries as they form the backbone of boost as a whole. For instance, a broken core library will have disastrous effects on the whole Boost library --core libraries should be very stable.
Determining wether a library is core can be part of the review. If the author of a library intends it to be a "core" library, he can explicitly say so and be subject for the review. A core library will have to accept more stringent requirements such as stability and non-dependence on other non-core libraries. [snip]
This raises an interesting question for me. I recently received a bug report against boost-dependent library that I work on (Adobe Source Libraries). One of our components supports move semantics. It makes use of a free function named move in namespace adobe. We were calling it unqualified from within other library routines. With boost 1.35 this will have to change. Why? Because some classes in the adobe library inherit from boost::equality_comparable. As a result, when adobe::move is called with such boost-derived classes, ADL pulls in the boost namespace and an ambiguity arises against the boost::move supplied by boost::thread. One might argue that best practice in the face of ADL is to be defensive and to avoid unqualified calls (when replaceability is not the intent). In fact, I will do just this for our next release. However this would be a little bit more pleasant if thread's boost::move was intended as a documented boost-wide move-semantics component. In fact we might consider dropping our own move component if such a boost component existed. At present however, it appears that the boost::move is more of a detail of boost/thread. Are we sure that now is the time to put move in the top level boost namespace? Thoughts? Mat
Mat Marcus wrote:
On Nov 30, 2007 8:47 PM, Joel de Guzman <joel@boost-consulting.com> wrote:
Hi,
As promised, here's a draft of the proposed "Boost Header And Namespace Policy" (attached) Comments, suggestions, etc. very welcome.
[snip]
The concept of "core library" is not new. boost::shared_ptr, for example, is a core library. It's pretty much common to all libraries. Having a core library gives us structure. Typically, "core" libraries have the highest number of dependencies. Looking at the dependency diagram of boost, the core libraries will be at the bottommost with lots of other libraries pointing acyclically at it. It's a must to avoid having them depend on other libraries in other layers above the core. More emphasis and constraints must be given to these "core" libraries as they form the backbone of boost as a whole. For instance, a broken core library will have disastrous effects on the whole Boost library --core libraries should be very stable.
Determining wether a library is core can be part of the review. If the author of a library intends it to be a "core" library, he can explicitly say so and be subject for the review. A core library will have to accept more stringent requirements such as stability and non-dependence on other non-core libraries. [snip]
This raises an interesting question for me. I recently received a bug report against boost-dependent library that I work on (Adobe Source Libraries). One of our components supports move semantics. It makes use of a free function named move in namespace adobe. We were calling it unqualified from within other library routines. With boost 1.35 this will have to change. Why? Because some classes in the adobe library inherit from boost::equality_comparable. As a result, when adobe::move is called with such boost-derived classes, ADL pulls in the boost namespace and an ambiguity arises against the boost::move supplied by boost::thread. One might argue that best practice in the face of ADL is to be defensive and to avoid unqualified calls (when replaceability is not the intent). In fact, I will do just this for our next release. However this would be a little bit more pleasant if thread's boost::move was intended as a documented boost-wide move-semantics component. In fact we might consider dropping our own move component if such a boost component existed. At present however, it appears that the boost::move is more of a detail of boost/thread. Are we sure that now is the time to put move in the top level boost namespace?
Thoughts?
If the thread library's move function is undocumented, then it has no business being in the root boost namespace but in boost::detail. If indeed that is the case, I consider it a bug that should be fixed. We should be more vigilant in the review period on such violations. More stringent requirements should be put in place for core libraries. We should try to restrict as much as possible what goes into the boost root namespace and directory. I think it's best to add: Only reviewed and documented components of a core library may be placed in the root boost namespace. Violations are considered bugs that must be fixed. in our core libraries policies. ((I did a quick search in boost thread's docs and indeed move is not documented.)) Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman wrote:
If the thread library's move function is undocumented, then it has no business being in the root boost namespace but in boost::detail. If indeed that is the case, I consider it a bug that should be fixed. We should be more vigilant in the review period on such violations. More stringent requirements should be put in place for core libraries. We should try to restrict as much as possible what goes into the boost root namespace and directory. I think it's best to add:
Only reviewed and documented components of a core library may be placed in the root boost namespace. Violations are considered bugs that must be fixed.
in our core libraries policies.
((I did a quick search in boost thread's docs and indeed move is not documented.))
100% agreement from me: "move" is such a common name that we can't afford to pollute namespace boost with it: unless we really mean to of course :-) Unfortunately the Boost.Thread docs aren't up to date, so I'm not sure what Anthony intended here: Anthony? Regards John.
"John Maddock" <john@johnmaddock.co.uk> writes:
Joel de Guzman wrote:
If the thread library's move function is undocumented, then it has no business being in the root boost namespace but in boost::detail. If indeed that is the case, I consider it a bug that should be fixed. We should be more vigilant in the review period on such violations. More stringent requirements should be put in place for core libraries. We should try to restrict as much as possible what goes into the boost root namespace and directory. I think it's best to add:
Only reviewed and documented components of a core library may be placed in the root boost namespace. Violations are considered bugs that must be fixed.
in our core libraries policies.
((I did a quick search in boost thread's docs and indeed move is not documented.))
100% agreement from me: "move" is such a common name that we can't afford to pollute namespace boost with it: unless we really mean to of course :-)
Unfortunately the Boost.Thread docs aren't up to date, so I'm not sure what Anthony intended here: Anthony?
Currently, the boost::move in question is an implementation detail of thread. It is an oversight that it is in the boost namespace and not the detail namespace. However, I did post a message asking for a fast-track review of it so that it could become a documented part of boost. The only response I got was "there's another implementation in the sandbox". Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
on Mon Dec 03 2007, Anthony Williams <anthony_w.geo-AT-yahoo.com> wrote:
However, I did post a message asking for a fast-track review of it so that it could become a documented part of boost. The only response I got was "there's another implementation in the sandbox".
And that implementation is fatally flawed, I'm ashamed to say. We should be using http://opensource.adobe.com/group__move__related.html -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams <dave@boost-consulting.com> writes:
on Mon Dec 03 2007, Anthony Williams <anthony_w.geo-AT-yahoo.com> wrote:
However, I did post a message asking for a fast-track review of it so that it could become a documented part of boost. The only response I got was "there's another implementation in the sandbox".
And that implementation is fatally flawed, I'm ashamed to say. We should be using http://opensource.adobe.com/group__move__related.html
How does that help with a movable-but-not-copyable type (such as boost::thread)? Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
on Mon Dec 03 2007, Anthony Williams <anthony_w.geo-AT-yahoo.com> wrote:
David Abrahams <dave@boost-consulting.com> writes:
on Mon Dec 03 2007, Anthony Williams <anthony_w.geo-AT-yahoo.com> wrote:
However, I did post a message asking for a fast-track review of it so that it could become a documented part of boost. The only response I got was "there's another implementation in the sandbox".
And that implementation is fatally flawed, I'm ashamed to say. We should be using http://opensource.adobe.com/group__move__related.html
How does that help with a movable-but-not-copyable type (such as boost::thread)?
I'm not sure, frankly; the author of that library believes all types are intrinsically both movable and copiable. Maybe for that purpose we need something else. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams <dave <at> boost-consulting.com> writes:
on Mon Dec 03 2007, Anthony Williams <anthony_w.geo-AT-yahoo.com> wrote:
David Abrahams <dave <at> boost-consulting.com> writes:
on Mon Dec 03 2007, Anthony Williams <anthony_w.geo-AT-yahoo.com> wrote:
However, I did post a message asking for a fast-track review of it so that it could become a documented part of boost. The only response I got was "there's another implementation in the sandbox".
And that implementation is fatally flawed, I'm ashamed to say. We should be using http://opensource.adobe.com/group__move__related.html
How does that help with a movable-but-not-copyable type (such as boost::thread)?
I'm not sure, frankly; the author of that library believes all types are intrinsically both movable and copiable. Maybe for that purpose we need something else.
Can I (again) propose the simple scheme I'm using for boost::thread? In boost/thread/detail/move.hpp we (now) have boost::detail::thread_move_t<T> as the transfer type and boost::detail::thread_move(T&) as the move-from-lvalue helper boost::thread then looks like this (move-related parts only) class thread { private: thread(thread&); // no copy. Note no const thread& operator=(thread&); // no assign. Note no const public: thread(detail::thread_move_t<thread>); // move constructor thread& operator=(detail::thread_move_t<thread>); // move assignment operator detail::thread_move_t<thread>(); // allow conversion to the //transfer type detail::thread_move_t<thread> move(); // make it explicit }; If you try and construct a thread object from an rvalue then it won't bind to the (private) copy constructor, as it can't bind to a non-const reference. However, a single user-defined conversion (to detail::thread_move_t<thread>) allows it to bind to the move constructor. Likewise for assignment. It works for gcc and MSVC, but not for Borland. When the test results come through for the new test_thread_move test, we'll know which other compilers it works on, too. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
On Dec 3, 2007 4:56 PM, David Abrahams <dave@boost-consulting.com> wrote:
on Mon Dec 03 2007, Anthony Williams <anthony_w.geo-AT-yahoo.com> wrote:
However, I did post a message asking for a fast-track review of it so that it could become a documented part of boost. The only response I got was "there's another implementation in the sandbox".
And that implementation is fatally flawed, I'm ashamed to say.
Are you refrerring to your move.hpp header? Why do you think is flawed? I've used it and it seemed to work well for all cases I tested (It was a move only object, non copiable). gpd
on Wed Dec 05 2007, "Giovanni Piero Deretta" <gpderetta-AT-gmail.com> wrote:
On Dec 3, 2007 4:56 PM, David Abrahams <dave@boost-consulting.com> wrote:
on Mon Dec 03 2007, Anthony Williams <anthony_w.geo-AT-yahoo.com> wrote:
However, I did post a message asking for a fast-track review of it so that it could become a documented part of boost. The only response I got was "there's another implementation in the sandbox".
And that implementation is fatally flawed, I'm ashamed to say.
Are you refrerring to your move.hpp header?
Yes.
Why do you think is flawed? I've used it and it seemed to work well for all cases I tested (It was a move only object, non copiable).
To the best of my memory: movable objects have a non-const RHS copy ctor, which prevents them from interoperating correctly in standard containers and other "legacy" code. Furthermore, they "infect" the objects that contain them with a non-const RHS copy ctor, too. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
Anthony Williams <anthony_w.geo <at> yahoo.com> writes:
Currently, the boost::move in question is an implementation detail of thread. It is an oversight that it is in the boost namespace and not the detail namespace.
I have now fixed this: I have renamed it to boost::detail::thread_move (and boost::detail::thread_move_t Anthony
Anthony Williams wrote:
Anthony Williams <anthony_w.geo <at> yahoo.com> writes:
Currently, the boost::move in question is an implementation detail of thread. It is an oversight that it is in the boost namespace and not the detail namespace.
I have now fixed this: I have renamed it to boost::detail::thread_move (and boost::detail::thread_move_t
Hmm... I guess that means users are not allowed to move Boost.Thread components. That's a shame, because I was using this feature in my development. Maybe moving to boost::threads namespace or just renaming them would be a better solution?
Andrey Semashev <andysem <at> mail.ru> writes:
Anthony Williams wrote:
Anthony Williams <anthony_w.geo <at> yahoo.com> writes:
Currently, the boost::move in question is an implementation detail of thread. It is an oversight that it is in the boost namespace and not the detail namespace.
I have now fixed this: I have renamed it to boost::detail::thread_move (and boost::detail::thread_move_t
Hmm... I guess that means users are not allowed to move Boost.Thread components. That's a shame, because I was using this feature in my development.
That's not the intention. This is (largely) an implementation detail to make up for the lack of rvalue references. boost::thread make_thread() { return boost::thread(some_func); } boost::thread x=make_thread(); "just works" --- I added an explicit test for it this morning. Moving from an lvalue used to be done with boost::move(x), and is now boost::detail::thread_move(x) or x.move() (if x is a boost::thread).
Maybe moving to boost::threads namespace or just renaming them would be a better solution?
We could just have boost::thread_move become a documented public interface, and have the type remain in boost::detail (since it *is* an implementation detail). Thoughts anyone? Roland? Anthony
Anthony Williams wrote:
Andrey Semashev <andysem <at> mail.ru> writes:
Anthony Williams wrote:
Anthony Williams <anthony_w.geo <at> yahoo.com> writes:
Currently, the boost::move in question is an implementation detail of thread. It is an oversight that it is in the boost namespace and not the detail namespace. I have now fixed this: I have renamed it to boost::detail::thread_move (and boost::detail::thread_move_t Hmm... I guess that means users are not allowed to move Boost.Thread components. That's a shame, because I was using this feature in my development.
That's not the intention. This is (largely) an implementation detail to make up for the lack of rvalue references.
boost::thread make_thread() { return boost::thread(some_func); }
boost::thread x=make_thread();
"just works" --- I added an explicit test for it this morning.
Moving from an lvalue used to be done with boost::move(x), and is now boost::detail::thread_move(x) or x.move() (if x is a boost::thread).
Maybe moving to boost::threads namespace or just renaming them would be a better solution?
We could just have boost::thread_move become a documented public interface, and have the type remain in boost::detail (since it *is* an implementation detail).
Actually, I used it to move locks, which don't have a move member. I worked around this issue by using defer_lock_t and swap but it looks clumsy and I would surely prefer using move in such cases. As for possible solution, I feel fine if thread_move is public and thread_move_t is private. I guess, we'll have to wait until the full-featured move library comes up in order to make use of the type.
Andrey Semashev <andysem@mail.ru> writes:
We could just have boost::thread_move become a documented public interface, and have the type remain in boost::detail (since it *is* an implementation detail).
Actually, I used it to move locks, which don't have a move member. I worked around this issue by using defer_lock_t and swap but it looks clumsy and I would surely prefer using move in such cases.
I have removed detail::thread_move(). Instead, I've added boost::move overloads for the movable types in boost.thread: thread, unique_lock<>, shared_lock<>, upgrade_lock<>. Since these overloads take specific parameter types, they shouldn't cause the problems that the unconstrained template caused before. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday 17 December 2007 09:59 am, Anthony Williams wrote:
I have removed detail::thread_move(). Instead, I've added boost::move overloads for the movable types in boost.thread: thread, unique_lock<>, shared_lock<>, upgrade_lock<>. Since these overloads take specific parameter types, they shouldn't cause the problems that the unconstrained template caused before.
Have you considered making the mutex types movable? I'm working on some mutex wrapper classes and it would be nice to be able to move construct the wrapper from an object of its templated mutex type, instead of assuming the mutex type is going to be default constructible. - -- Frank -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHZx2/5vihyNWuA4URApOtAJ9eOHs+FasCZtMf3dSIwVS1EIUMCwCeOBOb rFuxzBcJvDDbxGMRueM588M= =dr2Z -----END PGP SIGNATURE-----
Frank Mori Hess <frank.hess <at> nist.gov> writes:
On Monday 17 December 2007 09:59 am, Anthony Williams wrote:
I have removed detail::thread_move(). Instead, I've added boost::move overloads for the movable types in boost.thread: thread, unique_lock<>, shared_lock<>, upgrade_lock<>. Since these overloads take specific parameter types, they shouldn't cause the problems that the unconstrained template caused before.
Have you considered making the mutex types movable? I'm working on some mutex wrapper classes and it would be nice to be able to move construct the wrapper from an object of its templated mutex type, instead of assuming the mutex type is going to be default constructible.
On POSIX, mutex types are not movable. Making boost::mutex movable would mean that either we could not use pthread_mutex_t internally, or we would have to dynamically allocate it. I don't think it makes sense to restrict the implementation in that way. Anthony
On Dec 18, 2007, at 3:47 AM, Anthony Williams wrote:
Frank Mori Hess <frank.hess <at> nist.gov> writes:
On Monday 17 December 2007 09:59 am, Anthony Williams wrote:
I have removed detail::thread_move(). Instead, I've added boost::move overloads for the movable types in boost.thread: thread, unique_lock<>, shared_lock<>, upgrade_lock<>. Since these overloads take specific parameter types, they shouldn't cause the problems that the unconstrained template caused before.
Have you considered making the mutex types movable? I'm working on some mutex wrapper classes and it would be nice to be able to move construct the wrapper from an object of its templated mutex type, instead of assuming the mutex type is going to be default constructible.
On POSIX, mutex types are not movable. Making boost::mutex movable would mean that either we could not use pthread_mutex_t internally, or we would have to dynamically allocate it. I don't think it makes sense to restrict the implementation in that way.
Having one thread move a mutex while another thread has it locked could be a scary thing. That being said, a non-movable boost::mutex does not prohibit a wrapper around that type from being movable, if you're willing to accept alternative semantics. I'm not familiar with the boost move library, but the C++0X syntax would simply be: class my_wrapper { std::mutex mut_; // or boost::mutex public: my_wrapper() {} my_wrapper(my_wrapper&&) {} // same as default ctor my_wrapper& operator=(my_wrapper&&) {return *this;} // do nothing void lock() {mut_.lock();} bool try_lock() {return mut_.try_lock();} void unlock() {mut_.unlock();} }; -Howard
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tuesday 18 December 2007 11:48 am, Howard Hinnant wrote:
That being said, a non-movable boost::mutex does not prohibit a wrapper around that type from being movable, if you're willing to accept alternative semantics. I'm not familiar with the boost move library, but the C++0X syntax would simply be:
class my_wrapper { std::mutex mut_; // or boost::mutex public: my_wrapper() {} my_wrapper(my_wrapper&&) {} // same as default ctor my_wrapper& operator=(my_wrapper&&) {return *this;} // do nothing
void lock() {mut_.lock();} bool try_lock() {return mut_.try_lock();} void unlock() {mut_.unlock();} };
What I'm thinking of is more along the lines of template<typename Mutex> class my_wrapper { Mutex mut_; public: my_wrapper(Mutex &&mut): mut_(mut); // ... }; So I can have a my_wrapper<Mutex> without worrying about whether the template Mutex type has some weird constructor, or even if it is default constructible. I guess the best I could do along those lines is to have a constructor that allows the user to pass in a dynamically allocated Mutex, which the wrapper would take ownership of. Note, I've never really done anything with rvalue references or moveable types, so maybe this is an abuse of them? - -- Frank -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHaBBp5vihyNWuA4URAkM5AKC8cqksDE3Rs94i2DoT2C3nP4jMpACdF0uQ l5w1kCn4p7XFcCkBXX6fits= =/nDt -----END PGP SIGNATURE-----
On Dec 18, 2007, at 1:24 PM, Frank Mori Hess wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tuesday 18 December 2007 11:48 am, Howard Hinnant wrote:
That being said, a non-movable boost::mutex does not prohibit a wrapper around that type from being movable, if you're willing to accept alternative semantics. I'm not familiar with the boost move library, but the C++0X syntax would simply be:
class my_wrapper { std::mutex mut_; // or boost::mutex public: my_wrapper() {} my_wrapper(my_wrapper&&) {} // same as default ctor my_wrapper& operator=(my_wrapper&&) {return *this;} // do nothing
void lock() {mut_.lock();} bool try_lock() {return mut_.try_lock();} void unlock() {mut_.unlock();} };
What I'm thinking of is more along the lines of
template<typename Mutex> class my_wrapper { Mutex mut_; public: my_wrapper(Mutex &&mut): mut_(mut); // ... };
So I can have a my_wrapper<Mutex> without worrying about whether the template Mutex type has some weird constructor, or even if it is default constructible. I guess the best I could do along those lines is to have a constructor that allows the user to pass in a dynamically allocated Mutex, which the wrapper would take ownership of. Note, I've never really done anything with rvalue references or moveable types, so maybe this is an abuse of them?
This sort of sounds like the proposed std::unique_lock<Mutex> (which Anthony has generously prototyped on the boost trunk): http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2447.htm#MutexsLock... template <class Mutex> class unique_lock { public: typedef Mutex mutex_type; private: mutex_type* m_; bool owns_lock_; unique_lock(unique_lock const&) = delete; unique_lock& operator=(unique_lock const&) = delete; public: ... explicit unique_lock(mutex_type& m) : m_(&m), owns_lock_(true) { m_->lock(); } ... ~unique_lock() { if (owns_lock_) m_->unlock(); } unique_lock(unique_lock&& u) // transfers mutex lock ownership to this : m_(u.m_), owns_lock_(u.owns_lock_) { u.m_ = 0; u.owns_lock_ = false; } unique_lock& operator=(unique_lock&& u) // transfers mutex lock ownership to this { if (owns_lock_) m_->unlock(); owns_lock_ = u.owns_lock_; m_ = u.m_; u.owns_lock_ = false; u.m_ = 0; return *this; } void lock(); bool try_lock(); ... }; This is slightly differs from what you suggest in that the lifetime of the Mutex isn't managed, but only the lock ownership. But the "wrapper" is moveable. Use cases might look like: std::mutex mut; // mutex constructed and destructed with static storage duration std::unique_lock<std::mutex> get_a_lock() { // might do something more complex in here besides // simply returning the lock return std::unique_lock<std::mutex>(mut); // lock mut and return a wrapper to it } void foo() { std::unique_lock<std::mutex> the_lock = get_a_lock(); // movable lock allows returning from factory function // mut locked in here } // mut unlocked here Some of this is C++0X, like the ability to return movable but non- copyable items from factory functions. Is this getting close to what you were wanting to do? -Howard
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tuesday 18 December 2007 13:53 pm, Howard Hinnant wrote:
Some of this is C++0X, like the ability to return movable but non- copyable items from factory functions. Is this getting close to what you were wanting to do?
I think what I can do is just put a bunch of overloaded template constructors in my wrapper, sort of like template<typename Mutex> class my_mutex_wrapper { Mutex mut_; public: my_mutex_wrapper() {//...} template<typename A1> my_mutex_wrapper(A1 a1): mut_(a1) {//...} template<typename A1, typename A2> my_mutex_wrapper(A1 a1, A2 a2): mut_(a1, a2) {//...} // ... template<typename A1, typename A2, /*...*/, typename An> my_mutex_wrapper(A1 a1, A2 a2, /*...*/, An an): mut_(a1, a2, /*...*/, an) {//...} // ... }; One of the constructors will probably be useable by the Mutex template type, and the invalid ones won't instantiated. - -- Frank -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHaDfY5vihyNWuA4URAokTAJ9mO+lt43gXO4Az4sgJ/IlBsV6z/wCeOSRp 84NqtFb3jMLPcFPgoLfQdqY= =DMu/ -----END PGP SIGNATURE-----
On Dec 18, 2007, at 4:12 PM, Frank Mori Hess wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tuesday 18 December 2007 13:53 pm, Howard Hinnant wrote:
Some of this is C++0X, like the ability to return movable but non- copyable items from factory functions. Is this getting close to what you were wanting to do?
I think what I can do is just put a bunch of overloaded template constructors in my wrapper, sort of like
template<typename Mutex> class my_mutex_wrapper { Mutex mut_; public: my_mutex_wrapper() {//...}
template<typename A1> my_mutex_wrapper(A1 a1): mut_(a1) {//...}
template<typename A1, typename A2> my_mutex_wrapper(A1 a1, A2 a2): mut_(a1, a2) {//...}
// ...
template<typename A1, typename A2, /*...*/, typename An> my_mutex_wrapper(A1 a1, A2 a2, /*...*/, An an): mut_(a1, a2, / *...*/, an) {//...}
// ... };
One of the constructors will probably be useable by the Mutex template type, and the invalid ones won't instantiated.
Oh, you are going to so love C++0X! :-) template<typename Mutex> class my_mutex_wrapper { Mutex mut_; public: template<typename ...Args> my_mutex_wrapper(Args&& ...args): mut_(std::forward<Args>(args)...) {//...} }; This does several things for you: 1. This gives you all of your constructors from the default (0 arg) up to 1000 args, or whatever the compiler limit happens to be (far more than you could ever use), with only this one constructor. 2. The arguments will be "perfectly forwarded" without copying from the client of my_mutex_wrapper to your internal Mutex. If the client passes in a const lvalue, the Mutex constructor will see a const lvalue. If the client passes in a non-const rvalue, the Mutex will see a non-const rvalue. -Howard
Anthony Williams wrote:
Andrey Semashev <andysem@mail.ru> writes:
We could just have boost::thread_move become a documented public interface, and have the type remain in boost::detail (since it *is* an implementation detail). Actually, I used it to move locks, which don't have a move member. I worked around this issue by using defer_lock_t and swap but it looks clumsy and I would surely prefer using move in such cases.
I have removed detail::thread_move(). Instead, I've added boost::move overloads for the movable types in boost.thread: thread, unique_lock<>, shared_lock<>, upgrade_lock<>. Since these overloads take specific parameter types, they shouldn't cause the problems that the unconstrained template caused before.
Thanks, nice solution.
on Sat Dec 01 2007, "Mat Marcus" <mat-lists-AT-emarcus.org> wrote:
One of our components supports move semantics. It makes use of a free function named move in namespace adobe. We were calling it unqualified from within other library routines. With boost 1.35 this will have to change. Why? Because some classes in the adobe library inherit from boost::equality_comparable. As a result, when adobe::move is called with such boost-derived classes, ADL pulls in the boost namespace and an ambiguity arises against the boost::move supplied by boost::thread. One might argue that best practice in the face of ADL is to be defensive and to avoid unqualified calls (when replaceability is not the intent). In fact, I will do just this for our next release. However this would be a little bit more pleasant if thread's boost::move was intended as a documented boost-wide move-semantics component. In fact we might consider dropping our own move component if such a boost component existed. At present however, it appears that the boost::move is more of a detail of boost/thread. Are we sure that now is the time to put move in the top level boost namespace?
These are all important questions, but IMO they deserve their own thread. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
on Fri Nov 30 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
Hi,
As promised, here's a draft of the proposed "Boost Header And Namespace Policy" (attached) Comments, suggestions, etc. very welcome.
This is really great overall, Joel! Just a few remarks...
Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net Boost Header And Namespace Policy
There has been an unwritten standard practice on header placement and namespaces that has been developed over the years. The Boost policy gives library authors a huge amount of freedom to organize and evolve their libraries as they see fit. Authors of newer libraries, however, had to try and figure out from the way existing libraries are structured or ask help from fellow boosters with more boost experience in "boostifying" their code. With very few exceptions, this worked well in the past. Library authors stayed within the bounds of the standard practice. Yet, with the rate Boost is growing, it would be crucial to put the Boost standard practice in writing to avoid confusion in the future.
This short document will attempt to formalize the Boost standard practice on header placement, subdirectory and library organization, and namespace conventions. One purpose of the review process is to raise these issues at a proper time. It would be best to add adherence to the standard practice as part of the review.
Very nice so far.
* Core libraries whose documented public interface fits in just a few headers and a few pages of documentation can put all their public headers in boost/ and their components in boost::.
I'd like to mildly discourage this practice for new core libraries. Forwarding headers work well for that purpose and keep the file structure cleaner.
For a core library named foobar, the convention is as follows:
- One or more (preferably one) header file(s). Example:
boost/foobar.hpp
- foobar components (classes, types, constants etc.) in namespace boost. Example:
namespace boost { class foo {/*...*/}; class bar {/*...*/}; }
The concept of "core library" is not new. boost::shared_ptr, for example, is a core library. It's pretty much common to all libraries. Having a core library gives us structure.
The repetition of "library/ies" is confusing. I think you should say "The structure of every moderately-sized software project depends on core components."
Typically, "core" libraries have the highest number of dependencies.
You mean "dependents"
Looking at the dependency diagram of boost, the core libraries will be at the bottommost
Graphs don't have a bottommost. How about "leaves?"
with lots of other libraries pointing acyclically at it.
I don't think one can "point acyclically" either. Why not just say "dependency DAG" above?
It's a must to avoid having them depend on other libraries in other layers above the core.
How about, "A core library must not depend on any non-core libraries?"
More emphasis and constraints must be given to these "core" libraries as they form the backbone of boost as a whole. For instance, a broken core library will have disastrous effects on the whole Boost library --core libraries should be very stable.
Determining wether a library is core can be part of the review. If the author of a library intends it to be a "core" library, he can explicitly say so and be subject for the review. A core library will have to accept more stringent requirements such as stability and non-dependence on other non-core libraries.
* Utility libraries whose documented public interface fits in just a few headers and a few pages of documentation
I would say, "in one header and less than a page of documentation."
can put all their public headers in boost/utility and their components in boost::.
Again I'd like to mildly discourage that practice for the same reasons.
For a utility library named foobar, the convention is as follows:
- One or more (preferably one) header file(s). Example:
boost/utility/foobar.hpp
- foobar components (classes, types, constants etc.) in namespace boost. Example:
namespace boost { class foo {/*...*/}; class bar {/*...*/}; }
All utility libraries pass through the same Boost review process. In certain occasions, a small library that are common to one or more Boost libraries and has already been in extensive use may undergo a "Fast track" review for inclusion as a boost utility.
* Non-core libraries may
Please add ", at the author's discretion," here.
place a single consolidated convenience header in boost/, forwarding to files of the form boost/<libraryname>/<header>.hpp. Regardless, their documented public components are not in boost/ or namespace boost::
For a library named foobar, the convention is as follows:
- A single boost/foobar.hpp that forwards to all the headers
s/A single/An optional header/
of foobar:
boost/foobar.hpp
What does writing boost/foobar.hpp again add here?
- foobar components (classes, types, constants etc.) in namespace boost::foobar. Example:
namespace boost { namespace foobar { class foo {/*...*/}; class bar {/*...*/}; }}
- A subdirectory boost/foobar where all foobar headers are placed.
* Un-reviewed implementation details of libraries have been placed in boost/detail if they need to be used by other libraries and a subdirectory of boost/<libraryname>/ otherwise.
I don't think that's right. What usually happens is that they get placed in a subdirectory of boost/<libraryname>/ and then are moved to boost/detail when someone discovers they want to use use them outside of <libraryname>.
Their documented public components are placed in boost/detail and namespace boost::detail.
what public components? These are un-reviewed implementation details. I'm confused.
* Libraries may host sub libraries that may be used by other boost
nit: "sub" is not a word, so that should be "sublibraries" or "sub-libraries"
libraries. Such sub libraries are typically meant for future boost review. Their documented public components are placed in boost/<host-libraryname>/<libraryname>/<header>.hpp and namespace boost::<libraryname>.
One such example is the proto library that is hosted by xpressive. Another is fusion. The fusion library was once hosted by spirit. After passing the boost formal review the fusion library is now a full fledged boost library.
Again, this is really great. I think while we're at it, we'd better describe what happens under libs/, too. But if you like, we can nail down this part first. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
on Fri Nov 30 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
Hi,
As promised, here's a draft of the proposed "Boost Header And Namespace Policy" (attached) Comments, suggestions, etc. very welcome.
This is really great overall, Joel! Just a few remarks...
Thanks! I'll apply your comments (and edits) and come up with a more refined version after some more issues are ironed out.
Again, this is really great. I think while we're at it, we'd better describe what happens under libs/, too. But if you like, we can nail down this part first.
Ok, I'll see what I can do. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman <joel@boost-consulting.com> writes:
* Non-core libraries may place a single consolidated convenience header in boost/, forwarding to files of the form boost/<libraryname>/<header>.hpp. Regardless, their documented public components are not in boost/ or namespace boost::
What about boost.thread? Though the "general use" header is boost/thread.hpp, and the other headers are in boost/thread/xyz.hpp, the public interface is and always has been in namespace boost::, and changing this now would be a breaking change. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
Anthony Williams wrote:
Joel de Guzman <joel@boost-consulting.com> writes:
* Non-core libraries may place a single consolidated convenience header in boost/, forwarding to files of the form boost/<libraryname>/<header>.hpp. Regardless, their documented public components are not in boost/ or namespace boost::
What about boost.thread? Though the "general use" header is boost/thread.hpp, and the other headers are in boost/thread/xyz.hpp, the public interface is and always has been in namespace boost::, and changing this now would be a breaking change.
Seems that if thread is a non-core library then it indeed breaks the "standard practice". I don't think a breaking change is a good idea though. I'd like to hear other people's thoughts on this. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
Joel de Guzman wrote:
Anthony Williams wrote:
Joel de Guzman <joel@boost-consulting.com> writes:
* Non-core libraries may place a single consolidated convenience header in boost/, forwarding to files of the form boost/<libraryname>/<header>.hpp. Regardless, their documented public components are not in boost/ or namespace boost:: What about boost.thread? Though the "general use" header is boost/thread.hpp, and the other headers are in boost/thread/xyz.hpp, the public interface is and always has been in namespace boost::, and changing this now would be a breaking change.
Seems that if thread is a non-core library then it indeed breaks the "standard practice". I don't think a breaking change is a good idea though. I'd like to hear other people's thoughts on this.
Maybe the library should reside in its own sub-namespace and then this namespace should be imported into boost:: with using directive. This import would be optional, depending on some macro, and would be declared deprecated.
on Mon Dec 03 2007, Andrey Semashev <andysem-AT-mail.ru> wrote:
Joel de Guzman wrote:
Anthony Williams wrote:
Joel de Guzman <joel@boost-consulting.com> writes:
* Non-core libraries may place a single consolidated convenience header in boost/, forwarding to files of the form boost/<libraryname>/<header>.hpp. Regardless, their documented public components are not in boost/ or namespace boost:: What about boost.thread? Though the "general use" header is boost/thread.hpp, and the other headers are in boost/thread/xyz.hpp, the public interface is and always has been in namespace boost::, and changing this now would be a breaking change.
Seems that if thread is a non-core library then it indeed breaks the "standard practice". I don't think a breaking change is a good idea though. I'd like to hear other people's thoughts on this.
Maybe the library should reside in its own sub-namespace and then this namespace should be imported into boost:: with using directive. This import would be optional, depending on some macro, and would be declared deprecated.
More simply, boost/thread.hpp could do #include "boost/thread/whatever.hpp" #include "boost/thread/whateverelse.hpp" namespace boost { using namespace threads; } -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
on Mon Dec 03 2007, Andrey Semashev <andysem-AT-mail.ru> wrote:
Maybe the library should reside in its own sub-namespace and then this namespace should be imported into boost:: with using directive. This import would be optional, depending on some macro, and would be declared deprecated.
More simply, boost/thread.hpp could do
#include "boost/thread/whatever.hpp" #include "boost/thread/whateverelse.hpp" namespace boost { using namespace threads; }
Yep, or even better: #ifndef BOOST_THREAD_NO_IMPORTS namespace boost { using namespace threads; } #endif
Joel de Guzman <joel@boost-consulting.com> writes:
Anthony Williams wrote:
Joel de Guzman <joel@boost-consulting.com> writes:
* Non-core libraries may place a single consolidated convenience header in boost/, forwarding to files of the form boost/<libraryname>/<header>.hpp. Regardless, their documented public components are not in boost/ or namespace boost::
What about boost.thread? Though the "general use" header is boost/thread.hpp, and the other headers are in boost/thread/xyz.hpp, the public interface is and always has been in namespace boost::, and changing this now would be a breaking change.
Seems that if thread is a non-core library then it indeed breaks the "standard practice". I don't think a breaking change is a good idea though. I'd like to hear other people's thoughts on this.
Joel de Guzman <joel@boost-consulting.com> also writes:
Yes, I agree that everything that is supposed to be proposed for inclusion into the standard library should be at the "core".
That includes boost.thread. I'll leave things alone for now. Anthony -- Anthony Williams Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk Registered in England, Company Number 5478976. Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL
on Mon Dec 03 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
Anthony Williams wrote:
Joel de Guzman <joel@boost-consulting.com> writes:
* Non-core libraries may place a single consolidated convenience header in boost/, forwarding to files of the form boost/<libraryname>/<header>.hpp. Regardless, their documented public components are not in boost/ or namespace boost::
What about boost.thread? Though the "general use" header is boost/thread.hpp, and the other headers are in boost/thread/xyz.hpp, the public interface is and always has been in namespace boost::, and changing this now would be a breaking change.
Seems that if thread is a non-core library then it indeed breaks the "standard practice".
I think our practices of namespace subdivision were only getting established when Boost.Thread was introduced. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
Hello Joel, sorry if the issues below have already discussed, I don't have time right now to review the whole thread. ----- Mensaje original ----- De: Joel de Guzman <joel@boost-consulting.com> Fecha: Sábado, Diciembre 1, 2007 5:48 am Asunto: [boost] [RFC] Header/namespace policy draft Para: boost@lists.boost.org
Hi,
As promised, here's a draft of the proposed "Boost Header And Namespace Policy" (attached) Comments, suggestions, etc. very welcome. [...] * Non-core libraries may place a single consolidated convenience header in boost/, forwarding to files of the form boost/<libraryname>/<header>.hpp. Regardless, their documented public components are not in boost/ or namespace boost::
For a library named foobar, the convention is as follows:
- A single boost/foobar.hpp that forwards to all the headers of foobar:
boost/foobar.hpp
I see a couple of problems with the requirements that *all* Boost.Foobar headers be included from boost/foobar.hpp: 1. If the library is huge or it consists of several components among which the user has to select some for her specific needs, having a include-all header can lead to unnecesary compiler load. 2. Some functionality (serialization comes to mind) is best isolated to specific headers that the user has to explicitly include to bring that funcionality in, so as to not incur in unnecesay dependencies. So, I'd prefer that boost/foobar.hpp (if it exists) be regarded as a convenience header that includes all the headers in Boost.Foobar (when it's OK to do so) *or* a selection of the most usual components from the library.
- foobar components (classes, types, constants etc.) in namespace boost::foobar. Example:
namespace boost { namespace foobar { class foo {/*...*/}; class bar {/*...*/}; }}
This is not the case at least for one Boost library, namely Boost.Tuple, whose associated namespace is boost::tuples, with a postfixed 's'. The reason for this anomaly is that some compilers have problems with classes living in homonyn namespaces, a la boost::foobar::foobar, see http://www.boost.org/libs/tuple/doc/design_decisions_rationale.html Cheers, Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
JOAQUIN LOPEZ MU?Z:
namespace boost { namespace foobar
...
This is not the case at least for one Boost library, namely Boost.Tuple, whose associated namespace is boost::tuples, with a postfixed 's'.
It's also not the case for std::tuple, whose namespace is std, without a subnamespace. :-) One benefit of having everything in boost (or, more precisely, everything that is supposed to be proposed for inclusion into the standard library) is that name collisions become apparent earlier.
Peter Dimov wrote:
JOAQUIN LOPEZ MU?Z:
namespace boost { namespace foobar
...
This is not the case at least for one Boost library, namely Boost.Tuple, whose associated namespace is boost::tuples, with a postfixed 's'.
It's also not the case for std::tuple, whose namespace is std, without a subnamespace. :-)
One benefit of having everything in boost (or, more precisely, everything that is supposed to be proposed for inclusion into the standard library) is that name collisions become apparent earlier.
Boost.Tuple should have been a "core" library. Yes, I agree that everything that is supposed to be proposed for inclusion into the standard library should be at the "core". I'll add this in the draft. Thanks for the note. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
on Mon Dec 03 2007, Joel de Guzman <joel-AT-boost-consulting.com> wrote:
Peter Dimov wrote:
JOAQUIN LOPEZ MU?Z:
namespace boost { namespace foobar
...
This is not the case at least for one Boost library, namely Boost.Tuple, whose associated namespace is boost::tuples, with a postfixed 's'.
That convention is actually documented at http://www.boost.org/more/lib_guide.htm#Naming_consistency already. See also http://lists.boost.org/Archives/boost/2001/06/13360.php
It's also not the case for std::tuple, whose namespace is std, without a subnamespace. :-)
One benefit of having everything in boost (or, more precisely, everything that is supposed to be proposed for inclusion into the standard library) is that name collisions become apparent earlier.
Boost.Tuple should have been a "core" library.
It can be considered a "core" library. Remember, the "standard practice" is that a *small* core library *may* place its components directly in namespace boost, not that it must do so. And, as far as I'm concerned, taking advantage of that leeway should be discouraged.
From my point of view, it's mostly a way of legitimizing legacy design choices that have long standing so we don't disrupt the whole universe by trying to move boost::shared_ptr.
Yes, I agree that everything that is supposed to be proposed for inclusion into the standard library should be at the "core".
I'm not sure why that's a good idea, and furthermore I don't think it's tenable. We'll sometimes make decisions about attempts to standardize components long after a library has been accepted. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
on Mon Dec 03 2007, "Peter Dimov" <pdimov-AT-pdimov.com> wrote:
JOAQUIN LOPEZ MU?Z:
namespace boost { namespace foobar
...
This is not the case at least for one Boost library, namely Boost.Tuple, whose associated namespace is boost::tuples, with a postfixed 's'.
It's also not the case for std::tuple, whose namespace is std, without a subnamespace. :-)
How is that relevant? std::tuple is outside our purview.
One benefit of having everything in boost (or, more precisely, everything that is supposed to be proposed for inclusion into the standard library) is that name collisions become apparent earlier.
I don't see how such collisions will be revealed just by virtue of the names occupying the same namespace, and it seems premature to constrain two unrelated libraries to use entirely different names just because they might one day be standardized together. The committee has to make all kinds of small adjustments when libraries are standardized anyway. Resolving name collisions then is not such a big deal, I think. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
JOAQUIN LOPEZ MU?Z wrote:
Hello Joel, sorry if the issues below have already discussed, I don't have time right now to review the whole thread.
No problem. Your comments are very welcome!
As promised, here's a draft of the proposed "Boost Header And Namespace Policy" (attached) Comments, suggestions, etc. very welcome. [...] * Non-core libraries may place a single consolidated convenience header in boost/, forwarding to files of the form boost/<libraryname>/<header>.hpp. Regardless, their documented public components are not in boost/ or namespace boost::
For a library named foobar, the convention is as follows:
- A single boost/foobar.hpp that forwards to all the headers of foobar:
boost/foobar.hpp
I see a couple of problems with the requirements that *all* Boost.Foobar headers be included from boost/foobar.hpp:
1. If the library is huge or it consists of several components among which the user has to select some for her specific needs, having a include-all header can lead to unnecesary compiler load. 2. Some functionality (serialization comes to mind) is best isolated to specific headers that the user has to explicitly include to bring that funcionality in, so as to not incur in unnecesay dependencies.
So, I'd prefer that boost/foobar.hpp (if it exists) be regarded as a convenience header that includes all the headers in Boost.Foobar (when it's OK to do so) *or* a selection of the most usual components from the library.
Those are good points. I'll add them in the document. Fusion too does not have a convenience header, for the same reason. That's the real intent of this item. Thanks for clarifying.
- foobar components (classes, types, constants etc.) in namespace boost::foobar. Example:
namespace boost { namespace foobar { class foo {/*...*/}; class bar {/*...*/}; }}
This is not the case at least for one Boost library, namely Boost.Tuple, whose associated namespace is boost::tuples, with a postfixed 's'. The reason for this anomaly is that some compilers have problems with classes living in homonyn namespaces, a la boost::foobar::foobar, see
Indeed. I'll add that too. As for all rules, there are exceptions. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
on Fri Nov 16 2007, Beman Dawes <bdawes-AT-acm.org> wrote:
For the headers that appear in boost-root/boost, rather than a library specific subdirectory, it is very helpful if they self-identify what library they belong to. This allows scripts to associate the header with a library.
The usual way to do this is to include a comment. A typical example:
// See http://www.boost.org/libs/config for documentation
The exact format doesn't matter. What does matter is that the library name be found somewhere in the file, immediately following a string which contains "/libs/".
Below is the result of doing:
grep -L "/libs/" *.h*
I'd appreciate maintainers adding a library identifying comment to these files. Otherwise I'll do it, which carries a slight risk that I'll get it wrong.
Without giving precise directions for identifying the library (not just a typical example), I don't see how the result can be useful to scripts. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
on Fri Nov 16 2007, Beman Dawes <bdawes-AT-acm.org> wrote:
For the headers that appear in boost-root/boost, rather than a library specific subdirectory, it is very helpful if they self-identify what library they belong to. This allows scripts to associate the header with a library.
The usual way to do this is to include a comment. A typical example:
// See http://www.boost.org/libs/config for documentation
The exact format doesn't matter. What does matter is that the library name be found somewhere in the file, immediately following a string which contains "/libs/".
Below is the result of doing:
grep -L "/libs/" *.h*
I'd appreciate maintainers adding a library identifying comment to these files. Otherwise I'll do it, which carries a slight risk that I'll get it wrong.
Without giving precise directions for identifying the library (not just a typical example), I don't see how the result can be useful to scripts.
Why not enforce some Doxygen rules for such things among Boost developers, which would help Python scripts find what is necessary in header files ?
David Abrahams wrote:
on Fri Nov 16 2007, Beman Dawes <bdawes-AT-acm.org> wrote:
For the headers that appear in boost-root/boost, rather than a library specific subdirectory, it is very helpful if they self-identify what library they belong to. This allows scripts to associate the header with a library.
The usual way to do this is to include a comment. A typical example:
// See http://www.boost.org/libs/config for documentation
The exact format doesn't matter. What does matter is that the library name be found somewhere in the file, immediately following a string which contains "/libs/".
Below is the result of doing:
grep -L "/libs/" *.h*
I'd appreciate maintainers adding a library identifying comment to these files. Otherwise I'll do it, which carries a slight risk that I'll get it wrong.
Without giving precise directions for identifying the library (not just a typical example), I don't see how the result can be useful to scripts.
The usual approach is to look for "www.boost.org/libs/" and assume the next substring up to a "/" or space is the library name. This is used, for example, by the inspect tool. I use something similar in a bash script when I want to do a bunch of svn or other commands on a per library basis, using grep to find the library name for each header in boost-root/boost, and then a sed regular expression to turn the resulting header name and library name into the command I'm after. Saves a great deal of typing. --Beman
participants (17)
-
"JOAQUIN LOPEZ MU?Z" -
Andrey Semashev -
Anthony Williams -
Beman Dawes -
David Abrahams -
Edward Diener -
Frank Mori Hess -
Giovanni Piero Deretta -
Hartmut Kaiser -
Howard Hinnant -
Joel de Guzman -
John Maddock -
Jonathan Biggar -
Mat Marcus -
Peter Dimov -
Robert Ramey -
Sohail Somani