[type_traits] Rewrite and dependency free version

John Maddock

14 Jan 2015 14 Jan '15

7:58 p.m.

I'm going to throw this out there to see what folks think: In a branch called "Version2", see https://github.com/boostorg/type_traits/tree/Version2 is a partially rewritten type_traits lib: it doesn't yet contain all the traits of the old one, but there's a pretty good selection to test. The main pros for this version are: 1) Zero dependencies other than Boost.Config. 2) No horrible macro-obfuscation - all "real code", making it easier to read and maintain. 3) Substantially reduced template instantiations where practical - often half as many as before. 4) Substantially reduced #includes - the aim is that all files should include only what they need and no more - with what's needed determined by the actual implementation variant when appropriate. There were also a lot of spurious #includes left over from old workarounds no longer used. The main cons: 1) I don't know what I've broken, but surely something, you can help by testing but I would guess we would never be 100% sure it's all right until it's out in the field. This is the main reason for avoiding this until now. 2) MPL interoperability should still work, but files which expect type_traits headers to have included mpl ones will be broken unless they explicitly include what they need. 3) Current MPL inter-operability is fragile see https://github.com/boostorg/type_traits/blob/Version2/include/boost/type_tra... 4) Function dispatching to an mpl::bool_ involves an (inline) function call where as the old inheritance scheme did not. This should mostly disappear once optimizations are turned on, but is still a potential issue for some folks I suspect. 5) I haven't got to the "difficult" headers like common_type.hpp yet. 6) Whatever I've forgotten ;) Of course (2), (3) and (4) could be solved by making mpl's bool.hpp some kind of "core" header. So... what I need from you kind folks is: * Is this the right way forward? * Can you test with your favourite compilers and compare with results for develop - So far I've tested with msvc-10,11,12,14 Intel 14,15, GCC-4.5.4...4.9.1, plus a reasonably recent clang. * If anyone would like to help converting further traits all help would be much appreciated.... Oh and the compiler requirements in the docs are all wildly out of date too... Regards, John.

Show replies by date

Stephen Kelly

14 Jan 14 Jan

8:04 p.m.

John Maddock wrote:

...

So... what I need from you kind folks is:

* Is this the right way forward?

Thanks for your work on this! I haven't looked into it yet but I will, and I'll see where I can help. Do you intend this to replace the current type_traits implementation, or do you expect a future version of boost to ship both? Thanks, Steve.

John Maddock

8:24 p.m.

...

...
So... what I need from you kind folks is:

* Is this the right way forward?

Thanks for your work on this!

I haven't looked into it yet but I will, and I'll see where I can help.

Do you intend this to replace the current type_traits implementation, or do you expect a future version of boost to ship both?

Originally I was thinking of shipping both with the new as the default, and a "legacy" version for those who need it. But as I haven't found any breakages... yet.... I'm hoping that may not be needed. We'll see. Whatever, any transition would have to be very carefully handled. John.

Ion Gaztañaga

9:33 p.m.

El 14/01/2015 a las 20:58, John Maddock escribió:

...

I'm going to throw this out there to see what folks think:

In a branch called "Version2", see https://github.com/boostorg/type_traits/tree/Version2 is a partially rewritten type_traits lib: it doesn't yet contain all the traits of the old one, but there's a pretty good selection to test.

Very nice.

...

The main pros for this version are:

1) Zero dependencies other than Boost.Config.

We still have Boost.PP.

...

4) Substantially reduced #includes - the aim is that all files should include only what they need and no more - with what's needed determined by the actual implementation variant when appropriate. There were also a lot of spurious #includes left over from old workarounds no longer used.

For "is_function", you might test (I could help once things are stabilized) another approach that could avoid the use of Boost.PP: //For a function to pointer an lvalue of function type T //can be implicitly converted to a prvalue //pointer to that function. This does not apply to //non-static member functions because lvalues //that refer to non-static member functions do not exist. template <class T> struct is_reference_convertible_to_pointer { struct twochar { char dummy[2]; }; template <class U> static char test(U*); template <class U> static twochar test(...); static T& source(); static const bool value = sizeof(char) == sizeof(test<T>(source())); }; //Filter out: // - class types that might have implicit conversions // - void (to avoid forming a reference to void later) // - references (e.g.: filtering reference to functions) // - nullptr_t (convertible to pointer) template < class T , bool Filter = is_class_or_union<T>::value || is_void<T>::value || is_reference<T>::value || is_nullptr_t<T>::value > struct is_function_impl { static const bool value = is_reference_convertible_to_pointer<T>::value; }; template <class T> struct is_function_impl { static const bool value = false; }; template <class T> struct is_function : is_function_impl<T> {}; This could allow us to avoid the preprocessor also in "is_member_function_pointer": template <class T> struct is_member_function_pointer_cv { static const bool value = false; }; template struct is_member_function_pointer_cv : is_function<T> {}; template <class T> struct is_member_function_pointer : is_member_function_pointer_cv {}; Best, Ion

John Maddock

15 Jan 15 Jan

9:04 a.m.

...

Very nice.

...
The main pros for this version are:

1) Zero dependencies other than Boost.Config.

We still have Boost.PP.

No, just to be clear: *there is absolutely no dependency on the PP lib*. It is true that there are a couple of headers which have a "maintenance mode" that includes PP headers - but the result needs a search and replace afterwards to produce valid C++ code. No user code can ever include a PP header via type_traits. Ever. To make this clearer, one thing I considered was archiving the PP generating code somewhere, and removing it from the headers. John.

Ion Gaztañaga

10:49 a.m.

El 15/01/2015 a las 10:04, John Maddock escribió:

...

...
Very nice.

...
The main pros for this version are:

1) Zero dependencies other than Boost.Config.

We still have Boost.PP.

No, just to be clear: *there is absolutely no dependency on the PP lib*.

It is true that there are a couple of headers which have a "maintenance mode" that includes PP headers - but the result needs a search and replace afterwards to produce valid C++ code. No user code can ever include a PP header via type_traits. Ever.

To make this clearer, one thing I considered was archiving the PP generating code somewhere, and removing it from the headers.

Ah! Sorry, I see what you say. In any case with my alternative implementation you could avoid even Boost.PP for manteinance and avoid all those overloads. In any case, this should be a possible improvement for the future. Many thanks for the explanations Ion

Peter Dimov

2:34 p.m.

Ion Gaztañaga wrote:

...

In any case with my alternative implementation you could avoid even Boost.PP for manteinance and avoid all those overloads.

My first thought when looking at is_function was also to suggest "is_convertible", but then I saw that Ion has already got to it first, complete with the nullptr_t check.

Agustín K-ballo Bergé

5:39 p.m.

On 1/15/2015 11:34 AM, Peter Dimov wrote:

...

Ion Gaztañaga wrote:

...
In any case with my alternative implementation you could avoid even Boost.PP for manteinance and avoid all those overloads.

My first thought when looking at is_function was also to suggest "is_convertible", but then I saw that Ion has already got to it first, complete with the nullptr_t check.

This doesn't quite work as `void() const`, and any other combination of cv and ref qualifiers, is a function type but cannot be instantiated. Regards, -- Agustín K-ballo Bergé.- http://talesofcpp.fusionfenix.com

Peter Dimov

5:55 p.m.

Agustín K-ballo Bergé wrote:

...

On 1/15/2015 11:34 AM, Peter Dimov wrote:

...
Ion Gaztañaga wrote:

...
In any case with my alternative implementation you could avoid even Boost.PP for manteinance and avoid all those overloads.

My first thought when looking at is_function was also to suggest "is_convertible", but then I saw that Ion has already got to it first, complete with the nullptr_t check.

This doesn't quite work as `void() const`, and any other combination of cv and ref qualifiers, is a function type but cannot be instantiated.

You're right. The current PP-based implementation doesn't catch those either, though. Fortunately, if a compiler supports 'void () const &&', it probably has std::is_function as well, so we can just use that. :-) Although, now that I think of it, creating 'void() const' is in principle possible in C++03 as well. Not sure what the old type traits did with such types.

Peter Dimov

6:04 p.m.

...

Although, now that I think of it, creating 'void() const' is in principle possible in C++03 as well. Not sure what the old type traits did with such types.

boost::is_function says 0, even in C++11 mode. #include #include #include <iostream> #include <typeinfo> struct X { void f() const {} }; template< class T > struct Y { }; template< class C, class F > void test( F C::* /*pf*/ ) { std::cout << boost::core::demangle( typeid( F ).name() ) << std::endl; std::cout << boost::core::demangle( typeid( Y<F> ).name() ) << std::endl; std::cout << boost::is_function< F >::value << std::endl; } int main() { test( &X::f ); }

John Maddock

6:13 p.m.

...

...
Although, now that I think of it, creating 'void() const' is in principle possible in C++03 as well. Not sure what the old type traits did with such types.

boost::is_function says 0, even in C++11 mode.

#include #include #include <iostream> #include <typeinfo>

struct X { void f() const {} };

template< class T > struct Y { };

template< class C, class F > void test( F C::* /*pf*/ ) { std::cout << boost::core::demangle( typeid( F ).name() ) << std::endl; std::cout << boost::core::demangle( typeid( Y<F> ).name() ) << std::endl; std::cout << boost::is_function< F >::value << std::endl; }

int main() { test( &X::f ); }

Peter, can you file a bug report so this doesn't get lost? I'm trying hard to *not* change semantics or implementation strategy at this stage. One thing at a time! Thanks, John.

Peter Dimov

7:14 p.m.

John Maddock wrote:

...

...
boost::is_function says 0, even in C++11 mode.

Peter, can you file a bug report so this doesn't get lost?

Sure, https://svn.boost.org/trac/boost/ticket/10934 While looking at this, I also noticed that BOOST_NO_CXX11_HDR_TYPE_TRAITS is set for g++ and clang, even though they do have . The config test fails because they don't have is_trivially_* and aligned_union. Fair enough, but the rest of the traits are there and usable, so I'm not entirely sure about this. Of course, if we get the new and improved Boost.Type_traits, there might be no reason to ever use the std:: ones directly, provided that the Boost traits are never inferior to the std:: ones. (That is, where there's disagreement, we should be correct and the std:: trait wrong. :-) )

John Maddock

16 Jan 16 Jan

9:46 a.m.

...

While looking at this, I also noticed that BOOST_NO_CXX11_HDR_TYPE_TRAITS is set for g++ and clang, even though they do have . The config test fails because they don't have is_trivially_* and aligned_union. Fair enough, but the rest of the traits are there and usable, so I'm not entirely sure about this.

Me neither. Basically no one's complained so I've left it set. <shrug> I guess. Could go either way on that. John.

Andrey Semashev

9:53 a.m.

On Friday 16 January 2015 09:46:42 John Maddock wrote:

...

...
While looking at this, I also noticed that BOOST_NO_CXX11_HDR_TYPE_TRAITS is set for g++ and clang, even though they do have . The config test fails because they don't have is_trivially_* and aligned_union. Fair enough, but the rest of the traits are there and usable, so I'm not entirely sure about this.

Me neither. Basically no one's complained so I've left it set. <shrug> I guess. Could go either way on that. John.

AFAIR, by convention we set HDR macros when the header is fully usable. I think that's fair. We could introduce other sub-feature macros for finer grained checks, like BOOST_NO_CXX11_TYPE_INSPECTION_TRAITS, BOOST_NO_CXX11_TYPE_TRANSFORM_TRAITS, etc.

Peter Dimov

4:36 p.m.

Andrey Semashev wrote:

...

AFAIR, by convention we set HDR macros when the header is fully usable.

When the HDR macro is set, the #include is not guaranteed to compile, so the header can't be used and there is no point in defining any other macros.

John Maddock

5:35 p.m.

...

...
AFAIR, by convention we set HDR macros when the header is fully usable.

When the HDR macro is set, the #include is not guaranteed to compile, so the header can't be used and there is no point in defining any other macros.

The convention we've used in the past, is that the feature macro is unset once the feature becomes "usable enough", then add some extra macros to cover bugs, corner cases, and other such things. So I guess the question is at what point do we say that header is good enough? John.

Tim Blechmann

1 Feb 1 Feb

12:26 p.m.

...

...
...
While looking at this, I also noticed that BOOST_NO_CXX11_HDR_TYPE_TRAITS is set for g++ and clang, even though they do have . The config test fails because they don't have is_trivially_* and aligned_union. Fair enough, but the rest of the traits are there and usable, so I'm not entirely sure about this.

Me neither. Basically no one's complained so I've left it set. <shrug> I guess. Could go either way on that. John.

AFAIR, by convention we set HDR macros when the header is fully usable. I think that's fair. We could introduce other sub-feature macros for finer grained checks, like BOOST_NO_CXX11_TYPE_INSPECTION_TRAITS, BOOST_NO_CXX11_TYPE_TRANSFORM_TRAITS, etc.

not sure: with <atomic> BOOST_NO_CXX11_HDR_ATOMIC is not defined for some configurations which do not provide a complete implementation of std::atomic<>: clang/libc++ only very recently managed to provide a completely usable atomic<>, while other toolchains only provided a partial implementation for quite some time ... however it would definitely be a good convention only to define this macro once a header is fully implemented. tim

Edward Diener

15 Jan 15 Jan

12:17 a.m.

On 1/14/2015 2:58 PM, John Maddock wrote:

...

I'm going to throw this out there to see what folks think:

In a branch called "Version2", see https://github.com/boostorg/type_traits/tree/Version2 is a partially rewritten type_traits lib: it doesn't yet contain all the traits of the old one, but there's a pretty good selection to test.

How do I get this version in my local modular boost to test ? Won't it conflict with the type_traits already there if I just clone it directly ?

...

The main pros for this version are:

1) Zero dependencies other than Boost.Config. 2) No horrible macro-obfuscation - all "real code", making it easier to read and maintain. 3) Substantially reduced template instantiations where practical - often half as many as before. 4) Substantially reduced #includes - the aim is that all files should include only what they need and no more - with what's needed determined by the actual implementation variant when appropriate. There were also a lot of spurious #includes left over from old workarounds no longer used.

The main cons:

1) I don't know what I've broken, but surely something, you can help by testing but I would guess we would never be 100% sure it's all right until it's out in the field. This is the main reason for avoiding this until now. 2) MPL interoperability should still work, but files which expect type_traits headers to have included mpl ones will be broken unless they explicitly include what they need.

Anyone who assumes header files will be included, rather then include what's needed, for any C++ software deserves to be broken.

...

3) Current MPL inter-operability is fragile see https://github.com/boostorg/type_traits/blob/Version2/include/boost/type_tra...

4) Function dispatching to an mpl::bool_ involves an (inline) function call where as the old inheritance scheme did not. This should mostly disappear once optimizations are turned on, but is still a potential issue for some folks I suspect. 5) I haven't got to the "difficult" headers like common_type.hpp yet. 6) Whatever I've forgotten ;)

Of course (2), (3) and (4) could be solved by making mpl's bool.hpp some kind of "core" header.

So... what I need from you kind folks is:

* Is this the right way forward? * Can you test with your favourite compilers and compare with results for develop - So far I've tested with msvc-10,11,12,14 Intel 14,15, GCC-4.5.4...4.9.1, plus a reasonably recent clang.

I can test with some earlier versions of gcc as well as msvc-8 and msvc-9 as soon as I get the branch.

...

* If anyone would like to help converting further traits all help would be much appreciated.... Oh and the compiler requirements in the docs are all wildly out of date too...

Edward Diener

12:25 a.m.

On 1/14/2015 7:17 PM, Edward Diener wrote:

...

On 1/14/2015 2:58 PM, John Maddock wrote:

...
I'm going to throw this out there to see what folks think:

In a branch called "Version2", see https://github.com/boostorg/type_traits/tree/Version2 is a partially rewritten type_traits lib: it doesn't yet contain all the traits of the old one, but there's a pretty good selection to test.

How do I get this version in my local modular boost to test ? Won't it conflict with the type_traits already there if I just clone it directly ?

Ignore. Tortoise Git, which I use on Windows, didn't see the branch until I browsed the log. Then i was able to switch and track it.

Peter Dimov

12:51 a.m.

John Maddock wrote:

...

I'm going to throw this out there to see what folks think:

They very much approve of, and thank you for, your work. :-) This is so much better. :-)

...

In a branch called "Version2", see https://github.com/boostorg/type_traits/tree/Version2 is a partially rewritten type_traits lib: it doesn't yet contain all the traits of the old one, but there's a pretty good selection to test.

I wonder why you have decided to return a reference to mpl::bool_, instead of just ("stupidly") returning by value. Compilers like "stupid" code, they optimize it better. Typically, the use case is: void f( mpl::true_ ); void f( mpl::false_ ); f( is_pointer<X>() ); and if you return by value, it can construct directly into the argument, whereas by reference it needs to use the copy constructor. That said, I still think that it may be better to add the conversion on the MPL side; this will obviate the need for the fragile forward declarations (and it will support any integral constants, not just this one). We can, of course, in addition add a converting constructor to integral_constant as well, so that the "officially blessed" way to dispatch would be on true_type and false_type. This dispatch will work on std:: traits as well.

John Maddock

9 a.m.

...

...
In a branch called "Version2", see https://github.com/boostorg/type_traits/tree/Version2 is a partially rewritten type_traits lib: it doesn't yet contain all the traits of the old one, but there's a pretty good selection to test.

I wonder why you have decided to return a reference to mpl::bool_, instead of just ("stupidly") returning by value. Compilers like "stupid" code, they optimize it better. Typically, the use case is:

It was the only way to break the dependency to MPL (without modifying MPL that is). Can't return a forward declaration by value...

...

void f( mpl::true_ ); void f( mpl::false_ );

f( is_pointer<X>() );

and if you return by value, it can construct directly into the argument, whereas by reference it needs to use the copy constructor.

That said, I still think that it may be better to add the conversion on the MPL side; this will obviate the need for the fragile forward declarations (and it will support any integral constants, not just this one).

We can, of course, in addition add a converting constructor to integral_constant as well, so that the "officially blessed" way to dispatch would be on true_type and false_type. This dispatch will work on std:: traits as well.

+1. John.

Edward Diener

1:05 a.m.

On 1/14/2015 2:58 PM, John Maddock wrote:

...

I'm going to throw this out there to see what folks think:

In a branch called "Version2", see https://github.com/boostorg/type_traits/tree/Version2 is a partially rewritten type_traits lib: it doesn't yet contain all the traits of the old one, but there's a pretty good selection to test.

The main pros for this version are:

1) Zero dependencies other than Boost.Config. 2) No horrible macro-obfuscation - all "real code", making it easier to read and maintain. 3) Substantially reduced template instantiations where practical - often half as many as before. 4) Substantially reduced #includes - the aim is that all files should include only what they need and no more - with what's needed determined by the actual implementation variant when appropriate. There were also a lot of spurious #includes left over from old workarounds no longer used.

The main cons:

1) I don't know what I've broken, but surely something, you can help by testing but I would guess we would never be 100% sure it's all right until it's out in the field. This is the main reason for avoiding this until now. 2) MPL interoperability should still work, but files which expect type_traits headers to have included mpl ones will be broken unless they explicitly include what they need. 3) Current MPL inter-operability is fragile see https://github.com/boostorg/type_traits/blob/Version2/include/boost/type_tra...

4) Function dispatching to an mpl::bool_ involves an (inline) function call where as the old inheritance scheme did not. This should mostly disappear once optimizations are turned on, but is still a potential issue for some folks I suspect. 5) I haven't got to the "difficult" headers like common_type.hpp yet. 6) Whatever I've forgotten ;)

Of course (2), (3) and (4) could be solved by making mpl's bool.hpp some kind of "core" header.

So... what I need from you kind folks is:

* Is this the right way forward? * Can you test with your favourite compilers and compare with results for develop - So far I've tested with msvc-10,11,12,14 Intel 14,15, GCC-4.5.4...4.9.1, plus a reasonably recent clang.

Testing gcc on Windows 7 with mingw and mingw64: Running type_traits tests passed with msvc-8.0, msvc-9.0, gcc-4.4.0, gcc-4.5.0, gcc-4.5.2, and gcc-4.9.2. Failed with gcc-4.3.0 with: "testing.capture-output ..\..\..\bin.v2\libs\type_traits\test\has_nothrow_assign_test.test\gcc-mingw-4.3.0\debug\has_nothrow_assign_test.run ====== BEGIN OUTPUT ====== has_nothrow_assign_test.cpp:204: The expression: "::boost::has_nothrow_assign::value" had an invalid value (found 1, expected 0)" In type_traits with gcc versions below 4 there are lots of failures of the order of: "gcc.compile.c++ ..\..\..\bin.v2\libs\type_traits\test\is_virtual_base_of_test.test\gcc-mingw-3.4.5\debug\is_virtual_base_of_test.o ../../../boost/type_traits/is_virtual_base_of.hpp: In instantiation of `boost::detail::is_virtual_base_of_impl

...

::boost_type_traits_internal_struct_X': ../../../boost/type_traits/is_virtual_base_of.hpp:65: instantiated from `boost::detail::is_virtual_base_of_impl >' ../../../boost/type_traits/is_virtual_base_of.hpp:73: instantiated from `boost::detail::is_virtual_base_of_impl2' ../../../boost/type_traits/is_virtual_base_of.hpp:82: instantiated from `boost::is_virtual_base_of'"

etc. Of course I can't imagine anyone still using those versions. The function_types library depends on some type_traits implementation which is not in Version2, and tti uses function_types etc so tti fails.

...

* If anyone would like to help converting further traits all help would be much appreciated.... Oh and the compiler requirements in the docs are all wildly out of date too...

Where are the compiler requirements for type_traits mentioned ?

John Maddock

8:58 a.m.

...

Testing gcc on Windows 7 with mingw and mingw64:

Running type_traits tests passed with msvc-8.0, msvc-9.0, gcc-4.4.0, gcc-4.5.0, gcc-4.5.2, and gcc-4.9.2.

Failed with gcc-4.3.0 with:

"testing.capture-output ..\..\..\bin.v2\libs\type_traits\test\has_nothrow_assign_test.test\gcc-mingw-4.3.0\debug\has_nothrow_assign_test.run

====== BEGIN OUTPUT ====== has_nothrow_assign_test.cpp:204: The expression: "::boost::has_nothrow_assign::value" had an invalid value (found 1, expected 0)"

I assume the current version has the same failure?

...

In type_traits with gcc versions below 4 there are lots of failures of the order of:

"gcc.compile.c++ ..\..\..\bin.v2\libs\type_traits\test\is_virtual_base_of_test.test\gcc-mingw-3.4.5\debug\is_virtual_base_of_test.o

../../../boost/type_traits/is_virtual_base_of.hpp: In instantiation of `boost::detail::is_virtual_base_of_impl

...
::boost_type_traits_internal_struct_X': ../../../boost/type_traits/is_virtual_base_of.hpp:65: instantiated from `boost::detail::is_virtual_base_of_impl >' ../../../boost/type_traits/is_virtual_base_of.hpp:73: instantiated from `boost::detail::is_virtual_base_of_impl2' ../../../boost/type_traits/is_virtual_base_of.hpp:82: instantiated from `boost::is_virtual_base_of'"

etc. Of course I can't imagine anyone still using those versions.

You missed out the error message! But again, does the current version have the same failure?

...

The function_types library depends on some type_traits implementation which is not in Version2, and tti uses function_types etc so tti fails.

...
* If anyone would like to help converting further traits all help would be much appreciated.... Oh and the compiler requirements in the docs are all wildly out of date too...

Where are the compiler requirements for type_traits mentioned ?

There are out of date mentions in the docs - it all needs a *lot* of revising. If you mean "what are the requirements for this rewrite", then I don't know yet... *should* be the same as the current code modulo the usual SNAFU's John.

Edward Diener

6:16 p.m.

On 1/15/2015 3:58 AM, John Maddock wrote:

...

...
Testing gcc on Windows 7 with mingw and mingw64:

Running type_traits tests passed with msvc-8.0, msvc-9.0, gcc-4.4.0, gcc-4.5.0, gcc-4.5.2, and gcc-4.9.2.

Failed with gcc-4.3.0 with:

"testing.capture-output ..\..\..\bin.v2\libs\type_traits\test\has_nothrow_assign_test.test\gcc-mingw-4.3.0\debug\has_nothrow_assign_test.run

====== BEGIN OUTPUT ====== has_nothrow_assign_test.cpp:204: The expression: "::boost::has_nothrow_assign::value" had an invalid value (found 1, expected 0)"

I assume the current version has the same failure?

You are correct. There is no difference between the current version and Version2.

...

...
In type_traits with gcc versions below 4 there are lots of failures of the order of:

"gcc.compile.c++ ..\..\..\bin.v2\libs\type_traits\test\is_virtual_base_of_test.test\gcc-mingw-3.4.5\debug\is_virtual_base_of_test.o

../../../boost/type_traits/is_virtual_base_of.hpp: In instantiation of `boost::detail::is_virtual_base_of_impl

...
::boost_type_traits_internal_struct_X': ../../../boost/type_traits/is_virtual_base_of.hpp:65: instantiated from `boost::detail::is_virtual_base_of_impl >' ../../../boost/type_traits/is_virtual_base_of.hpp:73: instantiated from `boost::detail::is_virtual_base_of_impl2' ../../../boost/type_traits/is_virtual_base_of.hpp:82: instantiated from `boost::is_virtual_base_of'"

etc. Of course I can't imagine anyone still using those versions.

You missed out the error message!

Just warnings and no error message.

...

But again, does the current version have the same failure?

The current version has similar failures. I don't see an error message with the failures in either the current version or the version2 version for gcc-3.4.5, just lots of warnings with "...failed gcc.compile.c++ ... " messages. Are the type_traits tests set to fail if any warnings occur ? It hardly seems necessary for type_traits to support gcc-3. I shouldn't have bothered testing it anyway.

...

...
The function_types library depends on some type_traits implementation which is not in Version2, and tti uses function_types etc so tti fails.

...
* If anyone would like to help converting further traits all help would be much appreciated.... Oh and the compiler requirements in the docs are all wildly out of date too...

Where are the compiler requirements for type_traits mentioned ?

There are out of date mentions in the docs - it all needs a *lot* of revising.

I could not find any mention of compiler requirements in the type_traits docs, much less out of date mentions, other than in the topic "Support for Compiler Intrinsics".

...

If you mean "what are the requirements for this rewrite", then I don't know yet... *should* be the same as the current code modulo the usual SNAFU's

John Maddock

6:39 p.m.

...

...
You missed out the error message!

Just warnings and no error message.

...
But again, does the current version have the same failure?

The current version has similar failures. I don't see an error message with the failures in either the current version or the version2 version for gcc-3.4.5, just lots of warnings with "...failed gcc.compile.c++ ... " messages. Are the type_traits tests set to fail if any warnings occur ?

Ah yes, it has warnings-as-error turned on.

...

...
There are out of date mentions in the docs - it all needs a *lot* of revising.

I could not find any mention of compiler requirements in the type_traits docs, much less out of date mentions, other than in the topic "Support for Compiler Intrinsics".

Each trait should list what compiler features it requires for correct functionality. John.

Edward Diener

7:50 p.m.

On 1/15/2015 1:39 PM, John Maddock wrote:

...

...
...
You missed out the error message!

Just warnings and no error message.

...
But again, does the current version have the same failure?

The current version has similar failures. I don't see an error message with the failures in either the current version or the version2 version for gcc-3.4.5, just lots of warnings with "...failed gcc.compile.c++ ... " messages. Are the type_traits tests set to fail if any warnings occur ?

Ah yes, it has warnings-as-error turned on.

In that case, besides the warnings, gcc-3.4.5 compiles without error. For Version2 the same results occur except that gcc-3.4.5 complains: mpl_interop_test1.cpp:30:2: no newline at end of file for each of the mpl_interop_test source files.

...

...
...
There are out of date mentions in the docs - it all needs a *lot* of revising.

I could not find any mention of compiler requirements in the type_traits docs, much less out of date mentions, other than in the topic "Support for Compiler Intrinsics".

Each trait should list what compiler features it requires for correct functionality.

OK. I was looking more for "type_traits should work with xxx, yyy, zzz compilers" etc but I understand this would be difficult to keep up to date.

Edward Diener

8:12 p.m.

On 1/15/2015 1:39 PM, John Maddock wrote:

...

...
...
You missed out the error message!

Just warnings and no error message.

...
But again, does the current version have the same failure?

The current version has similar failures. I don't see an error message with the failures in either the current version or the version2 version for gcc-3.4.5, just lots of warnings with "...failed gcc.compile.c++ ... " messages. Are the type_traits tests set to fail if any warnings occur ?

Ah yes, it has warnings-as-error turned on.

...
...
There are out of date mentions in the docs - it all needs a *lot* of revising.

I could not find any mention of compiler requirements in the type_traits docs, much less out of date mentions, other than in the topic "Support for Compiler Intrinsics".

Each trait should list what compiler features it requires for correct functionality.

Trying gcc-3.4.2 it has the same result as gcc-3.4.5 using master and Version2 branches. Trying gcc-3.3.3 it complains: "cc1plus.exe: error: unrecognized option `-Wextra'" for every file whether using master or version2. Whether type_traits should compile with these old gcc-3 versions is up to you.

John Maddock

16 Jan 16 Jan

9:18 a.m.

...

Trying gcc-3.4.2 it has the same result as gcc-3.4.5 using master and Version2 branches.

Trying gcc-3.3.3 it complains:

"cc1plus.exe: error: unrecognized option `-Wextra'"

for every file whether using master or version2.

OK that's a Jamfile issue rather than a type_traits issue as such - as we haven't tested with anything that old in a long while I'm not overly bothered at this stage ;) Thanks, John.

Beman Dawes

15 Jan 15 Jan

2:13 a.m.

On Wed, Jan 14, 2015 at 2:58 PM, John Maddock wrote:

...

I'm going to throw this out there to see what folks think:

In a branch called "Version2", see https://github.com/boostorg/type_traits/tree/Version2 is a partially rewritten type_traits lib: it doesn't yet contain all the traits of the old one, but there's a pretty good selection to test.

The main pros for this version are:

1) Zero dependencies other than Boost.Config. 2) No horrible macro-obfuscation - all "real code", making it easier to read and maintain. 3) Substantially reduced template instantiations where practical - often half as many as before. 4) Substantially reduced #includes - the aim is that all files should include only what they need and no more - with what's needed determined by the actual implementation variant when appropriate. There were also a lot of spurious #includes left over from old workarounds no longer used.

Every one of those is a big win.

...

The main cons:

1) I don't know what I've broken, but surely something, you can help by testing but I would guess we would never be 100% sure it's all right until it's out in the field. This is the main reason for avoiding this until now. 2) MPL interoperability should still work, but files which expect type_traits headers to have included mpl ones will be broken unless they explicitly include what they need. 3) Current MPL inter-operability is fragile see https://github.com/boostorg/type_traits/blob/Version2/include/boost/type_tra... 4) Function dispatching to an mpl::bool_ involves an (inline) function call where as the old inheritance scheme did not. This should mostly disappear once optimizations are turned on, but is still a potential issue for some folks I suspect. 5) I haven't got to the "difficult" headers like common_type.hpp yet. 6) Whatever I've forgotten ;)

Those don't seem like showstoppers, either individually or in aggregate. Careful testing, and social engineering like announcing the 1.58 beta more widely than usually will help, too.

...

Of course (2), (3) and (4) could be solved by making mpl's bool.hpp some kind of "core" header.

So... what I need from you kind folks is:

* Is this the right way forward?

+1. The potential risks seem minor compared to the known gains. --Beman

Frédéric Bron

10:22 a.m.

Hi John,

...

2) No horrible macro-obfuscation - all "real code", making it easier to read and maintain.

Do you see any possibility/need to replace the use of macros in the operator traits (has_plus, has_less...)? They do basically all the same thing and I do not see how to change that. I do not think duplicating the code would be a good option. Cheers, Frédéric

pfultz2

16 Jan 16 Jan

6:08 p.m.

...

3) Current MPL inter-operability is fragile see https://github.com/boostorg/type_traits/blob/Version2/include/boost/type_tra...

Well not only the interoperability, but the functionality. As `integral_constant` is currently defined, this won't compile on C++11 or C++14: static_assert(boost::is_integral<int>(), "Not integral"); Which it does compile currently with boost type traits. I can try and send a pull request for this if you like. Paul -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

John Maddock

18 Jan 18 Jan

12:09 p.m.

...

I'm going to throw this out there to see what folks think:

In a branch called "Version2", see https://github.com/boostorg/type_traits/tree/Version2 is a partially rewritten type_traits lib: it doesn't yet contain all the traits of the old one, but there's a pretty good selection to test.

This rewrite is now complete except for common_type which is the same as the old version, and pulls in all kinds of dependencies. So... 1) Can folks please give this a test? Both the type_traits tests and whatever real world tests may be out there (or other Boost libraries of course). 2) What do we do about common_type? Thanks! John.

Peter Dimov

2:38 p.m.

John Maddock wrote:

...

2) What do we do about common_type?

Kill it with fire! Or, alternatively, move it to TypeOf, although this would not make a good Michael Bay movie.

Vicente J. Botet Escriba

3:42 p.m.

Le 18/01/15 15:38, Peter Dimov a écrit :

...

John Maddock wrote:

...
2) What do we do about common_type?

Kill it with fire!

Or, alternatively, move it to TypeOf, although this would not make a good Michael Bay movie.

If no one is using it I can move it to Boost.Chrono. Best, Vicente

Neil Groves

1 Feb 1 Feb

11:32 a.m.

On 18 Jan 2015 14:39, "Peter Dimov" wrote:

...

John Maddock wrote:

...
2) What do we do about common_type?

Kill it with fire!

I hope you are implying this is done while respecting our deprecation process. It looks like all of the other work can make it into the next release, which is awesome. We can't do this change for the next release without violating our deprecation procedure. I am currently helping companies decide to use Boost and one of the frequent fears is that upgrading Boost consumes a lot of time. I typically point to the deprecation procedure and our efforts to reduce this problem. Sure we still occasionally get it wrong (at least I do) but deliberately compromising external quality factors for internal quality gains is not the way to win customers. I can see the large gains from this considerable work. They are exciting, but let's not violate our procedure on deprecation. Can we get conditionally low dependency by using C++11 where available? I can see that there is still the problem of the publicly documented assertion mechanism. Perhaps the method of compile-time assertion can change without being considered a breaking interface change?

...

Or, alternatively, move it to TypeOf, although this would not make a good Michael Bay movie.

We can't move it for the next release without breaking our promises to our users. It is all super work and I'm sure that with a little extra thought we can get most or all of these gains in the next release. If we do it without breaking backward compatibility I think our users will be as excited about these changes as we are. Neil Groves _______________________________________________

...

Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

John Maddock

1:36 p.m.

...

...
...
2) What do we do about common_type?

Kill it with fire!

I hope you are implying this is done while respecting our deprecation process. It looks like all of the other work can make it into the next release, which is awesome.

Please see my other message - a dependency free version of common_type is possible for C++11 compilers and/or anything that supports GCC's typeof. That means VC10 or later, and pretty much any GCC (or emulation thereof) that we support. As long as the package manager supports some form of "optional dependency", users who need older or more obscure compilers can manually fulfil the dependency by downloading Boost.Typeof (and dependencies) as well. But not for the next release - I was thinking of merging to develop *after* the next release so as to not destabilize that process - then shake down in develop for at least one release cycle before merging to release. In other words in time for the Autumn boost release. John.

...

We can't do this change for the next release without violating our deprecation procedure. I am currently helping companies decide to use Boost and one of the frequent fears is that upgrading Boost consumes a lot of time. I typically point to the deprecation procedure and our efforts to reduce this problem. Sure we still occasionally get it wrong (at least I do) but deliberately compromising external quality factors for internal quality gains is not the way to win customers.

I can see the large gains from this considerable work. They are exciting, but let's not violate our procedure on deprecation.

Can we get conditionally low dependency by using C++11 where available? I can see that there is still the problem of the publicly documented assertion mechanism. Perhaps the method of compile-time assertion can change without being considered a breaking interface change?

...
Or, alternatively, move it to TypeOf, although this would not make a good Michael Bay movie.

We can't move it for the next release without breaking our promises to our users.

It is all super work and I'm sure that with a little extra thought we can get most or all of these gains in the next release. If we do it without breaking backward compatibility I think our users will be as excited about these changes as we are.

Neil Groves _______________________________________________

...
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Stephen Kelly

1:42 p.m.

John Maddock wrote:

...

As long as the package manager supports some form of "optional dependency", users who need older or more obscure compilers can manually fulfil the dependency by downloading Boost.Typeof (and dependencies) as well.

Have you considered splitting common_type into a separate library? Boost.MPL and Boost.Typeof don't seem to depend on it. Thanks, Steve.

Peter Dimov

3:56 p.m.

Neil Groves wrote:

...

...
Or, alternatively, move [common_type] to TypeOf, although this would not make a good Michael Bay movie.

We can't move it for the next release without breaking our promises to our users.

Why not?

Edward Diener

18 Jan 18 Jan

11:29 p.m.

On 1/18/2015 7:09 AM, John Maddock wrote:

...

...
I'm going to throw this out there to see what folks think:

In a branch called "Version2", see https://github.com/boostorg/type_traits/tree/Version2 is a partially rewritten type_traits lib: it doesn't yet contain all the traits of the old one, but there's a pretty good selection to test.

This rewrite is now complete except for common_type which is the same as the old version, and pulls in all kinds of dependencies.

So...

1) Can folks please give this a test? Both the type_traits tests and whatever real world tests may be out there (or other Boost libraries of course).

First tests for just type_traits using mingw/gcc on Windows shows no problems for gcc-4.8.1 on up. But this single target failure for gcc-4.7.2 on down:

...

gcc.compile.c++ ..\..\..\bin.v2\libs\type_traits\test\aligned_storage_a2_test.test\gcc-mingw-4.6.2\debug\aligned_storage_a2_test.o aligned_storage_a2_test.cpp: In function 'void do_check(const T&) [with T = boost::integral_constant]': aligned_storage_a2_test.cpp:78:78: instantiated from here aligned_storage_a2_test.cpp:38:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<1u, 1u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:38:18: error: missing braces around initializer for 'char [1]' [-Werror=missing-braces] aligned_storage_a2_test.cpp:78:78: instantiated from here aligned_storage_a2_test.cpp:49:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<2u, 1u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:49:18: error: missing braces around initializer for 'char [2]' [-Werror=missing-braces] aligned_storage_a2_test.cpp:78:78: instantiated from here aligned_storage_a2_test.cpp:63:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<1u, -1u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:63:18: error: missing braces around initializer for 'char [1]' [-Werror=missing-braces] aligned_storage_a2_test.cpp: In function 'void do_check(const T&) [with T = boost::integral_constant]': aligned_storage_a2_test.cpp:79:79: instantiated from here aligned_storage_a2_test.cpp:38:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<2u, 2u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:38:18: error: missing braces around initializer for 'char [2]' [-Werror=missing-braces] aligned_storage_a2_test.cpp:79:79: instantiated from here aligned_storage_a2_test.cpp:49:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<4u, 2u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:49:18: error: missing braces around initializer for 'char [4]' [-Werror=missing-braces] aligned_storage_a2_test.cpp:79:79: instantiated from here aligned_storage_a2_test.cpp:63:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<2u, -1u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:63:18: error: missing braces around initializer for 'char [2]' [-Werror=missing-braces] aligned_storage_a2_test.cpp: In function 'void do_check(const T&) [with T = boost::integral_constant]': aligned_storage_a2_test.cpp:80:77: instantiated from here aligned_storage_a2_test.cpp:38:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<4u, 4u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:38:18: error: missing braces around initializer for 'char [4]' [-Werror=missing-braces] aligned_storage_a2_test.cpp:80:77: instantiated from here aligned_storage_a2_test.cpp:49:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<8u, 4u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:49:18: error: missing braces around initializer for 'char [8]' [-Werror=missing-braces] aligned_storage_a2_test.cpp:80:77: instantiated from here aligned_storage_a2_test.cpp:63:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<4u, -1u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:63:18: error: missing braces around initializer for 'char [4]' [-Werror=missing-braces] aligned_storage_a2_test.cpp: In function 'void do_check(const T&) [with T = boost::integral_constant]': aligned_storage_a2_test.cpp:83:80: instantiated from here aligned_storage_a2_test.cpp:38:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<8u, 8u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:38:18: error: missing braces around initializer for 'char [8]' [-Werror=missing-braces] aligned_storage_a2_test.cpp:83:80: instantiated from here aligned_storage_a2_test.cpp:49:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<16u, 8u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:49:18: error: missing braces around initializer for 'char [16]' [-Werror=missing-braces] aligned_storage_a2_test.cpp:83:80: instantiated from here aligned_storage_a2_test.cpp:63:18: error: missing braces around initializer for 'boost::detail::aligned_storage::aligned_storage_imp<8u, -1u>::data_t' [-Werror=missing-braces] aligned_storage_a2_test.cpp:63:18: error: missing braces around initializer for 'char [8]' [-Werror=missing-braces] cc1plus.exe: all warnings being treated as errors

"g++" -ftemplate-depth-128 -O0 -fno-inline -Wall -pedantic -Werror -g -Wextra -Wno-uninitialized -DBOOST_ALL_NO_LIB=1 -I"..\..\.." -I"libs\tt2\light\include" -c -o "..\..\..\bin.v2\libs\type_traits\test\aligned_storage_a2_test.test\gcc-mingw-4.6.2\debug\aligned_storage_a2_test.o" "aligned_storage_a2_test.cpp"

...failed gcc.compile.c++ ..\..\..\bin.v2\libs\type_traits\test\aligned_storage_a2_test.test\gcc-mingw-4.6.2\debug\aligned_storage_a2_test.o...

...

2) What do we do about common_type?

Thanks! John.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Edward Diener

19 Jan 19 Jan

7:16 a.m.

On 1/18/2015 7:09 AM, John Maddock wrote:

...

...
I'm going to throw this out there to see what folks think:

In a branch called "Version2", see https://github.com/boostorg/type_traits/tree/Version2 is a partially rewritten type_traits lib: it doesn't yet contain all the traits of the old one, but there's a pretty good selection to test.

This rewrite is now complete except for common_type which is the same as the old version, and pulls in all kinds of dependencies.

So...

1) Can folks please give this a test? Both the type_traits tests and whatever real world tests may be out there (or other Boost libraries of course).

I am seeing that the type_traits macro BOOST_TT_AUX_TEMPLATE_ARITY_SPEC(i, name), which is in type_traits/detail/template_arity_spec.hpp, is used extensively in function_types and also in a few other libraries. In Version2 this header file no longer exists. So function_types fails and libraries which depend on function_types, such as tti, fails when type_traits is switched to Version2. I am not sure what this macro does but maybe we can move it to MPL, call it BOOST_MPL_AUX_TEMPLATE_ARITY_SPEC(i, name), and have function_types and other libraries which use it then change to the new header file in MPL and invoke the macro under it no name. The macro itself looks as if all its dependencies are on MPL.

John Maddock

9:21 a.m.

...

...
1) Can folks please give this a test? Both the type_traits tests and whatever real world tests may be out there (or other Boost libraries of course).

I am seeing that the type_traits macro BOOST_TT_AUX_TEMPLATE_ARITY_SPEC(i, name), which is in type_traits/detail/template_arity_spec.hpp, is used extensively in function_types and also in a few other libraries. In Version2 this header file no longer exists. So function_types fails and libraries which depend on function_types, such as tti, fails when type_traits is switched to Version2.

I am not sure what this macro does but maybe we can move it to MPL, call it BOOST_MPL_AUX_TEMPLATE_ARITY_SPEC(i, name), and have function_types and other libraries which use it then change to the new header file in MPL and invoke the macro under it no name. The macro itself looks as if all its dependencies are on MPL.

I noticed that one and other lib's are using those internal details :-( I think I can add back versions of those headers that don't actually use mpl, let me see.... John.

Peter Dimov

2:55 p.m.

Edward Diener wrote:

...

I am seeing that the type_traits macro BOOST_TT_AUX_TEMPLATE_ARITY_SPEC(i, name), which is in type_traits/detail/template_arity_spec.hpp, is used extensively in function_types and also in a few other libraries.

Wasn't this macro a VC6 workaround? As a first step we could just add an empty macro and see if anything today actually needs it.

John Maddock

4:26 p.m.

...

...
I am seeing that the type_traits macro BOOST_TT_AUX_TEMPLATE_ARITY_SPEC(i, name), which is in type_traits/detail/template_arity_spec.hpp, is used extensively in function_types and also in a few other libraries.

Wasn't this macro a VC6 workaround?

As a first step we could just add an empty macro and see if anything today actually needs it.

That's what I was planning, these macros are all basically superfluous now. John.

Edward Diener

5:49 p.m.

On 1/19/2015 11:26 AM, John Maddock wrote:

...

...
...
I am seeing that the type_traits macro BOOST_TT_AUX_TEMPLATE_ARITY_SPEC(i, name), which is in type_traits/detail/template_arity_spec.hpp, is used extensively in function_types and also in a few other libraries.

Wasn't this macro a VC6 workaround?

As a first step we could just add an empty macro and see if anything today actually needs it.

That's what I was planning, these macros are all basically superfluous now.

I am seeing in MPL empty_base.hpp: #include but this file does not exist in type_traits Version2.

John Maddock

20 Jan 20 Jan

1:14 p.m.

...

...
...
Wasn't this macro a VC6 workaround?

As a first step we could just add an empty macro and see if anything today actually needs it.

That's what I was planning, these macros are all basically superfluous now.

I am seeing in MPL empty_base.hpp:

#include

but this file does not exist in type_traits Version2.

I've added back "do nothing much" versions of those missing files. A lot more should build now. However, there are a couple of show-stoppers I don't know how to solve: 1) boost::integral_constant is no longer a model of mpl's integral_constant: http://www.boost.org/doc/libs/1_57_0/libs/mpl/doc/refmanual/integral-constan... As a result all of function_types lib is broken. I guess one could argue that if you want to do arithmetic on integral_constant's then one should use MPL all the way through. But it's a big breaking change. 2) There are few places (well, myself in multiprecision lib) where folks do dispatch on multiple conditions: do_something(mpl::true_ const&, mpl::true_ const&){} do_something(mpl::true_ const&, mpl::false_ const&){} template <bool B> do_something(mpl::true_ const&, mpl::bool_<B> const& {/*catch all case*/} Then: do_something(trait1<T>(), trait2<T>()); The problem here is that the final catch all case can not be found unless trait2 actually inherits from mpl::bool_ rather than being merely convertible to it. John.

Ion Gaztañaga

5:22 p.m.

El 20/01/2015 a las 14:14, John Maddock escribió:

...

Then:

do_something(trait1<T>(), trait2<T>());

The problem here is that the final catch all case can not be found unless trait2 actually inherits from mpl::bool_ rather than being merely convertible to it.

MPL bool should be constructible from any type that has a constant named "value", as Peter proposed. I think that's a good solution to maintain backwards compatibility, to invert the dependency between type traits and mpl. Best, Ion

Peter Dimov

5:31 p.m.

Ion Gaztañaga wrote:

...

MPL bool should be constructible from any type that has a constant named "value", as Peter proposed.

This still won't work for this case. 'B' can't be deduced.

John Maddock

5:43 p.m.

...

1) boost::integral_constant is no longer a model of mpl's integral_constant: http://www.boost.org/doc/libs/1_57_0/libs/mpl/doc/refmanual/integral-constan...

As a result all of function_types lib is broken. I guess one could argue that if you want to do arithmetic on integral_constant's then one should use MPL all the way through. But it's a big breaking change.

Never mind, this one is fixed now, function_types tests should all now work. John.

Peter Dimov

6:37 p.m.

John Maddock wrote:

...

Never mind, this one is fixed now, function_types tests should all now work.

Unrelated question, but is this correct? template struct aligned_storage_imp { union data_t { char buf[1]; ::boost::detail::max_align align_; } data_; void* address() const { return const_cast(this); } }; size_t(-1) is a special value for the alignment, not for the size parameter?

John Maddock

21 Jan 21 Jan

6:06 p.m.

...

...
Never mind, this one is fixed now, function_types tests should all now work.

Unrelated question, but is this correct?

Aaaah! No, not at all, good catch, fixed in Git. John.

Edward Diener

20 Jan 20 Jan

11:27 p.m.

On 1/20/2015 12:43 PM, John Maddock wrote:

...

...
1) boost::integral_constant is no longer a model of mpl's integral_constant: http://www.boost.org/doc/libs/1_57_0/libs/mpl/doc/refmanual/integral-constan...

As a result all of function_types lib is broken. I guess one could argue that if you want to do arithmetic on integral_constant's then one should use MPL all the way through. But it's a big breaking change.

Never mind, this one is fixed now, function_types tests should all now work.

The function_types test look good with gcc, clang, and msvc on Windows. I am seeing this with type_traits Version2 compiling gcc or clang on Windows : // test_has_data_compile.cpp ( reduced from TTI test to show problem ) #include #include int main() { return 0; } "C:/Programming/VersionControl/bninja_installed_clang/bin/clang++.exe" -c -x c++ -O0 -g -fno-inline -Wall -g -DBOOST_ALL_NO_LIB=1 -I"..\..\.." -o "..\..\..\bin.v2\libs\tti\test\test_has_data_compile.test\clang-linux-3.7.0\debug\test_has_data_compile.obj" "test_has_data_compile.cpp" In file included from test_has_data_compile.cpp:2: ..\..\..\boost/type_traits/remove_const.hpp:25:22: error: token is not a valid binary operator in a preprocessor subexpression #if !BOOST_WORKAROUND(__BORLANDC__, < 0x600) && !defined(__IBMCPP__) && !BOOST_WORKAROUND(__DMC__, BOOST_TESTED_AT(0x840)) ~~~~~~~~~~~~~~~~~^ 1 error generated. The error does not occur with VC++ on Windows.

Peter Dimov

11:34 p.m.

Edward Diener wrote:

...

..\..\..\boost/type_traits/remove_const.hpp:25:22: error: token is not a valid binary operator in a preprocessor subexpression #if !BOOST_WORKAROUND(__BORLANDC__, < 0x600) && !defined(__IBMCPP__) && !BOOST_WORKAROUND(__DMC__, BOOST_TESTED_AT(0x840)) ~~~~~~~~~~~~~~~~~^ 1 error generated.

An #include is missing.

Edward Diener

21 Jan 21 Jan

12:15 a.m.

On 1/20/2015 6:34 PM, Peter Dimov wrote:

...

Edward Diener wrote:

...
..\..\..\boost/type_traits/remove_const.hpp:25:22: error: token is not a valid binary operator in a preprocessor subexpression #if !BOOST_WORKAROUND(__BORLANDC__, < 0x600) && !defined(__IBMCPP__) && !BOOST_WORKAROUND(__DMC__, BOOST_TESTED_AT(0x840)) ~~~~~~~~~~~~~~~~~^ 1 error generated.

An #include is missing.

Thanks. I updated Version2 with the fix.

John Maddock

6:07 p.m.

...

...
...
..\..\..\boost/type_traits/remove_const.hpp:25:22: error: token is not a valid binary operator in a preprocessor subexpression #if !BOOST_WORKAROUND(__BORLANDC__, < 0x600) && !defined(__IBMCPP__) && !BOOST_WORKAROUND(__DMC__, BOOST_TESTED_AT(0x840)) ~~~~~~~~~~~~~~~~~^ 1 error generated.

An #include is missing.

Thanks. I updated Version2 with the fix.

Got it, thanks! John.

Edward Diener

12:42 a.m.

On 1/20/2015 12:43 PM, John Maddock wrote:

...

...
1) boost::integral_constant is no longer a model of mpl's integral_constant: http://www.boost.org/doc/libs/1_57_0/libs/mpl/doc/refmanual/integral-constan...

As a result all of function_types lib is broken. I guess one could argue that if you want to do arithmetic on integral_constant's then one should use MPL all the way through. But it's a big breaking change.

Never mind, this one is fixed now, function_types tests should all now work.

Lexical_cast tests are broken with: ..\..\../boost/lexical_cast/detail/converter_numeric.hpp: In static member function 'static bool boost::detail::lexical_cast_dynamic_num_ignoring_minus::try_convert(const Source&, Target&)': ..\..\../boost/lexical_cast/detail/converter_numeric.hpp:117:17: error: 'is_float' is not a member of 'boost' boost::is_float<Source>::value, ^ One of the funcion_types tests fails because of this and running the lexical_cast tests fails also.

John Maddock

7:17 p.m.

...

Lexical_cast tests are broken with:

..\..\../boost/lexical_cast/detail/converter_numeric.hpp: In static member function 'static bool boost::detail::lexical_cast_dynamic_num_ignoring_minus::try_convert(const Source&, Target&)': ..\..\../boost/lexical_cast/detail/converter_numeric.hpp:117:17: error: 'is_float' is not a member of 'boost' boost::is_float<Source>::value, ^

One of the funcion_types tests fails because of this and running the lexical_cast tests fails also.

It's using is_float without including the header, I'm finding quite a few of these, I've created a pull request. Also started a list of these pull requests: https://github.com/boostorg/type_traits/blob/Version2/changes%20pending.txt Please update if any other changes to other lib's are required, so far I have: ITERATOR ~~~~~~~~ is_lvalue_iterator.hpp needs to include mpl/bool.hpp https://github.com/boostorg/iterator/pull/11 LAMBDA ~~~~~~ Missing includes: ice.hpp see https://github.com/boostorg/lambda/pull/3 Lots of headers: https://github.com/boostorg/lambda/pull/4 LEXICAL_CAST ~~~~~~~~~~~ missing is_float.hpp in converter_numeric.hpp https://github.com/boostorg/lexical_cast/pull/8 There is also one runtime failure in Boost.Iterator with msvc-12 and the new type_traits (function_input_iterator_test) which I don't understand at all - anyone any ideas? Otherwise I'm just sort of hoping that something more obvious will show up elsewhere and allow a sneaky fix. John.

Stephen Kelly

1 Feb 1 Feb

9:16 a.m.

John Maddock wrote:

...

2) What do we do about common_type?

This is still not addressed, right? Can you say what the current state of the rewrite is, the intention of how it will be used (ie, replace existing vs maintained in parallel), and blockers to reaching that point? Thanks, Steve.

John Maddock

10:45 a.m.

...

...
2) What do we do about common_type?

This is still not addressed, right?

Can you say what the current state of the rewrite is, the intention of how it will be used (ie, replace existing vs maintained in parallel), and blockers to reaching that point?

I've been suffering from sudden laptop failure, which is why I've been silent on this - I'll post more when I'm up and running again - but the current version of the branch should have a common_factor which only has dependencies outside of Boost.Config for broken/obsolete compilers (please note: I haven't fully tested/checked this, and won't until my new laptop arrives sometime next week). In such a case I think it would be acceptable to manually mark up that header with some "don't track dependencies" directive and require users to manually add Boost.Typeof to their list of required modules if they're using an old compiler. Note that we need some kind of markup facility to deal with optional dependencies anyway - to avoid the "everything depends on Serialization" issue. So maybe something like: /* BPM nodepend BPM start_message Note: If you're using boost::common_type with a compiler that supports neither the decltype nor typeof keywords, then you will need to manually add the "typeof" library to your download list. BPM end_message */ Other than that, I've nearly finished testing the new version and IMO it looks good enough to simply replace the old version - but again more later. John.

Stephen Kelly

10:50 a.m.

John Maddock wrote:

...

<snip>

Thanks for the update.

...

Note that we need some kind of markup facility to deal with optional dependencies anyway - to avoid the "everything depends on Serialization" issue.

Circular dependencies won't work well with clang/c++17 'modules'. Strongly connected components form what I called 'incidental modules' in previous emails. Find a way to convince Robert to allow splitting serialization. Thanks, Steve.

John Maddock

1:39 p.m.

...

...
Note that we need some kind of markup facility to deal with optional dependencies anyway - to avoid the "everything depends on Serialization" issue.

Circular dependencies won't work well with clang/c++17 'modules'. Strongly connected components form what I called 'incidental modules' in previous emails.

Find a way to convince Robert to allow splitting serialization.

That doesn't solve the issue I was referring to: namely library X depends on Y *only* if you're using Y anyway. Serialization being a prime example: many libraries "apparently" depend on it, but the dependency is only a true one if you're using serialization anyway. Such interoperability should be positively encouraged, but I fear package management may actively discourage it unless we make the packaging smarter. John.

Stephen Kelly

2:08 p.m.

John Maddock wrote:

...

...
...
Note that we need some kind of markup facility to deal with optional dependencies anyway - to avoid the "everything depends on Serialization" issue.

Circular dependencies won't work well with clang/c++17 'modules'. Strongly connected components form what I called 'incidental modules' in previous emails.

Find a way to convince Robert to allow splitting serialization.

That doesn't solve the issue I was referring to: namely library X depends on Y *only* if you're using Y anyway. Serialization being a prime example: many libraries "apparently" depend on it, but the dependency is only a true one if you're using serialization anyway.

Yes. Reducing the dependencies of the part of serialization which are required in such cases would solve the problem in those cases. Depending on that part of serialization should be 'not a problem'. But, yes. All of this is 'an unsolved problem'. I think the serialization case is solvable though, distinct from the general 'library X library Y' problem. Thanks, Steve.

Peter Dimov

3:55 p.m.

John Maddock wrote:

...

...
Find a way to convince Robert to allow splitting serialization.

That doesn't solve the issue I was referring to: namely library X depends on Y *only* if you're using Y anyway. Serialization being a prime example: many libraries "apparently" depend on it, but the dependency is only a true one if you're using serialization anyway.

It's a true one. These libraries include serialization headers. If serialization is not installed, these libraries break. This is not like the other false dependencies in which an optional header in X includes something in Y. (You could turn that into a 'false dependency' by moving the serialization include into an optional header, in principle.) The reason Stephen talks about splitting serialization is that adding serialization support needs only a few includes, independent of the rest of the library. If these are separated into 'serialization_core', there'll be no problem with including them. The same problem exists with range(27). Most includes only use the core part, which is a few independent headers. Also a true dependency. The only 'false' dependency that comes to mind right now is function -> typeof.

Peter Dimov

4:26 p.m.

...

It's a true one. These libraries include serialization headers. If serialization is not installed, these libraries break. ... The reason Stephen talks about splitting serialization is that adding serialization support needs only a few includes, independent of the rest of the library. If these are separated into 'serialization_core', there'll be no problem with including them.

See for example this: http://www.pdimov.com/tmp/report-develop-c3bb6eb/date_time.html#serializatio... Also, for an even more dramatic case, this: http://www.pdimov.com/tmp/report-develop-c3bb6eb/uuid.html#serialization (You could call that a false dependency, but note what it only needs to include, and for what reason.)

Peter Dimov

4:32 p.m.

...

The reason Stephen talks about splitting serialization is that adding serialization support needs only a few includes, independent of the rest of the library.

And in fact if you look through http://www.pdimov.com/tmp/report-develop-c3bb6eb/serialization.html#reverse-... you'll see that many, if not most, of the serialization dependencies are like that.

Stephen Kelly

4:48 p.m.

Peter Dimov wrote:

...

...
The reason Stephen talks about splitting serialization is that adding serialization support needs only a few includes, independent of the rest of the library.

And in fact if you look through

http://www.pdimov.com/tmp/report-develop-c3bb6eb/serialization.html#reverse-...

you'll see that many, if not most, of the serialization dependencies are like that.

Yes, but Robert will not discuss splitting it. There's no point even trying to come up with a solution until that changes, but it can be pointed to as a problem for anyone willing to listen :). That's what I've been trying to point out for months. Thanks, Steve.

Robert Ramey

2 Feb 2 Feb

5:52 p.m.

Stephen Kelly-2 wrote

...

Peter Dimov wrote:

...
...
The reason Stephen talks about splitting serialization is that adding serialization support needs only a few includes, independent of the rest of the library.

And in fact if you look through

http://www.pdimov.com/tmp/report-develop-c3bb6eb/serialization.html#reverse-...

you'll see that many, if not most, of the serialization dependencies are like that.

Yes, but Robert will not discuss splitting it.

There's no point even trying to come up with a solution until that changes, but it can be pointed to as a problem for anyone willing to listen :).

That's what I've been trying to point out for months.

I think that there's a lot more too it than "splitting serialization". a) It's not at all clear what "splitting serialization" means. Making a two new DLLS for xml_archives in the same module? That wouldn't change the dependency graph which is driving this request. Making a boost level module just for the XML archives (which use spirit which is what creates the tendencies which are of concern there? or what? Either of these is a serious undertaking - in large part to support auto-linking and who knows what other surprises. Probably the bjam test suite would also have to be re-organized. And I don't know what all else. A huge amount of work with no gain in functionality. b) One possibility would be re-implement xml_archives not in terms of spirit. This would of course be counter productive. The main problem is the concept of "dependency" as revealed in our charts is misleading. When we say one module is dependent upon another what to we mean? If the tests use Boost.Test - is the module dependent on boost test? well, that would depend on whether or not one plans on running the tests. Is serialization dependent on spirit? that would depend on whether one is going to use xml archives or not. I don't know how our dependency charting tool define dependency. Any discussion of dependency has to start with the users particular application. If my app only uses a portion of the library, only that portion needs to be there (header only library) or be build (other libraries). We're already seeing the difficulty of trying to refactor on the basis of a definition of "dependency" which is not really defined. We're chasing our tails. Step back a) A useful tool would look like: tool < list of user application(s), test suites, etc. > list of headers and/or library *.cpp files required. b) the above could be used to select modules. For example, if a user of date-time doesn't use serialization header from date-time - then his app isn't dependent upon serialization so the he doesn't need to install it. The current module graph would show that he does. c) The above would lead to the conclusion that we need the concept of a "partial library installation" which of course opens up all kinds of issues. However, I can say that what I've done in several cases is to install the *.cpp files directly into the target apps rather than linking to the pre-built library. In other words I've actually used this model to good effect. I'm not prepared right now to advocate any radical change in how we do things - but i just want to make it clear that the current approach to "minimize dependencies" is too simplistic to be helpful and in fact is harmful. This topic needs more exploration. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Stephen Kelly

6:40 p.m.

Robert Ramey wrote:

...

I think that there's a lot more too it than "splitting serialization".

The starting point is establishing that there are circular dependencies and they are a problem. If we agree to that, then we can talk about resolving that. You don't agree that there is a problem.

...

The main problem is the concept of "dependency" as revealed in our charts is misleading. When we say one module is dependent upon another what to we mean?

Let's start with 'The headers include headers of another module.'

...

I don't know how our dependency charting tool define dependency.

The headers include headers of another module. You do know that. It has been explained before. Do you know anything about C++17/clang modules? Have you heard of the concept at least? Thanks, Steve.

Bjørn Roald

6:50 p.m.

On 02. feb. 2015 19:40, Stephen Kelly wrote:

...

Do you know anything about C++17/clang modules? Have you heard of the concept at least?

Sorry for stepping in here. If we have: a.h --> libA_Internals.h/cpp b.h --> libB_Internals.h/cpp ab.h --> a.h, b.h We have 3 modules, right? ab.h does not make libA or linB depend on it or each other. -- Bjørn

Peter Dimov

6:48 p.m.

Robert Ramey wrote:

...

I think that there's a lot more too it than "splitting serialization".

a) It's not at all clear what "splitting serialization" means. Making a two new DLLS for xml_archives in the same module? That wouldn't change the dependency graph which is driving this request. Making a boost level module just for the XML archives (which use spirit which is what creates the tendencies which are of concern there? or what? Either of these is a serious undertaking - in large part to support auto-linking and who knows what other surprises. Probably the bjam test suite would also have to be re-organized. And I don't know what all else. A huge amount of work with no gain in functionality.

None of the above. If you heed my suggestion

...

...
...
And in fact if you look through

http://www.pdimov.com/tmp/report-develop-c3bb6eb/serialization.html#reverse-...

you'll see that many, if not most, of the serialization dependencies are like that.

you'll see that the only library that uses the compiled part of Serialization and its archives is MPI, because it's the only library that actually serializes things. All the other libraries that depend on Serialization do that only to make their types serializable. As such, they use no archives and no DLLs. They use These headers (along with their dependencies), or some subset of them, can be separated into a "serialization core" module. Doing this right might still be a significant undertaking, of course. A cursory look shows that map.hpp includes boost/archive/detail/basic_iarchive.hpp, for instance. On the other hand, it's possible to start with only the part that's straightforward and advance incrementally towards the goal.

Stephen Kelly

7:10 p.m.

Peter Dimov wrote:

...

...
...
...
And in fact if you look through

http://www.pdimov.com/tmp/report-develop-c3bb6eb/serialization.html#reverse-...

you'll see that many, if not most, of the serialization dependencies are like that.

you'll see that the only library that uses the compiled part of Serialization and its archives is MPI, because it's the only library that actually serializes things.

Exactly.

...

All the other libraries that depend on Serialization do that only to make their types serializable. As such, they use no archives and no DLLs. They use

These should also be moved. http://thread.gmane.org/gmane.comp.lib.boost.devel/254577

...

These headers (along with their dependencies), or some subset of them, can be separated into a "serialization core" module.

Yes, if there is acknowledgement that there is a problem, and a desire to fix it (currently we have neither from Robert), we can look into what would be sensible. Thanks, Steve.

Peter Dimov

7:30 p.m.

Stephen Kelly wrote:

...

...

These should also be moved.

http://thread.gmane.org/gmane.comp.lib.boost.devel/254577

array(5) ⇢ assert(1) config(0) core(2) static_assert(1) throw_exception(2) and you want to make it depend on serialization(59)? :-)

Stephen Kelly

7:36 p.m.

Peter Dimov wrote:

...

Stephen Kelly wrote:

...
...

These should also be moved.

http://thread.gmane.org/gmane.comp.lib.boost.devel/254577

array(5) ⇢ assert(1) config(0) core(2) static_assert(1) throw_exception(2)

and you want to make it depend on serialization(59)? :-)

No, on serialization(5) or serialization(1) or so. (Actually I think that whatever is split away to define an interface should go into core.) But none of this is worth discussing until Robert (the maintainer - the one who holds all of the keys :) ) sees an issue and decides to do something about it. Thanks, Steve.

Peter Dimov

7:44 p.m.

Stephen Kelly wrote:

...

No, on serialization(5) or serialization(1) or so.

Even the core parts of Serialization depend at least on MPL and TypeTraits, so (5) or (1) would be hard to achieve. (12) would be better than (59), of course.

Peter Dimov

7:41 p.m.

...

Stephen Kelly wrote:

...
...

These should also be moved.

http://thread.gmane.org/gmane.comp.lib.boost.devel/254577

array(5) ⇢ assert(1) config(0) core(2) static_assert(1) throw_exception(2)

and you want to make it depend on serialization(59)? :-)

Note, by the way, that boost/serialization/array.hpp doesn't just implement serialization for boost::array; it also defines boost::serialization::array and implements serialization for std::array. So it doesn't belong to Array anyway. And boost/serialization/vector.hpp, of course, implements std::vector's serialization, so there's nowhere to move it to. boost/serialization/optional.hpp and boost/serialization/variant.hpp are another story. But only after there's a lightweight way to implement serialization.

Stephen Kelly

7:45 p.m.

Peter Dimov wrote:

...

...
Stephen Kelly wrote:

...
http://thread.gmane.org/gmane.comp.lib.boost.devel/254577

And boost/serialization/vector.hpp, of course, implements std::vector's serialization, so there's nowhere to move it to.

Yes, I snipped the wrong lines, but the link correctly says variant, not vector. Thanks, Steve.

Robert Ramey

7:57 p.m.

Peter Dimov-2 wrote

...

...
Stephen Kelly wrote: Note, by the way, that boost/serialization/array.hpp doesn't just implement serialization for boost::array; it also defines boost::serialization::array and implements serialization for std::array. So it doesn't belong to Array anyway.

this is the result of naming issues. the word array has been used for boost::array, std::array and the built-in C/C++ array. They all ended up in the same header so that one could find them when they were needed. the same applies to serialization of all the std library items. From my standpoint they should be in some separate place. but it's been of small consequence so they are where they are.

...

boost/serialization/optional.hpp and boost/serialization/variant.hpp are another story.

I never wanted these here. I wanted them to in their respective libraries but no one was willing to maintain them. At the time - I assume it's still true, we had a maintainer assigned to a module. We don't really have the concept of a maintainer handling part of a some module. I would be happy to must move option, variant (I think there's another one also) in to the respective library and be done with it. Probably low risk anyway because changes in he serialization library haven't broken any serialization implementation for a long, long time.

...

But only after there's a lightweight way to implement serialization.

I'm not sure what that refers to. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Robert Ramey

7:48 p.m.

Peter Dimov-2 wrote

...

<snip> These headers (along with their dependencies), or some subset of them, can be separated into a "serialization core" module.

OK then the "serialization core" needs tests - right? then wouldn't the serialization core module depend upon the archive part of the serialization library? Wouldn't that suck in the everything else as it does now. if one uses date-time but doesn't include date-time/serialization and everything works - how can one say that date-time is dependent upon the serialization library?

...

Doing this right might still be a significant undertaking, of course. A cursory look shows that map.hpp includes boost/archive/detail/basic_iarchive.hpp, for instance.

If one just included the header things might work - until someone needs to actually invoke the serialization of the type - at which case - either you've got a missing function when you try to link or you've had to make the serialization library dependent in any case.

...

On the other hand, it's possible to start with only the part that's straightforward and advance incrementally towards the goal.

For what it's worth, the serialization library namespace is already divided into "archive" and "serialization" since the very beginning with the exact idea of separating the serialization from the concept of archive. I caught a huge amount of strident criticism for doing this (much more than the current incident). But It looks to me that things are already structure more or less as you are recommending - but that this doesn't show on the dependency graph because the graph tries to implement a concept of module dependency rather than header dependency. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Peter Dimov

8:12 p.m.

Robert Ramey wrote:

...

Peter Dimov-2 wrote

...
<snip> These headers (along with their dependencies), or some subset of them, can be separated into a "serialization core" module.

OK then the "serialization core" needs tests - right? then wouldn't the serialization core module depend upon the archive part of the serialization library? Wouldn't that suck in the everything else as it does now.

No, test/ is not scanned for dependencies.

...

if one uses date-time but doesn't include date-time/serialization and everything works - how can one say that date-time is dependent upon the serialization library?

There is no such thing as date-time/serialization at present. It's not possible to not include it. But yes, in principle, if the serialization support were extracted into its own header, it might have been called an "optional dependency".

...

If one just included the header things might work - until someone needs to actually invoke the serialization of the type - at which case - either you've got a missing function when you try to link or you've had to make the serialization library dependent in any case.

You can't invoke the serialization of the type without having an archive, and if you have an archive, you have to have the rest of Serialization installed, otherwise where did the archive come from?

...

For what it's worth, the serialization library namespace is already divided into "archive" and "serialization" since the very beginning with the exact idea of separating the serialization from the concept of archive. I caught a huge amount of strident criticism for doing this (much more than the current incident). But It looks to me that things are already structure more or less as you are recommending - but that this doesn't show on the dependency graph...

The two hypothetical modules, 'archive' and 'serialization', would still be dependent on each other, because headers in serialization/ include headers in archive/. It's true that you have tried to keep them separate.

...

... because the graph tries to implement a concept of module dependency rather than header dependency.

As I have explained many times now, a per-header dependency graph is useless for practical purposes if modular releases and installations are the goal. Even if we assume header-only libraries, you still install and uninstall on a library level, not on a header level; apart from making things needlessly complicated, the documentation, for instance, can't automagically be split to only contain whatever portion of the headers you have. And for libraries that aren't header-only, it doesn't work at all. When the dependency scanner sees your #include , it has no idea whether a.hpp contains things that require the library to be built or not. Therefore, it has to assume that you depend on libs/X/src/*, hence you require the whole libs/X to be installed. Doing modular installations on a file level of granularity makes no sense, which is why no package manager in existence does it (to the best of my knowledge). They all operate on a package level of granularity, and therefore their dependency graphs are all on a package level.

Peter Dimov

8:30 p.m.

...

There is no such thing as date-time/serialization at present.

Oops, sorry, there is, I was wrong. The serialization support _is_ in separate headers. date_time •from •from •from •from and only needs nvp.hpp and split_free.hpp.

Robert Ramey

11:51 p.m.

The current situation with date-time is the following: user creates application which doesn't use serialization. Hence he doesn't include boost/data-time/serialization.hpp. His program compiles, links and runs. Clearly his program depends on boost/date-tiem but doesn't depend on boost serialization. I don't think this is a disputable fact. The module dependency tool may say that anything which depends upon date-time library is dependent upon serialization - but that is not true. I don't think this is disputable. So in my view the module decency tool should take one or more program sources as arguments and generate a list of modules which need to be included in order to compile/link and run the all the programs. This would tell a user exactly what he needs to know. When I test the date-time library I would include all the test programs as arguments and get one list of modules. Someone else might be only interested in one application and get a shorter list of modules. This would be useful to everyone. There will and can never exist at tool which gives you the exact list of required modules with specifying the applications to be built. So we're going to settle for some approximation. That's OK. it's better than nothing - but don't think that it's anything other than this. The current approximation is that of tracking the header inclusions. This includes more than a given application is likely to use so it's sort of overkill - this shows up as union of all possible modules which some application might need to build. It's not incorrect, just recognize that it's a blunt instrument. I can hear the cry - so what can we do so one doesn't have to download the whole of boost? Here's a suggestion which may satisfy those. Modest Proposal - Bridge module We create a new module - named "bridge". In this module there is a directory: boost/bridge/archive.hpp. This contains only forward declarations: class basic_iarchive; class basic_oarchive; and maybe some other miscellaneous stuff. This includes no headers from boost/archive. in date-time/serialization.hpp files in other libraries replace #include with #include Same is done in other libraries which implement serialization; Everything works the same as before. BUT the dependency tools now stop looking when they get to the bridge header. So our dependency display stops at date-time. I think this is an accurate reflection of what is desired. The building of applications which use date-time is exactly the same as it was before. The requirement to include (or not include) the boost serialization libraries is the same - that is it depends on which part of date-time the application actually uses. But there is a small difference - the application isn't including anything from boost/archive so the serialization library doesn't show up as dependent in the tools. Only the headers from boost/serialization are included. They are all header only and generally not dependent upon a lot of other boost stuff. This might be satisfactory. It does leave us with two different definitions of "module" though. since both boost/serialization and boost/archive are currently in the same "module". In fact most of the tests actually test things in serialization against all the "archives". So teasing this out could be a pain. I'm not even going to think about this now. I'm guessing that other libraries have some similar issues. This was raised previously by John Maddock under the heading "bridge libraries" he posed the question about how to deal with and we didn't really have an answer ... then. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Rene Rivera

3 Feb 3 Feb

12:16 a.m.

On Mon, Feb 2, 2015 at 5:51 PM, Robert Ramey wrote:

...

Modest Proposal - Bridge module

We create a new module - named "bridge". In this module there is a directory: boost/bridge/archive.hpp. This contains only forward declarations:

class basic_iarchive; class basic_oarchive;

and maybe some other miscellaneous stuff. This includes no headers from boost/archive.

in date-time/serialization.hpp files in other libraries replace

#include with #include

Same is done in other libraries which implement serialization;

Everything works the same as before. BUT the dependency tools now stop looking when they get to the bridge header. So our dependency display stops at date-time. I think this is an accurate reflection of what is desired.

The building of applications which use date-time is exactly the same as it was before. The requirement to include (or not include) the boost serialization libraries is the same - that is it depends on which part of date-time the application actually uses. But there is a small difference - the application isn't including anything from boost/archive so the serialization library doesn't show up as dependent in the tools. Only the headers from boost/serialization are included. They are all header only and generally not dependent upon a lot of other boost stuff. This might be satisfactory.

It does leave us with two different definitions of "module" though. since both boost/serialization and boost/archive are currently in the same "module". In fact most of the tests actually test things in serialization against all the "archives". So teasing this out could be a pain. I'm not even going to think about this now.

I'm guessing that other libraries have some similar issues. This was raised previously by John Maddock under the heading "bridge libraries" he posed the question about how to deal with and we didn't really have an answer ... then.

I think the above is what many other people are suggesting already. Unless I've misinterpreted the requests. With that in mind I suggested this < http://tinyurl.com/p2nt3co> as a library structure (suggestion #2). -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

Robert Ramey

12:29 a.m.

Rene Rivera-2 wrote

...

On Mon, Feb 2, 2015 at 5:51 PM, Robert Ramey <

...

ramey@

...

> wrote:

...
Modest Proposal - Bridge module

We create a new module - named "bridge". In this module there is a directory: boost/bridge/archive.hpp. This contains only forward declarations:

class basic_iarchive; class basic_oarchive;

I think the above is what many other people are suggesting already.

LOL - I didn't see any proposals which looked like this to me.

...

Unless I've misinterpreted the requests. With that in mind I suggested this < http://tinyurl.com/p2nt3co> as a library structure (suggestion #2).

So it seems that you would just separate the headers into "families" core and full and the dependency tool would just follow core? I didn't see that mentioned but I'm guessing that's what you're suggestion. I don't see any connection with my suggestion not that it matters. I think that this problem only effects a few libraries. The serialization library is the most obvious case but there may be a couple more. Naturally I prefer my idea above as it would relieve me of modifying dozens of files. Also, by including the "bridge" in the archive code it self, I can detect that the forwarding headers are actually identical to the implementation headers. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Vicente J. Botet Escriba

7:10 a.m.

...

I think the above is what many other people are suggesting already. Unless I've misinterpreted the requests. With that in mind I suggested this < http://tinyurl.com/p2nt3co> as a library structure (suggestion #2). Adding here the second option " Second

Le 03/02/15 01:16, Rene Rivera a écrit : option is to have each library have to explicit sublibs/parts. "core" and "full". Structure would be: * libs/<lib>/core/(build, doc, include, meta, test) * libs/<lib>/full/(build, doc, include, meta, test) Note, both core and full would be in the same git repo to make life easier for library authors and users that visit a library git repo on github. As it's only one place for everything for that one library. As a user, release manager, testing manager, and library author this would be easy to understand and manage than the current "some random set of libraries have core or not and in different places". " IMO, this is a minor variant of what we already have. Consider libs/<lib> to play the role of libs/<lib>/full. Your structure is more explicit, but I don't think we need to go so far as the submodule information is on the meta/libraries.json file. This doesn't mean that a library can not do that. I find very useful to split a library into submodules. * It helps to limit the dependencies as more fine grained, * it makes easier to move a submodule from one github repository to another as the structure of a submodule is the same as the structure of a library (a submodule it self). This could be useful if we want to transfer the maintenance of the submodule. Robert, your bridge proposal could just consist in creating libs/serialization/bridge/(build, doc, include, meta, test) and adding the submodule on the libs/serialization/meta/libraries.json file. Best, Vicente

Robert Ramey

8:05 p.m.

Vicente Botet wrote

...

Robert, your bridge proposal could just consist in creating

libs/serialization/bridge/(build, doc, include, meta, test)

and adding the submodule on the libs/serialization/meta/libraries.json file.

I'm a little gun shy of git submodule idea. I like the way the git setup is working for us but it cost me significant effort to get comfortable and familiar with it. Actually, I still feel I don't understand it all that well. That being said, I agree that your idea seems to faithfully mirror my proposal. I'm just assume that making the submodule would address the dependency report issue - I don't really know how the dependency reports determine dependency. And my proposal for the "bridge" module (maybe better called "forward" module is just that - a proposal. I would like time to think about that some more - a hear what other's have to say as well. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Stephen Kelly

8:30 p.m.

Robert Ramey wrote:

...

And my proposal for the "bridge" module (maybe better called "forward" module is just that - a proposal. I would like time to think about that some more - a hear what other's have to say as well.

* Educate yourself about C++17/Clang modules * If modular releases are a goal, as you claim, decide which tarballs should be created for releases. One repo one tarball or something different? * Determine how C++17/Clang modules relate to modular boost releases. * Determine whether forwarding headers fit into the very real scenario of a world where C++17/Clang modules are pervasive. Thanks, Steve.

Robert Ramey

8:40 p.m.

Stephen Kelly-2 wrote

...

Robert Ramey wrote:

...
And my proposal for the "bridge" module (maybe better called "forward" module is just that - a proposal. I would like time to think about that some more - a hear what other's have to say as well.

* Educate yourself about C++17/Clang modules

I confess that I haven't seen this in any way related to C++17. I would certainly be skeptical of anyone spending any time on something as speculative as a future C++ standard before it's finalized.

...

* If modular releases are a goal, as you claim, decide which tarballs should be created for releases. One repo one tarball or something different?

An interesting question. I haven't really thought of releases in terms of tar balls but I'll spend some time on this.

...

* Determine how C++17/Clang modules relate to modular boost releases.

Since you seem to have some information and/or thoughts about this why not save us all a lot of time and just post them here yourself?

...

* Determine whether forwarding headers fit into the very real scenario of a world where C++17/Clang modules are pervasive.

I don't see C++17/Clang modules (whatever they might be) pervasive in the current real world. As far as I can tell, my proposal for "bridging headers" is the only credible specific one suggested so far. I conceded that I haven't spent a lot of time thinking through the implications (though it seems that I've spent more than others). That's why I've asked for comments. So far I haven't got any real comment on it. I'm still thinking about it. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Stephen Kelly

9:02 p.m.

Robert Ramey wrote:

...

Stephen Kelly-2 wrote

...
Robert Ramey wrote:

...
And my proposal for the "bridge" module (maybe better called "forward" module is just that - a proposal. I would like time to think about that some more - a hear what other's have to say as well.

* Educate yourself about C++17/Clang modules

I confess that I haven't seen this in any way related to C++17. I would certainly be skeptical of anyone spending any time on something as speculative as a future C++ standard before it's finalized.

That's no reason not to educate yourself on it, and it's no reason not to try the clang implementation.

...

...
* Determine how C++17/Clang modules relate to modular boost releases.

Since you seem to have some information and/or thoughts about this why not save us all a lot of time and just post them here yourself?

C++17/Clang modules are selections of headers grouped together as a self- contained unit. The unit can have external dependencies. Cycles are bad. What way would boost headers be grouped together to make use of that?

...

...
* Determine whether forwarding headers fit into the very real scenario of a world where C++17/Clang modules are pervasive.

I don't see C++17/Clang modules (whatever they might be) pervasive in the current real world.

Insisting on not having foresight isn't as good a thing as you think it is.

...

As far as I can tell, my proposal for "bridging headers" is the only credible specific one suggested so far.

Erm, no. You pre-rejected any proposal based on splitting anything sensible out of the serialization repo. As the maintainer, you're the only one who gets to decide. Yay Boost! There's no reason to discuss anything which is pre-rejected. Does your proposal break the cycle which serialization participates in? If it doesn't, then my comment on your proposal is: choose a solution which solves the cycle. If it does, please point that out. Thanks, Steve.

Niall Douglas

10:01 p.m.

On 3 Feb 2015 at 22:02, Stephen Kelly wrote:

...

...
I confess that I haven't seen this in any way related to C++17. I would certainly be skeptical of anyone spending any time on something as speculative as a future C++ standard before it's finalized.

That's no reason not to educate yourself on it, and it's no reason not to try the clang implementation.

For the record, you can simulate clang's modules using any compiler by simply compiling in everything as a single translation unit. A simple shell script can create a file which includes all the source files at once. If it works as a single translation unit, it'll work under clang modules. If it doesn't, well then you've got some ODR violation going on (very common in source files which assume they own their translation unit) and it may or may not work under clang modules depending. I have taken care that API Bind works correctly in single translation unit use cases. I'll admit the gotchas surprised me at the time. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Stephen Kelly

10:11 p.m.

Niall Douglas wrote:

...

On 3 Feb 2015 at 22:02, Stephen Kelly wrote:

...
...
I confess that I haven't seen this in any way related to C++17. I would certainly be skeptical of anyone spending any time on something as speculative as a future C++ standard before it's finalized.

That's no reason not to educate yourself on it, and it's no reason not to try the clang implementation.

For the record, you can simulate clang's modules using any compiler by simply compiling in everything as a single translation unit.

The point is: either Boost is prepared for declaring 'this group of headers depends on that group' (and then taking advantage of the things that follow that declaration), or it is not. Serialization and Robert have been the biggest barriers to that because of the cycle. There are other cycles, but that's been the most problematic one. I was very sad when I saw that cycle in my analysis because I knew Robert would not see the problem and would block any change with all available energy and his fantasies about tracking header file dependencies and imaginary tooling. I tried to raise it as an issue anyway. Maybe in a year something will change. It took a long time for anyone in this community to take any notice of the concept of modularity at all, but now there seem to be a few people who get it. That took many many months though... Why is that? Thanks, Steve.

Robert Ramey

10:34 p.m.

Stephen Kelly-2 wrote

...

The point is: either Boost is prepared for declaring 'this group of headers depends on that group' (and then taking advantage of the things that follow that declaration), or it is not.

Hmmmm - that thinking is probably a large part of the source our current problems. I'm guessing that the idea C++17 modules will have to be refined quite a bit before they can become useful.

...

Serialization and Robert have been the biggest barriers to that because of the cycle. There are other cycles, but that's been the most problematic one.

as I said - I'm not seeing this.

...

I was very sad when I saw that cycle in my analysis because I knew Robert would not see the problem and would block any change with all available energy and his fantasies about tracking header file dependencies and imaginary tooling.

LOL

...

I tried to raise it as an issue anyway. Maybe in a year something will change. It took a long time for anyone in this community to take any notice of the concept of modularity at all, but now there seem to be a few people who get it. That took many many months though... Why is that?

Maybe your argument isn't convincing? Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Niall Douglas

4 Feb 4 Feb

12:19 a.m.

On 3 Feb 2015 at 23:11, Stephen Kelly wrote:

...

The point is: either Boost is prepared for declaring 'this group of headers depends on that group' (and then taking advantage of the things that follow that declaration), or it is not.

I really wish people would stop thinking in terms of headers. I know you don't, but for the others headers == source code. Much much better is to think in terms of namespaces. What _namespaces_ depend on which others ... if you map that, we're getting somewhere. (libclang makes that easy BTW)

...

I tried to raise it as an issue anyway. Maybe in a year something will change. It took a long time for anyone in this community to take any notice of the concept of modularity at all, but now there seem to be a few people who get it. That took many many months though... Why is that?

There has already been very significant progress in persuading people to move psychologically on this. I'd even say 60% of the ground has been covered. I'd suggest keep going. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

4:04 a.m.

Stephen Kelly wrote:

...

It took a long time for anyone in this community to take any notice of the concept of modularity at all, but now there seem to be a few people who get it.

<tilts head> Sure, nobody has ever heard of the concept of dependencies before you showed up to educate us.

Matt Calabrese

3 Feb 3 Feb

10:20 p.m.

On Tue, Feb 3, 2015 at 2:01 PM, Niall Douglas wrote:

...

On 3 Feb 2015 at 22:02, Stephen Kelly wrote:

...
...
I confess that I haven't seen this in any way related to C++17. I would certainly be skeptical of anyone spending any time on something as speculative as a future C++ standard before it's finalized.

That's no reason not to educate yourself on it, and it's no reason not to try the clang implementation.

For the record, you can simulate clang's modules using any compiler by simply compiling in everything as a single translation unit. A simple shell script can create a file which includes all the source files at once.

If it works as a single translation unit, it'll work under clang modules. If it doesn't, well then you've got some ODR violation going on (very common in source files which assume they own their translation unit) and it may or may not work under clang modules depending.

Wait, I'm confused... is this actually true? Maybe I'm missing something, but I can imagine a simple example of an anonymous namespace in two translation units that define similar functions (i.e. each would have been defined and used in their own cpp in the multi-translation-unit version). This is not an ODR violation but it would fail compilation if done as you suggest. I'm not very familiar with modules as they are currently, but it seems like the assertion must be erroneous. -- -Matt Calabrese

Niall Douglas

4 Feb 4 Feb

12:14 a.m.

On 3 Feb 2015 at 14:20, Matt Calabrese wrote:

...

...
For the record, you can simulate clang's modules using any compiler by simply compiling in everything as a single translation unit. A simple shell script can create a file which includes all the source files at once.

If it works as a single translation unit, it'll work under clang modules. If it doesn't, well then you've got some ODR violation going on (very common in source files which assume they own their translation unit) and it may or may not work under clang modules depending.

Wait, I'm confused... is this actually true? Maybe I'm missing something, but I can imagine a simple example of an anonymous namespace in two translation units that define similar functions (i.e. each would have been defined and used in their own cpp in the multi-translation-unit version). This is not an ODR violation but it would fail compilation if done as you suggest. I'm not very familiar with modules as they are currently, but it seems like the assertion must be erroneous.

That is a very good point. Ok, as I said before, but excluding anonymous namespaces. I never use anonymous namespaces in my own code, and so never encountered that problem. I'd imagine a macro concatenating __COUNTER__ solution with inline namespaces could let you work around anonymous namespace collision. Or else just remove and name them explicitly, or make them named and inline. (BTW I am actually fairly sure clang's C++ Modules support currently doesn't understand anonymous namespaces, and in fact I am right unintentionally. But if so I am right through accident, not design). Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Robert Ramey

3 Feb 3 Feb

10:26 p.m.

Stephen Kelly-2 wrote

...

What way would boost headers be grouped together to make use of that?

...
As far as I can tell, my proposal for "bridging headers" is the only credible specific one suggested so far.

Erm, no. You pre-rejected any proposal based on splitting anything sensible out of the serialization repo.

The only thing I recall along this lines was something about "splitting out" xml?_archives. It didn't include enough details for me to consider it credible.

...

As the maintainer, you're the only one who gets to decide.

In boost we more or less have the principle that he who does the work get's to decide how to do it. If the rest of us don't like it, it fails to pass review. Of course we can change the maintainer - but I don't see anyone lining up for this job.

...

Yay Boost!

agreed

...

There's no reason to discuss anything which is pre-rejected.

There's no reason to discuss anything that isn't really a proposal but rather a vague idea.

...

Does your proposal break the cycle which serialization participates in?

I've looked at the reports and I'm not aware of any cycles. Feel free to point out any of these. I would be curious about this.

...

If it doesn't, then my comment on your proposal is: choose a solution which solves the cycle. If it does, please point that out.

In my views cycles aren't an issue. The shouldn't exist and if they do they should be fixed. I'm wondering if we're thinking the same thing when we use the word "cycle". What my proposal seeks to address is the following: a) someone decides he want's to include the date_time library so he includes the header date_time.hpp. (personally I don't like to include "convenience headers" but people in fact do that). b) this in turn includes date_time/serialization.hpp c) which in turn includes archive/basic_iarchive and a bunch of other stuff. d) now when he builds, he finds that he has to download/install the whole serialization library even though it's not calling functions in it. e) so he looks at that dependency report and finds out he has to download/install 14 other modules. He decides it's too much hassle so he just writes it himself. Under my proposal date_time/serialization.hpp doesn't include archive/basic_iarchive.hpp. It just has forward declarations class boost::archive::basic_Iarchive; Now when his application is built, he doesn't have to download/install 14 other libraries. His app will build and run just fine. Of course if he actually uses anything from the serialization library, he'll have to download the "archive" part of it. But for now he's happy. He could currently avoid all this problem by just not including data_time/serialization.hpp unless he actually needs it. His application isn't dependent upon it. But the current dependency reports don't actually tell him that because they look at the headers in the module not in the app. This is why we need the dependency tools to generate dependencies for any group of source files. (*)But since we're boost, we want to protect library users from themselves and permit the usage of convenience headers. So we implement my scheme. There's still a problem. date_time/serialization.hpp refers to the serialization part of the library which is header only. So that would have to be installed. This one might be called "splitting" the library (though it's actually al ready split - our tools don't recognize this). This would require some thinking - Vicente posted an idea about this. Another question arises - what do we need the "bridge" module for if each other module can declare forward headers. Answer we don't - except that it's help for maintenance to centralize these in one place. In also include in the serialization library (archive part) build to guarantee that they are correct and we're all using the same ones. So that's what it comes down to: a) addressing the (*) above b) thinking more careful what it means for a module(vs an application) to be dependent and enhancing the tools to reveal the information that is needed to address the problem. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Stephen Kelly

10:44 p.m.

Robert Ramey wrote:

...

Stephen Kelly-2 wrote

...
What way would boost headers be grouped together to make use of that?

...
As far as I can tell, my proposal for "bridging headers" is the only credible specific one suggested so far.

Erm, no. You pre-rejected any proposal based on splitting anything sensible out of the serialization repo.

The only thing I recall along this lines was something about "splitting out" xml?_archives. It didn't include enough details for me to consider it credible.

You reported that you are opposed to any splitting of the serialization repo. There are many ways to split it. You're opposed, and you reject any proposal based on splitting even before it is made. There's no way anyone can get past that barrier. Also, as I wrote before, you must see the problem before talking about a solution with you is worth anyones time.

...

...
There's no reason to discuss anything which is pre-rejected.

There's no reason to discuss anything that isn't really a proposal but rather a vague idea.

You have to see the problem before talking about a solution with you is worth anyones time. You don't see the problem.

...

...
Does your proposal break the cycle which serialization participates in?

I've looked at the reports and I'm not aware of any cycles. Feel free to point out any of these. I would be curious about this.

You are very tiresome. You've been part of threads discussing the cycle.

...

...
If it doesn't, then my comment on your proposal is: choose a solution which solves the cycle. If it does, please point that out.

In my views cycles aren't an issue. The shouldn't exist and if they do they should be fixed. I'm wondering if we're thinking the same thing when we use the word "cycle".

Very tiresome.

...

d) now when he builds, he finds that he has to download/install the whole serialization library even though it's not calling functions in it.

Boost is monolithic. You get boost.zip and you get all libraries. Developer has no problem until you decide to modularize (hasn't happened yet!) and make modular releases which have dependencies. Is that a goal or not? Thanks, Steve.

Robert Ramey

11:26 p.m.

Stephen Kelly-2 wrote

...

...
d) now when he builds, he finds that he has to download/install the whole serialization library even though it's not calling functions in it.

Boost is monolithic. You get boost.zip and you get all libraries.

It is now - that is what we're trying to address

...

Developer has no problem until you decide to modularize (hasn't happened yet!) and make modular releases which have dependencies.

Hmmm - I'm not sure who you mean by "you"

...

Is that a goal or not?

I can't say I understand what your goal is. I can say What I believe we want to move toward is the ability for users to deploy a useful subset of boost in order to support his applications. I don't think that's unreasonable or controversial. My problem is that I don't see the current dependency tools. the way of using the information provided by these tools, and the conclusions and proposals stemming from them as actually moving us toward the above goal. I've made a more concrete proposal. I believe that some variation of this could be made to work. I think it addresses the real problems that users face. In order for it to work, tools have to be improved and/or boost policies might need to be tweaked (perhaps a "no convenience headers" policy). There is just no way simply "splitting a library" is going to really address anything and move us forward. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Stephen Kelly

11:43 p.m.

Robert Ramey wrote:

...

I can say What I believe we want to move toward is the ability for users to deploy a useful subset of boost in order to support his applications. I don't think that's unreasonable or controversial.

Then tell us the user-story. Does the user download a tarball or not? What's the rest of the story?

...

I've made a more concrete proposal. I believe that some variation of this could be made to work. I think it addresses the real problems that users face.

What if you are completely wrong and out of touch? What if your concept of modularity is completely wrong? What if you should educate yourself about modules, but you don't?

...

There is just no way simply "splitting a library" is going to really address anything and move us forward.

It's a pre-rejected idea, as I said. Thanks, Steve.

Robert Ramey

4 Feb 4 Feb

12:22 a.m.

Stephen Kelly-2 wrote

...

Robert Ramey wrote:

...
I can say What I believe we want to move toward is the ability for users to deploy a useful subset of boost in order to support his applications. I don't think that's unreasonable or controversial.

Then tell us the user-story. Does the user download a tarball or not? What's the rest of the story?

If it were done via tar balls the procedure would look something like the following: a) There would be one tar ball per library. I guess that means that if done today that would mean around 118 libraries. b) user builds his app. he starts out small and includes a couple of libraries. Every time he loads a library, he runs the app through one of the dependency tools. This would tell him what other libraries he needs. He would download these new tarballs, expand them and built it app. c) At any point he would have the minimum subset of boost required to build his app. d) if his "app" consisted if a group of files, he'd run the group through the dependency tools to get the library list. e) if he wanted to test one of the libraries - he'd run the library test directory through the dependency tool in order to get a list of libraries he needs to build and run the tests. Of course, this would indicate a different (larger) list of libraries that he would have to download and install. This how I see the system working. Honestly, I don't see how it could work any other way. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Peter Dimov

4:21 a.m.

Robert Ramey wrote:

...

If it were done via tar balls the procedure would look something like the following:

a) There would be one tar ball per library. I guess that means that if done today that would mean around 118 libraries.

... You seem to have missed the bpm discussions, Robert, so here's the "announcement": http://lists.boost.org/Archives/boost/2015/01/218757.php http://lists.boost.org/Archives/boost/2015/01/218891.php Reading about Beman's experience may be of value too: http://lists.boost.org/Archives/boost/2015/01/218934.php bpm doesn't quite work the way you envisage, in that the user doesn't run a dependency checker locally to determine what needs to be installed (as the dependency checker can't actually dependency-check something that isn't installed yet). Instead, when you say "bpm install serialization", it downloads a pre-created module dependency graph and then downloads and installs whatever the graph says are serialization's dependencies. Give bpm a go and you'll hopefully get a feel for what we're (seemingly endlessly) talking about.

Steve M. Robbins

4:32 a.m.

On Tue, Feb 03, 2015 at 04:22:00PM -0800, Robert Ramey wrote:

...

If it were done via tar balls the procedure would look something like the following:

a) There would be one tar ball per library. I guess that means that if done today that would mean around 118 libraries.

So ... 118 tarballs?

...

b) user builds his app. he starts out small and includes a couple of libraries. Every time he loads a library, he runs the app through one of the dependency tools. This would tell him what other libraries he needs. He would download these new tarballs, expand them and built it app.

I can't imagine this to be a very common use case. After about 2-3 build failures due to missing libraries, your user is going to desperately want "all of boost" in one tarball. But suppose I'm wrong. Once the user has got 43 required libraries downloaded and the app builds ... time passes and he uses some previously-unused functionality from one of those 43 libraries that now pulls in a 44th: build failure. Ugh. It's not how I want to work. When want to use library X, I want X plus all its dependencies for any conceiveable program: "apt-get install X-dev" and go. I don't want to have to keep running a dependency tool on my source code. -Steve

Robert Ramey

5:10 a.m.

Steve M. Robbins-2 wrote

...

On Tue, Feb 03, 2015 at 04:22:00PM -0800, Robert Ramey wrote: I can't imagine this to be a very common use case. After about 2-3 build failures due to missing libraries, your user is going to desperately want "all of boost" in one tar ball.

what about when boost hits 500 libraries? Is he still going to want the whole thing? He'll always be able to get the "whole thing". That's not an issue. The whole discussion is predicated on the idea that one shouldn't have to download the whole thing. For other users - this discussion is not relevant.

...

It's not how I want to work. When want to use library X, I want X plus all its dependencies for any conceiveable program:

Hmmm and you want to distribute DLLS with your program which are many times larger than necessary? Will your customers be happy with that?

...

"apt-get install X-dev" and go. I don't want to have to keep running a dependency tool on my source code.

OK - no problem, you're already good to go. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Nevin Liber

4:07 p.m.

On 3 February 2015 at 23:10, Robert Ramey wrote:

...

Steve M. Robbins-2 wrote

...
On Tue, Feb 03, 2015 at 04:22:00PM -0800, Robert Ramey wrote: I can't imagine this to be a very common use case. After about 2-3 build failures due to missing libraries, your user is going to desperately want "all of boost" in one tar ball.

what about when boost hits 500 libraries?

Boost added what, *two* libraries last year? At that rate, the 500 library problem won't happen until the *23rd century*. Lots of other stuff has to scale first: the number of submissions, the number of reviews, the number of maintainers, etc. And yes, even at 500, I still want them all, if it minimizes my dependency management problem. I might want to take a newer/older version of a specific library, but that's about it. -- Nevin ":-)" Liber mailto:nevin@eviloverlord.com (847) 691-1404

Paul A. Bristow

4:23 p.m.

...

-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Nevin Liber Sent: 04 February 2015 16:08 To: Renee Silverman Subject: Re: [boost] [type_traits] Rewrite and dependency free version

...

...
what about when boost hits 500 libraries?

Boost added what, *two* libraries last year? At that rate, the 500 library problem won't happen until the *23rd century*.

Lots of other stuff has to scale first: the number of submissions, the number of reviews, the number of maintainers, etc.

And yes, even at 500, I still want them all, if it minimizes my dependency management problem. I might want to take a newer/older version of a specific library, but that's about it.

I've kept out of these not-too-measured discussions so far, but I am feeling the same way as Nevin. The whole build and module system of C/C++ is now very outdated. Work on C++ modules is proceeding, if slowly. The effect of C++Modules is likely to be very big (and the benefits big too). All our work reducing dependency could be in vain, or even make things worse. I am strongly opposed to making any more than trivial changes for some so-far illusory benefits. Please - let's wait until C++ modules are looking a bit more promising before we do anything more. Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830

Robert Ramey

5:07 p.m.

Paul A. Bristow-2 wrote

...

I've kept out of these not-too-measured discussions so far, but I am feeling the same way as Nevin. The whole build and module system of C/C++ is now very outdated. Work on C++ modules is proceeding, if slowly.

The effect of C++Modules is likely to be very big (and the benefits big too). All our work reducing dependency could be in vain, or even make things worse. I am strongly opposed to making any more than trivial changes for some so-far illusory benefits.

Please - let's wait until C++ modules are looking a bit more promising before we do anything more.

I certainly agree with this sentiment. But I am interested in the idea that we could make relatively small changes which would help reduce dependencies. I'm also interested in see how we can address the "bridge header" problem to reduce dependencies. I think the tools that have been made provide interesting and useful information. I would like to see them enhanced somewhat. I actually believe that we can make a lot of improvement with some carefully considered small changes. I guess we're "carefully considering" those changes on this list. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Peter Dimov

5:24 p.m.

Paul A. Bristow wrote:

...

The effect of C++Modules is likely to be very big (and the benefits big too).

All our work reducing dependency could be in vain, or even make things worse.

No module system, however magical, will let you include a header from a module you haven't downloaded and installed. And therefore, if we cut the number of cross-module includes (also known as dependencies), this can only lower the number of the required downloads when you only want a subset of Boost, it can never increase them.

Robert Ramey

5:01 p.m.

Nevin Liber wrote

...

...
what about when boost hits 500 libraries Boost added what, *two* libraries last year? At that rate, the 500 library problem won't happen until the *23rd century*.

LOL - I get this. But to me there is a much larger picture. I believe that boost has largely accomplished it original mission - and quite successfully I might add. I believe that Boost has to "think bigger" to continue to thrive. If it doesn't do so, it will atrophy to a bunch of old men living on past glories maintaining a bunch of arcane libraries which "didn't make it into the standard". So I believe we need to embrace a new and grander vision for the future. This vision would include an expectation of reaching 500 libraries in the next 5-10 years. (my actual vision is even grander, but I won't divulge it here in order to spare myself even more ridicule).

...

Lots of other stuff has to scale first: the number of submissions, the number of reviews, the number of maintainers, etc.

Of course - and we're trying to change those things. The incubator is focused on trying to increase the number of submissions and reviews. Part of it also tries to address the number of maintainers. Another issue is that boost testing and deployment currently wouldn't scale to support this vision.

...

And yes, even at 500, I still want them all, if it minimizes my dependency management problem. I might want to take a newer/older version of a specific library, but that's about it.

No problem, you shall continue to have it. I'm interested in expanding the access to boost beyond our small circle of nerds. Not everyone is like us. I realize that the discussion regarding minimizing dependencies is getting tiresome. But that's because it has aspects which are difficult to address. I realize that for a number of people it's my particular problem - but it just turns out that the library I happen to maintain highlights the weaknesses of the solutions proposed so far. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Steve M. Robbins

8 p.m.

On Tue, Feb 03, 2015 at 09:10:34PM -0800, Robert Ramey wrote:

...

Steve M. Robbins-2 wrote

...
On Tue, Feb 03, 2015 at 04:22:00PM -0800, Robert Ramey wrote: I can't imagine this to be a very common use case. After about 2-3 build failures due to missing libraries, your user is going to desperately want "all of boost" in one tar ball.

what about when boost hits 500 libraries? Is he still going to want the whole thing?

On my *development* machine: yes.

...

He'll always be able to get the "whole thing". That's not an issue. The whole discussion is predicated on the idea that one shouldn't have to download the whole thing. For other users - this discussion is not relevant.

Yes, I get that. I understand not wanting to download everything. For my use, package managers for linux distributions, Perl, Python, Ruby, etc, have basically figured it out. I gather that the "bpm" does the same for Boost. I guess I don't personally see a use-case for basing the dependencies on what my code today happens to include. It seems terribly fragile to me.

...

...
It's not how I want to work. When want to use library X, I want X plus all its dependencies for any conceiveable program:

Hmmm and you want to distribute DLLS with your program which are many times larger than necessary? Will your customers be happy with that?

I see that as a different problem. I'm certainly willing to pick and choose what to distribute and at that point I'd be willing to run a script on my source code to generate a minimal set of distributables. Preferably, it would be automatic and integrated into my build script.

...

...
"apt-get install X-dev" and go. I don't want to have to keep running a dependency tool on my source code.

OK - no problem, you're already good to go.

Yes. I respond only because I've never personally encountered your use case and was curious how widespread it really is. I'm sure you have more experience than I so I'd like to learn from it. Best, -Steve

Niall Douglas

12:27 a.m.

On 3 Feb 2015 at 15:26, Robert Ramey wrote:

...

I've made a more concrete proposal. I believe that some variation of this could be made to work. I think it addresses the real problems that users face. In order for it to work, tools have to be improved and/or boost policies might need to be tweaked (perhaps a "no convenience headers" policy). There is just no way simply "splitting a library" is going to really address anything and move us forward.

That last statement is simply untrue Robert. Have a look at https://ci.nedprod.com/view/Boost.AFIO/job/Boost.AFIO%20Build/ where the CI is now build testing all platforms for standalone use as well as Boost use. Standalone = no Boost whatsoever. So yes, splitting a library definitely can address a lot. I might add that the standalone build is 2x-5x faster than the Boost build for all unit tests. It's an enormous productivity, as well as convenience and ease of install, boon. And it really ought to be considered as the *recommended* future for all new and most existing Boost libraries instead of being constantly dismissed out of hand. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Niall Douglas

3 Feb 3 Feb

9:56 p.m.

On 3 Feb 2015 at 12:40, Robert Ramey wrote:

...

As far as I can tell, my proposal for "bridging headers" is the only credible specific one suggested so far. I conceded that I haven't spent a lot of time thinking through the implications (though it seems that I've spent more than others). That's why I've asked for comments. So far I haven't got any real comment on it. I'm still thinking about it.

Your bridging headers sound like a more brittle and harder to maintain equivalent to my API binding proposal. Mine does require a very new compiler however, and yours I expect does not. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

12:26 a.m.

Robert Ramey wrote: ...

...

Modest Proposal - Bridge module

We create a new module - named "bridge". In this module there is a directory: boost/bridge/archive.hpp. This contains only forward declarations:

class basic_iarchive; class basic_oarchive;

and maybe some other miscellaneous stuff. This includes no headers from boost/archive.

in date-time/serialization.hpp files in other libraries replace

#include with #include

Date_time doesn't include basic_iarchive.hpp. It doesn't include anything from boost/archive. It only includes boost/serialization/nvp.hpp and boost/serialization/split_free.hpp Look at the dependency report. Don't try to solve problems that don't exist.

Robert Ramey

1:07 a.m.

Peter Dimov-2 wrote

...

Robert Ramey wrote:

...

...
Modest Proposal - Bridge module

We create a new module - named "bridge". In this module there is a directory: boost/bridge/archive.hpp. This contains only forward declarations:

class basic_iarchive; class basic_oarchive;

and maybe some other miscellaneous stuff. This includes no headers from boost/archive.

in date-time/serialization.hpp files in other libraries replace

#include <boost/archive/basic_iarchive.hpp> with #include <boost/bridge/archive.hpp>

Date_time doesn't include basic_iarchive.hpp. It doesn't include anything from boost/archive. It only includes

boost/serialization/nvp.hpp

and

boost/serialization/split_free.hpp

Look at the dependency report. Don't try to solve problems that don't exist.

OK - I looked at the report and I think I understand the problem better now. I had presumed that the report followed header inclusions to determine dependencies. I see now that it doesn't. It looks that as soon as it finds finds that split_free is in the serialization module it just assumes that everything in the serialization module has to be included if date-time/serialization.hpp is included. That includes archive, xml_archive spirite - the whole ball of wax. Consider refining the dependency checker so that it doesn't consider the whole module but rather the top level directory. In this case, if you just included dependencies in boost/serialization rather than boost/serialization + boost/archive, we might already have what we need. I'm thinking this is the thrust of Rene's suggestion. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/type-traits-Rewrite-and-dependency-free-v... Sent from the Boost - Dev mailing list archive at Nabble.com.

Rene Rivera

4:01 a.m.

On Mon, Feb 2, 2015 at 7:07 PM, Robert Ramey wrote:

...

Peter Dimov-2 wrote

...
Look at the dependency report. Don't try to solve problems that don't exist.

OK - I looked at the report and I think I understand the problem better now. I had presumed that the report followed header inclusions to determine dependencies. I see now that it doesn't. It looks that as soon as it finds finds that split_free is in the serialization module it just assumes that everything in the serialization module has to be included if date-time/serialization.hpp is included. That includes archive, xml_archive spirite - the whole ball of wax. Consider refining the dependency checker so that it doesn't consider the whole module but rather the top level directory. In this case, if you just included dependencies in boost/serialization rather than boost/serialization + boost/archive, we might already have what we need. I'm thinking this is the thrust of Rene's suggestion.

Yes, such a split is a variation of my suggestion. We can standardize such a split. So that all libraries follow a structure that allow dependency analyses, and anyone else, to easily determine and use the parts that they need. Problem right now is that even if some particular library has core vs full already it's not in a structure that anyone can easily use. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

John Maddock

2 Feb 2 Feb

10:59 a.m.

...

...
The reason Stephen talks about splitting serialization is that adding serialization support needs only a few includes, independent of the rest of the library.

That may well be true, however, if the answer to every issue is "split it up" then something else is very wrong. It makes no sense to me to have multiple "micro-libraries" scattered over multiple repositories: no one will ever be able to find anything! As an aside: Multiprecision supports serialization, but without including any of that library's headers - rather it just provides the interface that the Serialization lib expects. While the magic-macros that Serialization provides to simplify providing support are useful for quick and dirty prototyping of code, I suspect that many (most, all?) of these dependencies could be removed by writing real code instead. So perhaps this is more a question of documentation in the Serialization lib, and officially-blessing interoperability via code, as well as the Serialization-provided macros? None of which is relevant to the common_type issue BTW. John.

Stephen Kelly

6:34 p.m.

John Maddock wrote:

...

...
...
The reason Stephen talks about splitting serialization is that adding serialization support needs only a few includes, independent of the rest of the library.

That may well be true, however, if the answer to every issue is "split it up" then something else is very wrong.

I agree. I advocate against splitting and I advocate for merging some core modules. Splitting is the answer to *this* issue.

...

It makes no sense to me to have multiple "micro-libraries" scattered over multiple repositories: no one will ever be able to find anything!

As an aside: Multiprecision supports serialization, but without including any of that library's headers - rather it just provides the interface that the Serialization lib expects. While the magic-macros that Serialization provides to simplify providing support are useful for quick and dirty prototyping of code, I suspect that many (most, all?) of these dependencies could be removed by writing real code instead. So perhaps this is more a question of documentation in the Serialization lib, and officially-blessing interoperability via code, as well as the Serialization-provided macros?

Or splitting off a proper interface for those libraries to implement.

...

None of which is relevant to the common_type issue BTW.

Right. This started because I pointed out that serialization is not a general 'library X library Y' problem which you suggested in a previous email. Thanks, Steve.

Peter Dimov

6:53 p.m.

John Maddock wrote:

...

As an aside: Multiprecision supports serialization, but without including any of that library's headers - rather it just provides the interface that the Serialization lib expects.

That's how things should be, in principle. But I thought that you need at least nvp.hpp in order to support XML archives?

3570

Age (days ago)

3588

Last active (days ago)

List overview

Download

115 comments

22 participants

participants (22)

Agustín K-ballo Bergé
Andrey Semashev
Beman Dawes
Bjørn Roald
Edward Diener
Frédéric Bron
Ion Gaztañaga
John Maddock
John Maddock
Matt Calabrese
Neil Groves
Nevin Liber
Niall Douglas
Paul A. Bristow
Peter Dimov
pfultz2
Rene Rivera
Robert Ramey
Stephen Kelly
Steve M. Robbins
Tim Blechmann
Vicente J. Botet Escriba