[serialization] Static data members.

Many of the problems that CodeWarrior, and True64, have with the serialization library stem from the behavior of CW to not call constructors for static data members in one particular class in the library. I've come to think that this is standard conforming behavior. The serialization library relies on a static member of a template class being constructed at global initialization time to figure out what all the serialized classes are. The problem is that the template class is being instantiated implicitly. Which for most compilers also instantiates any static members of that template class. It's my understanding that this violates 14.7.1.7: "The implicit instantiation of a class template does not cause any static data members of that class to be implicitly instantiated." Question I have.. Is my understanding correct? Or is this a bug in CW part? For reference the code in question is in "boost/serialization/void_cast.hpp".. http://cvs.sourceforge.net/viewcvs.py/boost/boost/boost/serialization/void_c... http://tinyurl.com/9s5jb The template class is void_caster_primitive, and it's instantiated from the void_cast_register function at the bottom. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org

Rene Rivera <grafik.list@redshift-software.com> writes:
Many of the problems that CodeWarrior, and True64, have with the serialization library stem from the behavior of CW to not call constructors for static data members in one particular class in the library. I've come to think that this is standard conforming behavior.
The serialization library relies on a static member of a template class being constructed at global initialization time to figure out what all the serialized classes are. The problem is that the template class is being instantiated implicitly. Which for most compilers also instantiates any static members of that template class. It's my understanding that this violates 14.7.1.7:
"The implicit instantiation of a class template does not cause any static data members of that class to be implicitly instantiated."
Question I have.. Is my understanding correct? Or is this a bug in CW part?
Your understanding is certainly correct. It's standard practice in traits templates, for example, to do something like: template <class T> struct is_whatever { static T& x; template <class U> char test(U&); template <class U> char (& test( whatever<U>& ) )[2]; enum { value = sizeof(test(x)) == 2 }; }; Nobody ever even writes an initializer for x. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:
Rene Rivera <grafik.list@redshift-software.com> writes:
Many of the problems that CodeWarrior, and True64, have with the serialization library stem from the behavior of CW to not call constructors for static data members in one particular class in the library. I've come to think that this is standard conforming behavior.
The serialization library relies on a static member of a template class being constructed at global initialization time to figure out what all the serialized classes are. The problem is that the template class is being instantiated implicitly. Which for most compilers also instantiates any static members of that template class. It's my understanding that this violates 14.7.1.7:
"The implicit instantiation of a class template does not cause any static data members of that class to be implicitly instantiated."
Question I have.. Is my understanding correct? Or is this a bug in CW part?
Your understanding is certainly correct. It's standard practice in traits templates, for example, to do something like:
template <class T> struct is_whatever { static T& x;
template <class U> char test(U&);
template <class U> char (& test( whatever<U>& ) )[2];
enum { value = sizeof(test(x)) == 2 }; };
Nobody ever even writes an initializer for x.
This is different than our case. we have: template<class T> struct A { static T t; ... }; template<class T> T A<T>::t and somewhere in the program we do void f(){ T & t = A<T>::t; } We expect A<T>::t to be instanticiated and the program to link and run as expected. In fact on most compilers it does this. This group includes those known to be the most conforming ones including the latest GCC, VC and Comeau compilers. With CW, it seems that A<T> t is allocated but the constructor for t isn't being called - at least at pre-compile time. It might be being called on first usage as it would be in the following case. void f() { static T t; .. } Now Rene has looked into this and raises the possibility that CW is correct in its behavior - and all the other compilers are wrong. Rene's quote from the standard would suggest to me that A<T>::t should never even be instantiated. If that's the case why is A<T>::t even being allocated? This is quite murky to me. Robert Ramey

Robert Ramey wrote:
David Abrahams wrote:
Rene Rivera <grafik.list@redshift-software.com> writes:
Many of the problems that CodeWarrior, and True64, have with the serialization library stem from the behavior of CW to not call constructors for static data members in one particular class in the library. I've come to think that this is standard conforming behavior.
The serialization library relies on a static member of a template class being constructed at global initialization time to figure out what all the serialized classes are. The problem is that the template class is being instantiated implicitly. Which for most compilers also instantiates any static members of that template class. It's my understanding that this violates 14.7.1.7:
"The implicit instantiation of a class template does not cause any static data members of that class to be implicitly instantiated."
Question I have.. Is my understanding correct? Or is this a bug in CW part?
Your understanding is certainly correct. It's standard practice in traits templates, for example, to do something like:
template <class T> struct is_whatever { static T& x;
template <class U> char test(U&);
template <class U> char (& test( whatever<U>& ) )[2];
enum { value = sizeof(test(x)) == 2 }; };
Nobody ever even writes an initializer for x.
This is different than our case. we have:
template<class T> struct A { static T t; ... };
template<class T> T A<T>::t
and somewhere in the program we do
void f(){ T & t = A<T>::t; }
To correct that a bit.. In the code in question we actually do the equivalent of: template <typename T> void f() { void * t = &A<T>::t; } Hence the implicit aspect of the instantiation. Which I think CW decides is not sufficient for the semantic requirement that the other rules in 14.7.1 refer to. But it's not clear how the other rules of 14.7.1 should apply for the implicit instantiation case that 14.7.1.7 refers to. -- Well at least not clear to me :-) But 14.7.1.1 does say this... which possibly supports what 14.7.1.7 says. 14.7.1.1: "The implicit instantiation of a class template specialization causes the implicit instantiation of the declarations, _but not of the definitions_ or default arguments, of the class member functions, member classes, _static data members_ and member templates; and it causes the implicit instantiation of the definitions of member anonymous unions."
We expect A<T>::t to be instanticiated and the program to link and run as expected. In fact on most compilers it does this. This group includes those known to be the most conforming ones including the latest GCC, VC and Comeau compilers.
With CW, it seems that A<T> t is allocated but the constructor for t isn't being called - at least at pre-compile time. It might be being called on first usage as it would be in the following case.
void f() { static T t; .. }
It's not getting called, no matter what. Or more precisely CW is not adding the constructor+instance to the global initialization function list that CW generates. The only circumstance it generates the init function is one explicitly defines the member: struct B; template A<B>::t;
Now Rene has looked into this and raises the possibility that CW is correct in its behavior - and all the other compilers are wrong. Rene's quote from the standard would suggest to me that A<T>::t should never even be instantiated. If that's the case why is A<T>::t even being allocated?
This is quite murky to me.
Somewhat murky for me also. I think this not an area of compilers that gets exercised this way. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org

Rene Rivera <grafik.list@redshift-software.com> writes:
To correct that a bit.. In the code in question we actually do the equivalent of:
template <typename T> void f() { void * t = &A<T>::t; }
That clearly requires A<T>::t to exist, and thus causes its instantiation: 14.7.1 Implicit instantiation [temp.inst] 1 Unless a class template specialization has been explicitly instantiated (14.7.2) or explicitly specialized (14.7.3), the class template specialization is implicitly instantiated when the specialization is referenced in a context that requires a completely-defined object type or when the completeness of the class type affects the semantics of the program. The implicit instantiation of a class template specialization causes the implicit instantiation of the declarations, but not of the definitions or default arguments, of the class member functions, member classes, static data members and member templates; and it causes the implicit instantiation of the definitions of member anonymous unions. Unless a member of a class template or a member template has been explicitly instantiated or explicitly specialized, the specialization of the member is implicitly instantiated when the specialization is referenced in a context that requires the member definition to exist; in particular, the initialization (and any associated side-effects) of a static data member does not occur unless the static data member is itself used in a way that requires the definition of the static data member to exist. Note in particular that last sentence.
Hence the implicit aspect of the instantiation. Which I think CW decides is not sufficient for the semantic requirement that the other rules in 14.7.1 refer to. But it's not clear how the other rules of 14.7.1 should apply for the implicit instantiation case that 14.7.1.7 refers to. --
I think you mean 14.7.1ΒΆ7 or 14.7.1/7 if you prefer.
Well at least not clear to me :-) But 14.7.1.1 does say this... which possibly supports what 14.7.1.7 says.
14.7.1.1: "The implicit instantiation of a class template specialization causes the implicit instantiation of the declarations, _but not of the definitions_ or default arguments, of the class member functions, member classes, _static data members_ and member templates; and it causes the implicit instantiation of the definitions of member anonymous unions."
But you missed the most important part of that paragraph ;-)
We expect A<T>::t to be instanticiated and the program to link and run as expected. In fact on most compilers it does this. This group includes those known to be the most conforming ones including the latest GCC, VC and Comeau compilers. With CW, it seems that A<T> t is allocated but the constructor for t isn't being called - at least at pre-compile time. It might be being called on first usage as it would be in the following case. void f() { static T t; .. }
It's not getting called, no matter what. Or more precisely CW is not adding the constructor+instance to the global initialization function list that CW generates.
I think it's a bug and should be reported to MW, but that said, check out boost/python/detail/force_instantiate.hpp, which I use for this exact purpose. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote: [..] Got all that :-)
classes, _static data members_ and member templates; and it causes the implicit instantiation of the definitions of member anonymous unions."
But you missed the most important part of that paragraph ;-)
I didn't really miss it... It would be CW who missed it ;-)
It's not getting called, no matter what. Or more precisely CW is not adding the constructor+instance to the global initialization function list that CW generates.
I think it's a bug and should be reported to MW, but that said, check out boost/python/detail/force_instantiate.hpp, which I use for this exact purpose.
I just did, doesn't help :-( -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org

Rene Rivera <grafik.list@redshift-software.com> writes:
David Abrahams wrote:
[..]
Got all that :-)
classes, _static data members_ and member templates; and it causes the implicit instantiation of the definitions of member anonymous unions." But you missed the most important part of that paragraph ;-)
I didn't really miss it... It would be CW who missed it ;-)
It's not getting called, no matter what. Or more precisely CW is not adding the constructor+instance to the global initialization function list that CW generates. I think it's a bug and should be reported to MW, but that said, check out boost/python/detail/force_instantiate.hpp, which I use for this exact purpose.
I just did, doesn't help :-(
You have to ensure that force_instantiate(foo<X>::bar) gets called, of course. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:
Rene Rivera <grafik.list@redshift-software.com> writes:
David Abrahams wrote:
I think it's a bug and should be reported to MW, but that said, check out boost/python/detail/force_instantiate.hpp, which I use for this exact purpose.
I just did, doesn't help :-(
You have to ensure that force_instantiate(foo<X>::bar) gets called, of course.
Of course :-) What I did was this: template<class Derived, class Base> BOOST_DLLEXPORT const void * void_cast_register( const Derived * /* dnull = NULL */, const Base * /* bnull = NULL */ ){ boost::python::detail::force_instantiate( void_cast_detail::void_caster_primitive< const Derived, const Base >::instance); return & void_cast_detail::void_caster_primitive< const Derived, const Base >::instance; } -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org

Rene Rivera <grafik.list@redshift-software.com> writes:
Of course :-) What I did was this:
template<class Derived, class Base> BOOST_DLLEXPORT const void * void_cast_register( const Derived * /* dnull = NULL */, const Base * /* bnull = NULL */ ){ boost::python::detail::force_instantiate( void_cast_detail::void_caster_primitive< const Derived, const Base >::instance); return & void_cast_detail::void_caster_primitive< const Derived, const Base >::instance; }
Ouch! Looks like a nasty bug. If you can generate a small test case I can pass it directly to their compiler engineer. -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams wrote:
Rene Rivera <grafik.list@redshift-software.com> writes:
Of course :-) What I did was this:
template<class Derived, class Base> BOOST_DLLEXPORT const void * void_cast_register( const Derived * /* dnull = NULL */, const Base * /* bnull = NULL */ ){ boost::python::detail::force_instantiate( void_cast_detail::void_caster_primitive< const Derived, const Base >::instance); return & void_cast_detail::void_caster_primitive< const Derived, const Base
::instance; }
Ouch! Looks like a nasty bug. If you can generate a small test case I can pass it directly to their compiler engineer.
I'll try. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org

Rene Rivera wrote:
David Abrahams wrote:
Rene Rivera <grafik.list@redshift-software.com> writes:
Of course :-) What I did was this:
template<class Derived, class Base> BOOST_DLLEXPORT const void * void_cast_register( const Derived * /* dnull = NULL */, const Base * /* bnull = NULL */ ){ boost::python::detail::force_instantiate( void_cast_detail::void_caster_primitive< const Derived, const Base >::instance); return & void_cast_detail::void_caster_primitive< const Derived, const Base
::instance; }
Ouch! Looks like a nasty bug. If you can generate a small test case I can pass it directly to their compiler engineer.
I'll try.
This has been very interesting. Now I'm wondering why test_void_cast passes. It seems to be a simple test case to me and should fail according to the above. But it passes on all the CW compilers. Robert Ramey

Robert Ramey wrote:
Rene Rivera wrote:
David Abrahams wrote:
Rene Rivera <grafik.list@redshift-software.com> writes:
Of course :-) What I did was this:
template<class Derived, class Base> BOOST_DLLEXPORT const void * void_cast_register( const Derived * /* dnull = NULL */, const Base * /* bnull = NULL */ ){ boost::python::detail::force_instantiate( void_cast_detail::void_caster_primitive< const Derived, const Base >::instance); return & void_cast_detail::void_caster_primitive< const Derived, const Base
::instance; }
Ouch! Looks like a nasty bug. If you can generate a small test case I can pass it directly to their compiler engineer.
I'll try.
This has been very interesting. Now I'm wondering why test_void_cast passes. It seems to be a simple test case to me and should fail according to the above. But it passes on all the CW compilers.
I'm starting to suspect that it has something to do with the template depth and complexity. In test_void_cast the void_cast_register gets called directly, so it doesn't have problems. But at some point in calling it indirectly it likely gets confused. Don't really know.. As I've been unable to duplicate the problem. Here's my current, now fairly complicated as I keep adding to it, test.. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org #include <iostream> #include <boost/mpl/eval_if.hpp> #include <boost/mpl/identity.hpp> #include <boost/type_traits/is_pod.hpp> struct Z { Z() {} virtual ~Z() {} }; struct X { const Z & m; X(const Z & n); virtual ~X(); static void reg(const X *); }; template <typename T> struct A : public X { static const A a; A(); }; template <typename T> A<T>::A() : X(*(new Z())) { std::cout << __PRETTY_FUNCTION__ << '\n'; this->reg(& a); } template <typename T> const A<T> A<T>::a; template <typename T> const void * instantiate(const T * t) { return &A<T>::a; } template <typename T> struct Q { struct nothing { static void invoke() { } }; template <typename S> struct something { static void invoke() { instantiate<S>(static_cast<const S *>(0)); } }; static void invoke() { typedef typename boost::mpl::eval_if< typename boost::is_pod<T>, boost::mpl::identity< something<T> >, boost::mpl::identity< nothing > >::type typex; typex::invoke(); } }; template <typename T> inline const T & base_obj(const T & t) { Q<T>::invoke(); return t; } template <typename T> std::pair<std::string,T*> make_nvp(const char * c, T & t) { return std::make_pair(c,&t); } int main() { int n = 0; make_nvp("int",base_obj(n)); std::cout << __PRETTY_FUNCTION__ << '\n'; return 0; } X::X(const Z & n) : m(n) { std::cout << __PRETTY_FUNCTION__ << '\n'; } X::~X() { std::cout << __PRETTY_FUNCTION__ << '\n'; } void X::reg(const X *) { std::cout << __PRETTY_FUNCTION__ << '\n'; }

Rene Rivera <grafik.list@redshift-software.com> writes:
I'm starting to suspect that it has something to do with the template depth and complexity. In test_void_cast the void_cast_register gets called directly, so it doesn't have problems. But at some point in calling it indirectly it likely gets confused. Don't really know.. As I've been unable to duplicate the problem. Here's my current, now fairly complicated as I keep adding to it, test..
<snip> At some point you have to give up and decide only the guy with the code can solve it. If you send me a compressed preprocessed file that doesn't compile along with the rationale for why it should not fail I'll pass it along. Cheers, -- Dave Abrahams Boost Consulting www.boost-consulting.com
participants (3)
-
David Abrahams
-
Rene Rivera
-
Robert Ramey