[python] Tru64/CXX function template issues
Hello, unfortunately boost.python is unusable on that platform because most of the tests fail at runtime with a stack overflow. The reason is illustrated in the following code sample: ---%<--- void foo(int); template <class T> void foo(T) { foo(0); } int main() { foo(0); } --->%--- When compiling this with CXX 6.5, the code compiles and links cleanly, despite the fact that the overload for foo() is not defined. Instead the compiler instantiates foo<int>(int) and then uses the instantiation for the call to foo() which of course leads to a stack overflow. I had a look at the standard what should happen in this case, but after reading [temp.over] I'm not really sure if we don't indeed end up with two candidate functions for the overload resolution of foo(), one being the overload itself and the other the specialization of the function template for int. Could anyone please clarify this? As for a possible workaround, replacing the overload with a specialization of the function template would work, AFAICT. Would such a patch be accepted? Markus
Markus Schöpflin <markus.schoepflin@comsoft.de> writes:
Hello,
unfortunately boost.python is unusable on that platform because most of the tests fail at runtime with a stack overflow. The reason is illustrated in the following code sample:
---%<--- void foo(int); template <class T> void foo(T) { foo(0); }
int main() { foo(0); } --->%---
When compiling this with CXX 6.5, the code compiles and links cleanly, despite the fact that the overload for foo() is not defined. Instead the compiler instantiates foo<int>(int) and then uses the instantiation for the call to foo() which of course leads to a stack overflow.
I'm confused. I was pretty sure Ralf was running Boost.Python on that compiler. Ralf?
I had a look at the standard what should happen in this case, but after reading [temp.over] I'm not really sure if we don't indeed end up with two candidate functions for the overload resolution of foo(), one being the overload itself and the other the specialization of the function template for int. Could anyone please clarify this?
The non-template function should be preferred. The function template should never even get instantiated.
As for a possible workaround, replacing the overload with a specialization of the function template would work, AFAICT. Would such a patch be accepted?
If it's a localized patch somewhere in Boost.Python, it might. But this sounds like the sort of problem that would require sweeping fixes. No? -- Dave Abrahams Boost Consulting www.boost-consulting.com
David Abrahams wrote:
Markus Schöpflin <markus.schoepflin@comsoft.de> writes:
[snip]
I'm confused. I was pretty sure Ralf was running Boost.Python on that compiler. Ralf?
Hmm, the tests have been failing for 1.32 as well. It was the same problem.
I had a look at the standard what should happen in this case, but after reading [temp.over] I'm not really sure if we don't indeed end up with two candidate functions for the overload resolution of foo(), one being the overload itself and the other the specialization of the function template for int. Could anyone please clarify this?
The non-template function should be preferred. The function template should never even get instantiated.
Could you perhaps point me to the section in the standard where this is specified? [temp.over] doesn't require this, I think.
As for a possible workaround, replacing the overload with a specialization of the function template would work, AFAICT. Would such a patch be accepted?
If it's a localized patch somewhere in Boost.Python, it might. But this sounds like the sort of problem that would require sweeping fixes. No?
I'll have to try to be sure, but I think you are right, it will require tweaking much code. Markus
Markus Schöpflin wrote:
David Abrahams wrote:
The non-template function should be preferred. The function template should never even get instantiated.
Could you perhaps point me to the section in the standard where this is specified? [temp.over] doesn't require this, I think.
[over.match.best] is what you need.
"Peter Dimov" <pdimov@mmltd.net> writes:
Markus Schöpflin wrote:
David Abrahams wrote:
The non-template function should be preferred. The function template should never even get instantiated.
Could you perhaps point me to the section in the standard where this is specified? [temp.over] doesn't require this, I think.
[over.match.best] is what you need.
Thanks, Peter. I know these things, but don't know where they are :) -- Dave Abrahams Boost Consulting www.boost-consulting.com
--- Markus Schöpflin <markus.schoepflin@comsoft.de> wrote:
unfortunately boost.python is unusable on that platform because most of the tests fail at runtime with a stack overflow. The reason is illustrated in the following code sample:
I am using Boost.Python all the time under Tru64 with cxx 6.5, and have been doing so for many years. I guess there must be a difference in how we compile. I am not using bjam but a SCons-based build system. The result of building with the Boost CVS from earlier this morning, including the cxx commands, is here: http://cci.lbl.gov/cctbx_build/results/2005_06_15_0643/tru64_py241_build_log All our regression tests (many) passed.
As for a possible workaround, replacing the overload with a specialization of the function template would work, AFAICT. Would such a patch be accepted?
I'd first experiment with the compiler options. Cheers, Ralf __________________________________ Discover Yahoo! Use Yahoo! to plan a weekend, have fun online and more. Check it out! http://discover.yahoo.com/
Ralf W. Grosse-Kunstleve wrote:
I am using Boost.Python all the time under Tru64 with cxx 6.5, and have been doing so for many years. I guess there must be a difference in how we compile. I am not using bjam but a SCons-based build system. The result of building with the Boost CVS from earlier this morning, including the cxx commands, is here:
http://cci.lbl.gov/cctbx_build/results/2005_06_15_0643/tru64_py241_build_log
All our regression tests (many) passed.
Could you please try the sample I posted and tell me if it compiles and links ok with your compiler version? Markus
--- Markus Schöpflin <markus.schoepflin@web.de> wrote:
Ralf W. Grosse-Kunstleve wrote:
I am using Boost.Python all the time under Tru64 with cxx 6.5, and have been doing so for many years. I guess there must be a difference in how we compile. I am not using bjam but a SCons-based build system. The result of building with the Boost CVS from earlier this morning, including the cxx commands, is here:
http://cci.lbl.gov/cctbx_build/results/2005_06_15_0643/tru64_py241_build_log
All our regression tests (many) passed.
Could you please try the sample I posted and tell me if it compiles and links ok with your compiler version?
% cxx -V Compaq C++ V6.5-042 for Compaq Tru64 UNIX V5.1 (Rev. 732) Compiler Driver V6.5-042 (cxx) cxx Driver % cat z.cpp void foo(int); template <class T> void foo(T) { foo(0); } int main() { foo(0); } % cxx -std strict_ansi -tlocal -O2 -fast z.cpp ld: Unresolved: foo(int) % cxx -std strict_ansi -tlocal z.cpp ld: Unresolved: foo(int) % cxx -std strict_ansi z.cpp ld: Unresolved: foo(int) % cxx -std strict_ansi -model ansi z.cpp ld: Unresolved: foo(int) % cxx -std ansi z.cpp % a.out # takes "forever" % cxx z.cpp % a.out # takes "forever" It seems to be important to use "strict_ansi". That's what I have been using for Boost.Python compilation all the time. Cheers, Ralf __________________________________ Discover Yahoo! Use Yahoo! to plan a weekend, have fun online and more. Check it out! http://discover.yahoo.com/
Ralf W. Grosse-Kunstleve wrote:
--- Markus Schöpflin <markus.schoepflin@web.de> wrote:
Could you please try the sample I posted and tell me if it compiles and links ok with your compiler version?
% cxx -V Compaq C++ V6.5-042 for Compaq Tru64 UNIX V5.1 (Rev. 732) Compiler Driver V6.5-042 (cxx) cxx Driver
That's the same version as I'm using.
It seems to be important to use "strict_ansi". That's what I have been using for Boost.Python compilation all the time.
Oh well, that's the difference then. The tru64cxx65 toolset uses "ansi", and not "strict_ansi". The compiler is way to strict for most of the boost libraries in that mode. Hmm, maybe -std strict_ansi together with -nopure_cname and -timplicit_local is worth a try, then. I'll see if I can figure out something. Markus
Ralf W. Grosse-Kunstleve wrote:
It seems to be important to use "strict_ansi". That's what I have been using for Boost.Python compilation all the time.
Got a statement from the compiler support team: <quote> In the default relaxed ansi mode, we are more permissive and support the ARM non-standard feature named "guiding declaration". So while the non-template function declaration is seen, the definition is still supplied by the template function. In the strict_ansi mode, we conform to the ISO standard and look to the non-template function for a definition. </quote> Seems like we might have to do the switch to strict_ansi, then. Markus
Ralf W. Grosse-Kunstleve wrote:
I am using Boost.Python all the time under Tru64 with cxx 6.5, and have been doing so for many years. I guess there must be a difference in how we compile. I am not using bjam but a SCons-based build system. The result of building with the Boost CVS from earlier this morning, including the cxx commands, is here:
http://cci.lbl.gov/cctbx_build/results/2005_06_15_0643/tru64_py241_build_log
All our regression tests (many) passed.
Do you use the patched header file for <iotraits> when compiling boost.python? Because I just tried compiling with different options and now I got bitten by the missing support for std::fpos_t. cxx -c -std strict_ansi -nopure_cname -noimplicit_include -timplicit_local -ptr "../../../bin/boost/libs/python/build/libboost_python.so/tru64cxx65-042/debug/cxx_repository" -msg_display_number -msg_disable 186,450,1115 -DBOOST_PYTHON_DYNAMIC_LIB -DBOOST_PYTHON_SOURCE -g3 -O0 -inline none -version V6.5-042 -ieee -gall -D__CNAME_OVERLOADS -model ansi -I"../../../bin/boost/libs/python/build" -I"/usr/include" -I"/net/camos/opt/include/python2.3" -I"/net/camos/opt/exec/OSF1-V5.1-alpha/include/python2.3" -I"/net/users/schoepflin/src/boost/boost-HEAD" -o "../../../bin/boost/libs/python/build/libboost_python.so/tru64cxx65-042/debug/function.o" "/net/users/schoepflin/src/boost/boost-HEAD/libs/python/build/../src/object/function.cpp" ...failed tru64cxx65-C++-action ../../../bin/boost/libs/python/build/libboost_python.so/tru64cxx65-042/debug/function.o... tru64cxx65-C++-action ../../../bin/boost/libs/python/build/libboost_python.so/tru64cxx65-042/debug/inheritance.o cxx: Error: /usr/lib/cmplrs/cxx/V6.5-042/include/cxx/iotraits, line 246: #415 no suitable constructor exists to convert from "int" to "std::fpos_t" , _RWfp(fpos_t(0)) -----------------^ cxx: Error: /usr/lib/cmplrs/cxx/V6.5-042/include/cxx/iotraits, line 330: #413 no suitable conversion function from "std::fpos_t" to "long" exists long var=_RWfp; -------------^ ... Markus
--- Markus Schöpflin <markus.schoepflin@comsoft.de> wrote:
Ralf W. Grosse-Kunstleve wrote:
I am using Boost.Python all the time under Tru64 with cxx 6.5, and have been doing so for many years. I guess there must be a difference in how we compile. I am not using bjam but a SCons-based build system. The result of building with the Boost CVS from earlier this morning, including the cxx commands, is here:
http://cci.lbl.gov/cctbx_build/results/2005_06_15_0643/tru64_py241_build_log
All our regression tests (many) passed.
Do you use the patched header file for <iotraits> when compiling boost.python? Because I just tried compiling with different options and now I got bitten by the missing support for std::fpos_t.
I don't think so. I am using the Boost tree as I get it with cvs update. If you want to find out exactly what I am doing go to http://cci.lbl.gov/cctbx_build/all.html and copy the source bundle including Python into a new empty directory on your Tru64 machine. Then simply run perl cctbx_python_241_bundle.selfx and watch the cxx commands fly by. Cheers, Ralf __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Ralf W. Grosse-Kunstleve wrote:
--- Markus Schöpflin <markus.schoepflin@comsoft.de> wrote:
Do you use the patched header file for <iotraits> when compiling boost.python? Because I just tried compiling with different options and now I got bitten by the missing support for std::fpos_t.
I don't think so. I am using the Boost tree as I get it with cvs update.
[snip] Thanks for the pointer, I'll go and try it. But could you please be so kind and check in your "pyconfig.h" for the setting of _XOPEN_SOURCE? I strongly suspect it's something below 500. If not, could you please check the iotraits header file for the following code snippet, located somewhere at line 138. If it's the same, then I have no explanation for the difference we get. But if the condition includes another "|| (defined(__osf__) && ((_XOPEN_SOURCE+0) >= 500)))" then you somehow have aquired the iotraits header fix. #if defined(__DECCXX) && !defined(__DECFIXCXXL997) && (defined(__VMS)||defined(__linux__)) inline fpos(long, long, state_type); #else inline fpos(long, fpos_t, state_type); TIA, Markus
Ralf W. Grosse-Kunstleve wrote:
If you want to find out exactly what I am doing go to
http://cci.lbl.gov/cctbx_build/all.html
and copy the source bundle including Python into a new empty directory on your Tru64 machine. Then simply run
perl cctbx_python_241_bundle.selfx
and watch the cxx commands fly by.
Ok, I did this and yes, it works. And the essential difference is... -I/usr/include !!! boost.build somehow thinks it necessary to include this path in the compiler command line and this makes things break. Now I have to check why boost.build is doing this and how to stop it from doing it. Thanks for the help, Markus
Markus Schöpflin wrote:
Ralf W. Grosse-Kunstleve wrote:
If you want to find out exactly what I am doing go to
http://cci.lbl.gov/cctbx_build/all.html
and copy the source bundle including Python into a new empty directory on your Tru64 machine. Then simply run
perl cctbx_python_241_bundle.selfx
and watch the cxx commands fly by.
Ok, I did this and yes, it works. And the essential difference is... -I/usr/include !!!
boost.build somehow thinks it necessary to include this path in the compiler command line and this makes things break.
Ralf, either you're a genious or it's pure coincidence that you are not hit by the bug in the iotraits header file. :-) To summarize, this only compiles on Tru64 if /usr/include is not in the include path. For this to be the case, it must not be specified on the command line and the pure_cname compiler option must be in effect. To quote from the cxx man page:
-[no]pure_cname [Tru64] When used in conjunction with the <cname> headers, -pure_cname defines C names in namespace std only as specified by the C++ Interna- tional Standard. When used with cname.h or <cname> headers, -nopure_cname defines C names in namespace std and at global namespace scope. The -pure_cname option is enabled by default when you specify the -std strict_ansi and -std strict_ansi_errors options. The -pure_cname option also defines the __PURE_CNAME macro.
This is pretty bad because quite a few boost libraries depend on more than only the C names as specified in the standard. (thread and fs at least, IIRC, but I think there are more) So I think there is no general solution to make this work for Tru64. I could try and remove the -nopure_cname from the compiler command line and see which libraries fail after that, but I have to admit I'm not too hopeful that this will lead us anywhere. Markus
--- Markus Schöpflin <markus.schoepflin@comsoft.de> wrote:
Ralf, either you're a genious or it's pure coincidence that you are not hit by the bug in the iotraits header file. :-)
I like to take the best of everything, that would make me a pure genious. Thanks! :)
This is pretty bad because quite a few boost libraries depend on more than only the C names as specified in the standard. (thread and fs at least, IIRC, but I think there are more)
In the hot days of Boost.Python development we were very careful to use only ANSI features. Anything else is non-portable and bound to break somewhere, sooner or later. I.e. I think the libraries that don't compile with -std strict_ansi should be fixed. Cheers, Ralf __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
"Ralf W. Grosse-Kunstleve" <rwgk@yahoo.com> writes:
--- Markus Schöpflin <markus.schoepflin@comsoft.de> wrote:
Ralf, either you're a genious or it's pure coincidence that you are not hit by the bug in the iotraits header file. :-)
I like to take the best of everything, that would make me a pure genious. Thanks! :)
This is pretty bad because quite a few boost libraries depend on more than only the C names as specified in the standard. (thread and fs at least, IIRC, but I think there are more)
In the hot days of Boost.Python development we were very careful to use only ANSI features. Anything else is non-portable and bound to break somewhere, sooner or later. I.e. I think the libraries that don't compile with -std strict_ansi should be fixed.
pthreads is an extension to what's required by ANSI. I think that might make Boost.Threads impossible. -- Dave Abrahams Boost Consulting www.boost-consulting.com
David Abrahams wrote:
"Ralf W. Grosse-Kunstleve" <rwgk@yahoo.com> writes:
This is pretty bad because quite a few boost libraries depend on more than only the C names as specified in the standard. (thread and fs at least, IIRC, but I think there are more)
In the hot days of Boost.Python development we were very careful to use only ANSI features. Anything else is non-portable and bound to break somewhere, sooner or later. I.e. I think the libraries that don't compile with -std strict_ansi should be fixed.
pthreads is an extension to what's required by ANSI. I think that might make Boost.Threads impossible.
Well, every boost library that relies on more than ANSI would be affected. That is at least filesystem, thread and test because each of them needs some POSIX feature to work properly. And CXX is quite strict in strict ansi mode. Nothing is there besides the things promised in the standard. For boost.test for example that means, including <signal.h> doesn't give you sigaction, even not if the POSIX feature macros say it's there. So I think as long as the bug in iotraits is not fixed, boost.python will not work on this platform, at least not with the boost.build toolset. :-( Markus
Markus Schöpflin <markus.schoepflin@web.de> writes:
David Abrahams wrote:
"Ralf W. Grosse-Kunstleve" <rwgk@yahoo.com> writes:
This is pretty bad because quite a few boost libraries depend on more than only the C names as specified in the standard. (thread and fs at least, IIRC, but I think there are more)
In the hot days of Boost.Python development we were very careful to use only ANSI features. Anything else is non-portable and bound to break somewhere, sooner or later. I.e. I think the libraries that don't compile with -std strict_ansi should be fixed.
pthreads is an extension to what's required by ANSI. I think that might make Boost.Threads impossible.
Well, every boost library that relies on more than ANSI would be affected. That is at least filesystem, thread and test because each of them needs some POSIX feature to work properly.
And CXX is quite strict in strict ansi mode. Nothing is there besides the things promised in the standard. For boost.test for example that means, including <signal.h> doesn't give you sigaction, even not if the POSIX feature macros say it's there.
So I think as long as the bug in iotraits is not fixed, boost.python will not work on this platform, at least not with the boost.build toolset. :-(
Note: when using Boost.Build you are supposed to #include one of its headers first. That means boost/python/detail/prefix.hpp gets #included before anything else, which gives you an opportunity to play all sorts of nasty tricks like #define ios_traits broken_ios_traits #include <iotraits> #undef ios_traits struct ios_traits // the working version { } to work around brokenness. HTH, -- Dave Abrahams Boost Consulting www.boost-consulting.com
David Abrahams wrote:
Markus Schöpflin <markus.schoepflin@web.de> writes:
So I think as long as the bug in iotraits is not fixed, boost.python will not work on this platform, at least not with the boost.build toolset. :-(
Note: when using Boost.Build you are supposed to #include one of its headers first. That means boost/python/detail/prefix.hpp gets #included before anything else, which gives you an opportunity to play all sorts of nasty tricks like
#define ios_traits broken_ios_traits #include <iotraits> #undef ios_traits
struct ios_traits // the working version {
}
to work around brokenness.
The problem is, there is no working version of iotraits in this case. To summarize: - The <iotraits> header file has broken support for std::fpos_t when it's not an integral value. - The definition of std::fpos_t is normally taken from /usr/include/stdio.h. - std::fpos_t as defined in stdio.h is not an integer value whenever _XOPEN_SOURCE >= 500. pyconfig.h forces the value to 600. - When the compiler option pure_cname is in effect and /usr/include is not on the command line as include path, a different stdio.h is selected, which defines std::fpos_t as an itegral value, ignoring the setting of _XOPEN_SOURCE. The last item is the explanation, why Ralf's setup works, because pure_cname is in effect by default whenever strict_ansi is in effect. Now, the bb toolset cannot use pure_cname because test, fs, threads and maybe others need POSIX features. And I don't see how playing tricks in some boost.python header file could help here. Do you have an idea? TIA, Markus
participants (5)
-
David Abrahams -
Markus Schöpflin -
Markus Schöpflin -
Peter Dimov -
Ralf W. Grosse-Kunstleve