[filesystem] Apparent non-compliant code in current operations.hpp, ...

While building 1_34_1 on various platforms we support, I came across some compile errors for the AIX compiler, xlC/vacpp that weren't caught by gcc or MSVC, possibly due to being hidden by a level of indirection. The errors were mostly undefined classes where the template class was fully defined at the time of instantiation. I apologize if my explanation isn't clear or totally correct - I'm not an expert on the C++ standard, and got a lot of the actual explanation from coworkers that know it better than I do. But it's up to me to do the actual reporting and to try to get it to work for us on all the platforms we support. My original post was a few days ago, with the subject "[filesystem] vacpp errors building 1_34_1 filesystem library on AIX". I am because it may look like a vacpp problem, which is an unsupported compiler, but it's not really. It just happens that xlC was the only compiler that caught the error, I think. My original post can also be read at this link: http://permalink.gmane.org/gmane.comp.lib.boost.user/31028 The problem is that the class basic_directory_iterator is not fully defined until after it is used by remove_all(). There were a few other similar errors which I can provide if someone takes an interest in this. If I am wrong, or if there is a reason this won't be fixed, it would be helpful to know, so we can decide what to do. -- Marty Fried Senior Software Developer, Aldon USA http://www.aldon.com

Marty Fried wrote: [snip compilation problem with filesystem] 1) Could you try if it works with the trunk version? There have been a few fixes in this area, but I don't know if those will help you. 2) Filesystem has passed the tests for 1.34.1 with cxx on Tru64 in strict ansi mode, which is usually good at finding this kind of errors. So could you please post the minimal test case you found, this allows me to check if cxx should have found the problem, and if yes, why I didn't find it. 3) If you have a fix and are confident that it works, open a ticket on the boost trac system, attaching a patch, preferably against the trunk version. If you don't have a fix but are confident that it is an error, open a ticket anyway. ;-) 4) Which front-end does xlC use? EDG? HTH, Markus

Back on Thu, 18 Oct 2007 09:53:36 +0200, in gmane.comp.lib.boost.user,Markus
Schöpflin
Marty Fried wrote:
[snip compilation problem with filesystem]
1) Could you try if it works with the trunk version? There have been a few fixes in this area, but I don't know if those will help you.
2) Filesystem has passed the tests for 1.34.1 with cxx on Tru64 in strict ansi mode, which is usually good at finding this kind of errors. So could you please post the minimal test case you found, this allows me to check if cxx should have found the problem, and if yes, why I didn't find it.
3) If you have a fix and are confident that it works, open a ticket on the boost trac system, attaching a patch, preferably against the trunk version. If you don't have a fix but are confident that it is an error, open a ticket anyway. ;-)
4) Which front-end does xlC use? EDG?
Thanks Markus, 1. I had looked over the trunk version, and compared it with the existing 1.34.1 version. I didn't see anything that looked like it was remotely related to this problem. If you insist, I can try to pull down the files; I don't know offhand how to do this correctly, but I imagine I can figure out a way to do it. 2. My coworker came up with this minimal test case. It has one level of indirection rather than 2 like filesystem, because that's all that's required to exhibit the problem. Without the one level of indirection, MSVC and GCC both also display the error. As I mentioned in my original post, this sample was fed into a web form on the Comeau site, which will check the code for compliancy with the standard. It's the best we have available for testing. As I understand it, the mechanism is that the real function remove_all() (not the template function) forces instantiation of the template function remove_all(), which in turn forces the instantiation of the template function remove_all_aux(), which in turn forces the instantiation of basic_directory_iterator, before its definition later in the file. I think the test case is small enough to post in this message, so here it is: /******** Test case ********/ template <typename T> class foo; template <typename T> inline void templatefunc() { foo<int> f; } inline void func() { templatefunc<int>(); } template <typename T> class foo { }; /******** Results from Comeau compiler *********/ Comeau C/C++ 4.3.9 (Mar 27 2007 17:24:47) for ONLINE_EVALUATION_BETA1 Copyright 1988-2007 Comeau Computing. All rights reserved. MODE:strict errors C++ C++0x_extensions "ComeauTest.c", line 6: error: incomplete type is not allowed foo<int> f; ^ "ComeauTest.c", line 6: warning: variable "f" was declared but never referenced foo<int> f; ^ 1 error detected in the compilation of "ComeauTest.c". /******** End test case ********/ 3. Our fix would most likely be to compile the filesystem library with less strict compatibility checking. I'm not envolved at all with the standards process, so I can't really know for sure how confident I should be that this is an error. I've heard that Comeau C++ was used to find error in other compilers, like gcpp, so that is the main reason I think it's an error. 4. We use xlC_r, the reentrant version of xlC (xlc++). I don't really know what EDG is, but I'm pretty sure we don't use it. FWIW, we currently have version 8, but will try to get version 9, which was recently released. But I suspect that would only be more strict rather than less so. Please let me know if you need more information. -- Marty Fried Senior software engineer Aldon http://www.aldon.com

Note for boost.devel readers: The original problem report can be found at http://permalink.gmane.org/gmane.comp.lib.boost.user/31028. Currently I think the reported error is really a problem in boost code, if the reproducer below is indeed correct. But I'm still puzzled why cxx in strict ansi mode isn't complaining. Did anyone encounter this problem already? Note to original poster: See below for answers. Marty Fried wrote:
Back on Thu, 18 Oct 2007 09:53:36 +0200, in gmane.comp.lib.boost.user,Markus Schöpflin
wrote: Marty Fried wrote:
[snip compilation problem with filesystem]
1) Could you try if it works with the trunk version? There have been a few fixes in this area, but I don't know if those will help you.
2) Filesystem has passed the tests for 1.34.1 with cxx on Tru64 in strict ansi mode, which is usually good at finding this kind of errors. So could you please post the minimal test case you found, this allows me to check if cxx should have found the problem, and if yes, why I didn't find it.
3) If you have a fix and are confident that it works, open a ticket on the boost trac system, attaching a patch, preferably against the trunk version.. If you don't have a fix but are confident that it is an error, open a ticket anyway. ;-)
4) Which front-end does xlC use? EDG?
Thanks Markus,
1. I had looked over the trunk version, and compared it with the existing 1.34.1 version. I didn't see anything that looked like it was remotely related to this problem. If you insist, I can try to pull down the files; I don't know offhand how to do this correctly, but I imagine I can figure out a way to do it.
No worry, I just remembered that we have regression test runs for xlC on the trunk. See http://beta.boost.org/development/tests/trunk/developer/summary.html for details on the state of xlC. Looks like xlC has its problems with boost code.
2. My coworker came up with this minimal test case. It has one level of indirection rather than 2 like filesystem, because that's all that's required to exhibit the problem. Without the one level of indirection, MSVC and GCC both also display the error. As I mentioned in my original post, this sample was fed into a web form on the Comeau site, which will check the code for compliancy with the standard. It's the best we have available for testing. As I understand it, the mechanism is that the real function remove_all() (not the template function) forces instantiation of the template function remove_all(), which in turn forces the instantiation of the template function remove_all_aux(), which in turn forces the instantiation of basic_directory_iterator, before its definition later in the file.
I think the test case is small enough to post in this message, so here it is: /******** Test case ********/
template <typename T> class foo;
template <typename T> inline void templatefunc() { foo<int> f; }
inline void func() { templatefunc<int>(); }
template <typename T> class foo { };
/******** Results from Comeau compiler *********/
Comeau C/C++ 4.3.9 (Mar 27 2007 17:24:47) for ONLINE_EVALUATION_BETA1 Copyright 1988-2007 Comeau Computing. All rights reserved. MODE:strict errors C++ C++0x_extensions
"ComeauTest.c", line 6: error: incomplete type is not allowed foo<int> f; ^
"ComeauTest.c", line 6: warning: variable "f" was declared but never referenced foo<int> f; ^
1 error detected in the compilation of "ComeauTest.c".
/******** End test case ********/
Hmm, cxx has this to say: cxx: Error: foo.cxx, line 6: Cannot create variable "f" of incomplete type "foo<int>". detected during instantiation of "void templatefunc<T>() [with T=int]" at line 11 foo<int> f; ------------^ Now I'm left wondering why the tests on Tru64 don't show this error. Just to be sure, do you get the error while compiling the filesystem lib itself or while using the compiled filesystem lib? If you get the error during compilation of the library, are you sure that above code is a valid reproducer for the original problem? (I'm not implying that it isn't, I just want to make sure it is.)
3. Our fix would most likely be to compile the filesystem library with less strict compatibility checking. I'm not envolved at all with the standards process, so I can't really know for sure how confident I should be that this is an error. I've heard that Comeau C++ was used to find error in other compilers, like gcpp, so that is the main reason I think it's an error.
Well, I usually use Comeau to back up my claims when reporting an error, like you do. I'll cross post this to the developer list, maybe someone can confirm that this is indeed an error.
4. We use xlC_r, the reentrant version of xlC (xlc++). I don't really know what EDG is, but I'm pretty sure we don't use it. FWIW, we currently have version 8, but will try to get version 9, which was recently released. But I suspect that would only be more strict rather than less so.
EDG is a compiler front end, see http://www.edg.com/index.php?location=c_frontend for more details. But it seems xlC isn't using EDG. Markus

Back on Fri, 19 Oct 2007 11:37:11 +0200, in gmane.comp.lib.boost.user,Markus
Schöpflin
Note for boost.devel readers: The original problem report can be found at http://permalink.gmane.org/gmane.comp.lib.boost.user/31028.
Currently I think the reported error is really a problem in boost code, if the reproducer below is indeed correct. But I'm still puzzled why cxx in strict ansi mode isn't complaining.
Did anyone encounter this problem already?
Note to original poster: See below for answers.
Marty Fried wrote:
Back on Thu, 18 Oct 2007 09:53:36 +0200, in gmane.comp.lib.boost.user,Markus Schöpflin
wrote: Marty Fried wrote:
[snip compilation problem with filesystem]
1) Could you try if it works with the trunk version? There have been a few fixes in this area, but I don't know if those will help you.
2) Filesystem has passed the tests for 1.34.1 with cxx on Tru64 in strict ansi mode, which is usually good at finding this kind of errors. So could you please post the minimal test case you found, this allows me to check if cxx should have found the problem, and if yes, why I didn't find it.
Thanks Markus,
1. I had looked over the trunk version, and compared it with the existing 1.34.1 version. I didn't see anything that looked like it was remotely related to this problem.
No worry, I just remembered that we have regression test runs for xlC on the trunk. See http://beta.boost.org/development/tests/trunk/developer/summary.html for details on the state of xlC. Looks like xlC has its problems with boost code.
I seems to me that something strange is going on; I'm not able to even compile the filesystem library without errors. It seems like the regression tests succeeded with the compile and link, if I'm reading it correctly. Could it be that the test is done with an older version of xlC, that may not have checked for this condidtion? We are using version 8; version 9 recently came out. I want to go back to aix and try the exact command line listed for the vacpp test output, to see what result I get. I'll report back.
Hmm, cxx has this to say:
cxx: Error: foo.cxx, line 6: Cannot create variable "f" of incomplete type "foo<int>". detected during instantiation of "void templatefunc<T>() [with T=int]" at line 11 foo<int> f; ------------^
Now I'm left wondering why the tests on Tru64 don't show this error. Just to be sure, do you get the error while compiling the filesystem lib itself or while using the compiled filesystem lib?
If you get the error during compilation of the library, are you sure that above code is a valid reproducer for the original problem? (I'm not implying that it isn't, I just want to make sure it is.)
I believe the test case is valid, but there is one difference that may matter here... the test case has one less level of indirection than the actual code. We didn't think it made a difference, and we wanted to simplify as much as possible; the test showed that the error is caught by xlC and Comeau, but not by gcc or msvc. However, it could be that one more level of indirection is needed to hide the error from Tru64, too. In other words, maybe Tru64 is better than gcc and MSVC at finding this test case error, but not as good as xlC/Comeau with one more level of indirection to mask it. Would you like me to modify the test case? I appreciate your looking into this - I was worried you might dismiss it as a problem with an unsupported compiler - not that I think it really is that simple. Assuming it's really an error, it will probably eventually become a problem with a supported compiler. Please let me know what I can do next to help out. -- Marty Fried Senior software engineer Aldon http://www.aldon.com

Marty Fried wrote:
Back on Fri, 19 Oct 2007 11:37:11 +0200, in gmane.comp.lib.boost.user,Markus Schöpflin
wrote: ...
No worry, I just remembered that we have regression test runs for xlC on the trunk. See http://beta.boost.org/development/tests/trunk/developer/summary.html for details on the state of xlC. Looks like xlC has its problems with boost code.
I seems to me that something strange is going on; I'm not able to even compile the filesystem library without errors. It seems like the regression tests succeeded with the compile and link, if I'm reading it correctly. Could it be that the test is done with an older version of xlC, that may not have checked for this condidtion? We are using version 8; version 9 recently came out.
You can tell what version of a compiler is being used by looking at the config library's conf_info output. For the current regression results, it's reporting: IBM Visual Age version 900 _CHAR_UNSIGNED =1 _CPPUNWIND =1 __cplusplus =199711L __STDC__ =0 _WCHAR_T =1 __CHAR_UNSIGNED__ =1 __EXCEPTIONS =1 __powerpc__ =1 _WCHAR_T =1 __IBMCPP__ =900
I want to go back to aix and try the exact command line listed for the vacpp test output, to see what result I get. I'll report back.
Another aspect is that our tests are running on the code as it currently stands, not on the 1.34.1 code.
Hmm, cxx has this to say:
cxx: Error: foo.cxx, line 6: Cannot create variable "f" of incomplete type "foo<int>". detected during instantiation of "void templatefunc<T>() [with T=int]" at line 11 foo<int> f; ------------^
Now I'm left wondering why the tests on Tru64 don't show this error. Just to be sure, do you get the error while compiling the filesystem lib itself or while using the compiled filesystem lib?
If you get the error during compilation of the library, are you sure that above code is a valid reproducer for the original problem? (I'm not implying that it isn't, I just want to make sure it is.)
I believe the test case is valid, but there is one difference that may matter here... the test case has one less level of indirection than the actual code. We didn't think it made a difference, and we wanted to simplify as much as possible; the test showed that the error is caught by xlC and Comeau, but not by gcc or msvc. However, it could be that one more level of indirection is needed to hide the error from Tru64, too. In other words, maybe Tru64 is better than gcc and MSVC at finding this test case error, but not as good as xlC/Comeau with one more level of indirection to mask it. Would you like me to modify the test case?
I appreciate your looking into this - I was worried you might dismiss it as a problem with an unsupported compiler - not that I think it really is that simple. Assuming it's really an error, it will probably eventually become a problem with a supported compiler.
If the code isn't standard compliant, I'd like to fix it. But it looks like version 9 of vacpp is accepting it, or possible the code has changed enough that the problem no longer exists.
Please let me know what I can do next to help out.
If you get some sample code that you think exactly reproduces the problem, and Comeau reports an error, please send it to me. Thanks, --Beman

Back on Fri, 19 Oct 2007 14:22:17 -0400, in gmane.comp.lib.boost.user,Beman
Dawes
Marty Fried wrote:
I seems to me that something strange is going on; I'm not able to even compile the filesystem library without errors. It seems like the regression tests succeeded with the compile and link, if I'm reading it correctly. Could it be that the test is done with an older version of xlC, that may not have checked for this condidtion? We are using version 8; version 9 recently came out.
Another aspect is that our tests are running on the code as it currently stands, not on the 1.34.1 code.
If the code isn't standard compliant, I'd like to fix it. But it looks like version 9 of vacpp is accepting it, or possible the code has changed enough that the problem no longer exists.
I really don't think the code has changed in a way to fix this problem, although I haven't been able to build the trunk version. It came down as CR/LF line endings for my windows box, but there is only the shell script, not the batch file, for building, and the shell scripts don't work with CR/LF. So, I need to translate to LF only to be able to build.
If you get some sample code that you think exactly reproduces the problem, and Comeau reports an error, please send it to me.
I can do that; I think my coworker originally did have a more exact sample, but took out the outer level of indirection to make it simpler. I'll see about putting it back, and renaming the functions to the same names as the filesystem uses, rather than foo, etc. :) By the way, are my messages formatted OK? I'm using a newsreader instead of email, and I'm seeing a lot of garbage sometimes - I'm not sure if it's my posts, other posts, or my reader. I could attach the sample code rather than just pasting it in the message if it's not clear (assuming that attachments are acceptable). -- Marty Fried Senior software engineer Aldon http://www.aldon.com

Back on Fri, 19 Oct 2007 14:22:17 -0400, in gmane.comp.lib.boost.user,Beman
Dawes
If the code isn't standard compliant, I'd like to fix it. But it looks like version 9 of vacpp is accepting it, or possible the code has changed enough that the problem no longer exists.
Please let me know what I can do next to help out.
If you get some sample code that you think exactly reproduces the problem, and Comeau reports an error, please send it to me.
Hi, Sorry I didn't get back to you sooner... my coworker that did the original tests could not come up with a good sample very easily, so we postponed it until we got a copy of the latest xLC installed. With the latest version, the Boost code compiles without errors, so I am guessing that the compiler was either incorrect, or the standard has changed in this respect. All the code now compiles on AIX, except, of course, we can't use the "install" option for some reason. Thanks for everyone's help. -- Marty Fried Senior software engineer Aldon http://www.aldon.com
participants (3)
-
Beman Dawes
-
Markus Schöpflin
-
Marty Fried