CW 9.3 results (or lack thereof) - current state
After one day and one night of investigation, I think (!) I've finally managed to make bjam report correct results for my regressions. As it seems, the solution is not calling cv_env.bat before running the test. This batch file is also called again before each test case and it seems that the environment gets somewhat confused then: during the execution of the cmd file the errorlevel is still set to 0 after the test case has been executed although the test case definitely reported something different. What I know for sure is that this problem occurs independent of the OS and the machine (see the temporary 'slapeta_experimental' results) and whether the test is started by regression.py or not. The results for CW-9.3 seem to be correct now and thus there should be some more failing test cases for this toolset now. As for my Intel 8.0 results, I hope there are no similar interferences because they don't run without calling iclvars.bat once. I must manually make a testcase fail to verify this (before I fixed the tss cleanup, Intel failed, too, but this wasn't reported, either). That's where I am at the moment. Cheers, Stefan
Stefan Slapeta <stefan@slapeta.com> writes:
After one day and one night of investigation, I think (!) I've finally managed to make bjam report correct results for my regressions.
As it seems, the solution is not calling cv_env.bat before running the test. This batch file is also called again before each test case and it seems that the environment gets somewhat confused then: during the execution of the cmd file the errorlevel is still set to 0 after the test case has been executed although the test case definitely reported something different.
Calling cw_env.bat may be needed in order to establish the PATH needed to find the C++ runtime DLLs. If you eliminated that call, maybe we need to find a way to do the same thing that doesn't mess up the errorlevel. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
Calling cw_env.bat may be needed in order to establish the PATH needed to find the C++ runtime DLLs. If you eliminated that call, maybe we need to find a way to do the same thing that doesn't mess up the errorlevel.
I talked with Stefan off-list and I think I have a solution to the problem. Hopefully it works and I'll commit the change :-) -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com - 102708583/icq
Rene Rivera wrote:
David Abrahams wrote:
Calling cw_env.bat may be needed in order to establish the PATH needed to find the C++ runtime DLLs. If you eliminated that call, maybe we need to find a way to do the same thing that doesn't mess up the errorlevel.
I talked with Stefan off-list and I think I have a solution to the problem. Hopefully it works and I'll commit the change :-)
OK. I believe the problem is now resolved. The problem is a "bug" introduced in the cwenv.bat of CodeWarrior 9. There is now an additional last action this script takes: ======== rem Not sure why we sometimes have errors... set ERRORLEVEL=0 ======== Doing that is just plain wrong. It forever sets the error from that point forward. So any attempt to read the result value of any program run afterwards will result in getting that zero back. And the Windows documentation even warns about this (from >help set)... ======== If Command Extensions are enabled, then there are several dynamic environment variables that can be expanded but which don't show up in the list of variables displayed by SET. These variable values are computed dynamically each time the value of the variable is expanded. If the user explicitly defines a variable with one of these names, then that definition will override the dynamic one described below: ======== So if you are one of the persons with CW9 I would suggest filling a bug for this. To set the problem directly one can use this script: ======== call "cwenv.bat" -quiet set "ERRORLEVEL=" echo =====%ERRORLEVEL%===== dir q:\ >nul 2>nul set status=%ERRORLEVEL% echo =====%status%===== dir c:\ >nul 2>nul set status=%ERRORLEVEL% echo =====%status%===== ======== Which has the workaround for the problem, the second line. Running it will produce: =====0===== =====1===== =====0===== If you remove that second line the output becomes; =====0===== =====0===== =====0===== My suggestion for them to fix it is to use a command that is guaranteed to succeed to clear the ERRORLEVEL like "cd >nul". -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com - 102708583/icq
Rene Rivera <grafik.list@redshift-software.com> writes:
Rene Rivera wrote:
David Abrahams wrote:
Calling cw_env.bat may be needed in order to establish the PATH needed to find the C++ runtime DLLs. If you eliminated that call, maybe we need to find a way to do the same thing that doesn't mess up the errorlevel. I talked with Stefan off-list and I think I have a solution to the problem. Hopefully it works and I'll commit the change :-)
OK. I believe the problem is now resolved. The problem is a "bug" introduced in the cwenv.bat of CodeWarrior 9. There is now an additional last action this script takes:
======== rem Not sure why we sometimes have errors... set ERRORLEVEL=0 ========
Doing that is just plain wrong. It forever sets the error from that point forward.
In some sense this is my fault. I reported to MW that their setup script often exits with an error code despite success, which prevents cwenv.bat && mwcc ... from succeeding. If you konw the right fix, would you please email Ed Swartz (Cc'd here) and let him know?
My suggestion for them to fix it is to use a command that is guaranteed to succeed to clear the ERRORLEVEL like "cd >nul".
Ah. Well, maybe there's no need to notify him, then ;-) Boy, the .bat language sucks. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
Stefan Slapeta writes:
After one day and one night of investigation, I think (!) I've finally managed to make bjam report correct results for my regressions.
As it seems, the solution is not calling cv_env.bat before running the test. This batch file is also called again before each test case and it seems that the environment gets somewhat confused then: during the execution of the cmd file the errorlevel is still set to 0 after the test case has been executed although the test case definitely reported something different.
What I know for sure is that this problem occurs independent of the OS and the machine (see the temporary 'slapeta_experimental' results) and whether the test is started by regression.py or not.
The results for CW-9.3 seem to be correct now and thus there should be some more failing test cases for this toolset now.
As for my Intel 8.0 results, I hope there are no similar interferences because they don't run without calling iclvars.bat once. I must manually make a testcase fail to verify this (before I fixed the tss cleanup, Intel failed, too, but this wasn't reported, either).
That's where I am at the moment.
Stefan, thanks for pursuing this to the successful resolution! -- Aleksey Gurtovoy MetaCommunications Engineering
participants (4)
-
Aleksey Gurtovoy -
David Abrahams -
Rene Rivera -
Stefan Slapeta