Unexpected behaviour of boost::filesystem::equivalent
Hi booster, its quite a long time ago since I could follow the threads here more thoroughly :-( , so the following information might not be new one. During tests of the current Win32 implementation of boost::filesystem::equivalent I observed that the function unexpectingly fails (my test environment was Win XP SP 1 with MSVC7.1) if it compares a "soft link" created via the DOS command "subst" (or its API analogon ::DefineDosDeviceA/W) with the corresponding mapped path object (Lets say I map M: to the existing directory C:\temp). To get an understanding of the test failure I debugged the implementation and found that actually the first test part (getting the handles and comparing the VSN's + further data) gave the expected positive result, but that the very last step, i.e. the comparison of both st_dev values, gave a final unequality. Did others also observe this failure and don't you think that the expected result should show the identity of the underlying path objects? Of course there might be legitimate arguments for the decision for the current behaviour, but I would like to know about them. I am not an expert to argue about the comments in the source code which say that "In theory, volume serial numbers are sufficient to distinguish between devices, but in practice VSN's are sometimes duplicated, so device id is also checked.[..]" The question now is: Should different (MS DOS) device names pointing to the same underlying (Windows NT) device not be compared equivalent? I could think of the following test extension (at least under Windows, regrettably I am not a posix expert to provide a similar solution): If the VSN' and further data have returned a positive result, but the check of both st_dev values fails, one should try to check whether the underlying devices are actually the same, e.g. using the ::QueryDosDeviceA/W API function. What do you think? Greetings, Daniel Krügler
At 02:39 AM 11/22/2004, Daniel Krügler wrote:
its quite a long time ago since I could follow the threads here more thoroughly :-( , so the following information might not be new one.
During tests of the current Win32 implementation of boost::filesystem::equivalent I observed that the function unexpectingly fails (my test environment was Win XP SP 1 with MSVC7.1) if it compares a "soft link" created via the DOS command "subst" (or its API analogon ::DefineDosDeviceA/W) with the corresponding mapped path object (Lets say I map M: to the existing directory C:\temp).
To get an understanding of the test failure I debugged the implementation and found that actually the first test part (getting the handles and comparing the VSN's + further data) gave the expected positive result, but that the very last step, i.e. the comparison of both st_dev values, gave a final unequality.
Ugh! That's ugly.
Did others also observe this failure
AFAIK, it is a new test case.
and don't you think that the expected result should show the identity of the underlying path objects?
Of course there might be legitimate arguments for the decision for the current behaviour, but I would like to know about them. I am not an expert to argue about the comments in the source code which say that
"In theory, volume serial numbers are sufficient to distinguish between devices, but in practice VSN's are sometimes duplicated, so device id is also checked.[..]"
The question now is: Should different (MS DOS) device names pointing to the same underlying (Windows NT) device not be compared equivalent?
I could think of the following test extension (at least under Windows, regrettably I am not a posix expert to provide a similar solution):
If the VSN' and further data have returned a positive result, but the check of both st_dev values fails, one should try to check whether the underlying devices are actually the same, e.g. using the ::QueryDosDeviceA/W API function.
What do you think?
Worth experimenting with. I'll try to dust of the test program and give it a try. Thanks, --Beman
At 02:39 AM 11/22/2004, Daniel Krügler wrote:
Hi booster,
its quite a long time ago since I could follow the threads here more thoroughly :-( , so the following information might not be new one.
During tests of the current Win32 implementation of boost::filesystem::equivalent I observed that the function unexpectingly fails (my test environment was Win XP SP 1 with MSVC7.1) if it compares a "soft link" created via the DOS command "subst" (or its API analogon ::DefineDosDeviceA/W) with the corresponding mapped path object (Lets say I map M: to the existing directory C:\temp). ...
If the VSN' and further data have returned a positive result, but the check of both st_dev values fails, one should try to check whether the underlying devices are actually the same, e.g. using the ::QueryDosDeviceA/W API function.
I couldn't see an easy way to determine the drive needed for calling ::QueryDosDeviceA/W. As an alternative, I removed the st_dev test and added a last_write_time test. Documentation also updated. See current CVS. To make it easier to experiment, I also added in the test directory a program equivalent.cpp. This seems to solve the problem. Please give it a try, and let me know if you are still having any problems. Thanks, --Beman
Does using last_write_time leave an opportunity for false negative because it has been updated in between obtaining the value for each filename? Would using creation_time ( if available ) be safer ? Keith Burton -----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Beman Dawes Sent: 26 November 2004 19:43 To: boost@lists.boost.org; boost@lists.boost.org Subject: Re: [boost] Unexpected behaviour of boost::filesystem::equivalent I couldn't see an easy way to determine the drive needed for calling ::QueryDosDeviceA/W. As an alternative, I removed the st_dev test and added a last_write_time test. Documentation also updated. See current CVS. To make it easier to experiment, I also added in the test directory a program equivalent.cpp. This seems to solve the problem. _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
At 01:42 AM 11/27/2004, Keith Burton wrote:
Does using last_write_time leave an opportunity for false negative because it has been updated in between obtaining the value for each filename?
Holding open the first fir handle until the second has also been processed protects against race conditions caused by defragging. But not against a file size or last write time change. I'm not sure there is anything that can be done about that. Race conditions can often poison filesystem operations; that's just a fact-of-life with most operating systems.
Would using creation_time ( if available ) be safer ?
My feeling was that if a filesystem only supported a single date (as some legacy systems did), then it was likely to be last_write_time. While there are theoretical holes in the current logic, I'm hoping that it provides a practical implementation useful in real-world applications. If users run into practical problems, I hope they will report them here right away. --Beman
Beman Dawes schrieb:
At 02:39 AM 11/22/2004, Daniel Krügler wrote:
Hi booster,
its quite a long time ago since I could follow the threads here more thoroughly :-( , so the following information might not be new one.
During tests of the current Win32 implementation of boost::filesystem::equivalent I observed that the function unexpectingly fails (my test environment was Win XP SP 1 with MSVC7.1) if it compares a "soft link" created via the DOS command "subst" (or its API analogon ::DefineDosDeviceA/W) with the corresponding mapped path object (Lets say I map M: to the existing directory C:\temp). ...
If the VSN' and further data have returned a positive result, but the check of both st_dev values fails, one should try to check whether the underlying devices are actually the same, e.g. using the ::QueryDosDeviceA/W API function.
I couldn't see an easy way to determine the drive needed for calling ::QueryDosDeviceA/W.
As an alternative, I removed the st_dev test and added a last_write_time test. Documentation also updated. See current CVS. To make it easier to experiment, I also added in the test directory a program equivalent.cpp. This seems to solve the problem.
Please give it a try, and let me know if you are still having any problems.
Sorry for the late response Beman! Although I think that the proposed test could be done (I will send you some code in 1/2 days), I will also test your new implementation. Greetings, Daniel
Good morning Beman Dawes, Beman Dawes schrieb:
At 09:28 AM 11/29/2004, Daniel Krügler wrote:
Although I think that the proposed test could be done (I will send you some code in 1/2 days), I will also test your new implementation.
No hurry; I'm leaving tomorrow for a week of vacation (and not taking a computer either!)
Your implementation works fine, although I wonder how it interacts with the original assertions made in the code (to which I can't give definite statements, but they seemed to suggest that the new implementation should not be sufficient). Greetings, Daniel
At 05:48 AM 12/6/2004, Daniel Krügler wrote:
Your implementation works fine, although I wonder how it interacts with the original assertions made in the code (to which I can't give definite statements, but they seemed to suggest that the new implementation should not be sufficient).
There continues to be concern on Windows, particularly where the media isn't a hard disk but is something like a CD-ROM, floppy, or memory card which has been formatted and duplicated by a program which ignores the specification and intent of volume serial number and last write time, and possibly file name. It seems a shame to give up on equivalence testing because of such an extreme corner case. Any feedback about situations actually encountered by users will be appreciated. Thanks, --Beman PS: Other operating systems besides Windows may have similar issues; volume serial numbers (or equivalent) were common on batch processing systems that had to cope with a large number of mounts per second (due to automated tape library systems.)
participants (3)
-
Beman Dawes -
Daniel Krügler -
Keith Burton