[filesystem] Should recursive_directory_iterator follow directory symlinks by default?

recursive_directory_iterator currently always follows directory symlinks It seems to me that whether or not this behavior is desirable is application dependent. Thus I'd like to add an option to either following directory symlinks or not. Which should the default be? Following symlinks by default avoids breaking existing code, so that's my mild preference, but I'm wondering if anyone has a strong argument for not following directory symlinks by default? --Beman PS: Windows identifies symlinks to directories, so it is possible to have a dangling symlink to a non-existent directory. recursive_directory_iterator currently tries to recurse into the non-existent directory. That's just a bug and I'm about to fix it.

At Wed, 9 Feb 2011 09:15:24 -0500, Beman Dawes wrote:
recursive_directory_iterator currently always follows directory symlinks
It seems to me that whether or not this behavior is desirable is application dependent.
Thus I'd like to add an option to either following directory symlinks or not. Which should the default be?
Following symlinks by default avoids breaking existing code, so that's my mild preference, but I'm wondering if anyone has a strong argument for not following directory symlinks by default?
It might take extra logic on the user's part to avoid infinite recursions in a case where a directory symlinks an ancestor, so maybe it would make sense to have symlink-following be an explicit choice. -- Dave Abrahams BoostPro Computing http://www.boostpro.com

On Wed, 2011-02-09 at 11:29 -0500, Dave Abrahams wrote:
At Wed, 9 Feb 2011 09:15:24 -0500, Beman Dawes wrote:
recursive_directory_iterator currently always follows directory symlinks [...] Thus I'd like to add an option to either following directory symlinks or not. Which should the default be? It might take extra logic on the user's part to avoid infinite recursions in a case where a directory symlinks an ancestor, so maybe it would make sense to have symlink-following be an explicit choice.
I agree with Dave. Existing code *might* have been written with all the code that checks for loops and escaping from a local tree. But... Having written code using the recursive_directory_iterator (on Windows XP), I know for a fact that I have never taken account of the fact that it might happen because on that platform: it just isn't an issue that is ever going to occur. On *nix, my mindset has been coloured by the default behaviour of commands that process directories: they don't follow symlinks unless you tell them to, so I've implicitly assumed (incorrectly) that's what any recursive directory iterator would do. Coding to safely handle automatically following of symlinks on a recursive directory scan is sufficiently annoying that I'd suggest: (a) most people haven't done it (properly); and (b) adding an extra flag when it is to be done is a good reminder that the code is going to need to be more complex. As long as the change is documented in the release notes, I'd have thought that would be sufficient. Phil -- Phil Richards, <news@derived-software.ltd.uk>

On Sat, Feb 12, 2011 at 3:37 AM, Phil Richards <news@derived-software.ltd.uk> wrote: ...>
Coding to safely handle automatically following of symlinks on a recursive directory scan is sufficiently annoying that I'd suggest: (a) most people haven't done it (properly); and (b) adding an extra flag when it is to be done is a good reminder that the code is going to need to be more complex.
Good points. They further reinforce the decision to make not following symlinks the default. Thanks, --Beman

Beman Dawes wrote:
recursive_directory_iterator currently always follows directory symlinks
It seems to me that whether or not this behavior is desirable is application dependent.
+1
Thus I'd like to add an option to either following directory symlinks or not. Which should the default be?
*nix follows symlinks by default. One must use special APIs to recognize and inspect symlinks. I should think consistency across platforms would be good. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

On 02/09/2011 08:43 AM, Stewart, Robert wrote:
Beman Dawes wrote:
recursive_directory_iterator currently always follows directory symlinks
It seems to me that whether or not this behavior is desirable is application dependent.
+1
Thus I'd like to add an option to either following directory symlinks or not. Which should the default be?
*nix follows symlinks by default. One must use special APIs to recognize and inspect symlinks. I should think consistency across platforms would be good.
That isn't really true. There isn't any POSIX function for recursive directory walking. It is simply a matter of whether lstat or stat is used to check whether directory entries are themselves directories. As one example, GNU find does not follow symlinks by default.

Thus I'd like to add an option to either following directory symlinks or not. Which should the default be?
*nix follows symlinks by default. One must use special APIs to recognize and inspect symlinks. I should think consistency across platforms would be good.
That isn't really true. There isn't any POSIX function for recursive directory walking. It is simply a matter of whether lstat or stat is used to check whether directory entries are themselves directories. As one example, GNU find does not follow symlinks by default.
And also following symlinks requires special handling to prevent going round and round in circles in the pathological cases.... which is not to say the option of a symlink following API wouldn't be useful. John.

On Wed, Feb 9, 2011 at 12:54 PM, John Maddock <boost.regex@virgin.net> wrote:
Thus I'd like to add an option to either following directory symlinks or not. Which should the default be?
*nix follows symlinks by default. One must use special APIs to recognize and inspect symlinks. I should think consistency across platforms would be good.
That isn't really true. There isn't any POSIX function for recursive directory walking. It is simply a matter of whether lstat or stat is used to check whether directory entries are themselves directories. As one example, GNU find does not follow symlinks by default.
And also following symlinks requires special handling to prevent going round and round in circles in the pathological cases.... which is not to say the option of a symlink following API wouldn't be useful.
I'm convinced. In the long run, it is [slightly] safer for the default to not follow directory symlinks. So that will become the default. Thanks to everyone who commented, --Beman

Jeremy Maitin-Shepard wrote:
On 02/09/2011 08:43 AM, Stewart, Robert wrote:
Beman Dawes wrote:
I'd like to add an option to either following directory symlinks or not. Which should the default be?
*nix follows symlinks by default. One must use special APIs to recognize and inspect symlinks. I should think consistency across platforms would be good.
That isn't really true. There isn't any POSIX function for recursive directory walking. It is simply a matter of whether lstat or stat is used to check whether directory entries are themselves directories. As one example, GNU find does not follow symlinks by default.
Right, but I meant that normal file APIs follow symlinks, so doing so during directory iteration would seem consistent. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
participants (6)
-
Beman Dawes
-
Dave Abrahams
-
Jeremy Maitin-Shepard
-
John Maddock
-
Phil Richards
-
Stewart, Robert