Marsh Ray wrote:

On 10/18/2010 09:58 AM, Jeff Flinn wrote:

...

In looking at implementing a portable home_directory_path function a few questions arise.

Based on http://en.wikipedia.org/wiki/Home_directory, POSIX and other UNIX flavors 'appear' to be a straight forward use of the "HOME" environment variable. Does anyone have any experience where the "HOME" variable is not the proper source of the user's home directory?

Windows NT and above have the "USERPROFILE" environment variable. Using this would be the easiest to use. But I've seen discussion that this may not always be properly defined, in particular if the user has reset their home directory to a non-system drive.

I wouldn't count on it for anything critical.

Can you expand on that? What problems have you encountered.

...

I haven't tracked down it's availability on WinCE. Does anyone have experience with this platform?

There is the windows GetUserProfileDirectory api function also available since NT4.

That's a the call to use if indeed you want "the root directory of the specified user's profile". http://msdn.microsoft.com/en-us/library/bb762280.aspx

_But it's important to be prepared for case where the code is executing as a user who does not actually have a profile on the machine!_

Of course. home_directory_path will have the same error capabilities as the recently added temp_directory_path function

You might be better off calling SHGetFolderPath with a CSIDL value like CSIDL_APPDATA. http://msdn.microsoft.com/en-us/library/bb762181.aspx http://msdn.microsoft.com/en-us/library/bb762494.aspx

...

This is in the Userenv.lib/dll, obviously adding a link dependancy. How are these sorts of system link dependencies handled? Through bjam? Or is it acceptable/better to use #pragma comment(lib,"Userenv.lib")?

Ah, SHGetFolderPath uses shell32.dll, but doesn't require and import library. I'll take a look at using this with CSIDL_PROFILE which corresponds to the GetUserProfileDirectory function.

...

In looking at some client code using home_directory_path, I see that they generally are looking for the user's Desktop directory(Mac & Windows). Obviously this only makes sense on GUI platforms. The issue with merely getting this path via home_directory_path() / "Desktop" is IIRC, that at least on windows "Desktop" is localized and may not exist with that name on non-english localized systems. Windows has the SHGetFolderPath api. I need to look into Mac's native support. Any thoughts on the general usefulness of a portable desktop_directory_path?

It seems to me like if you're going to work with a "Desktop", you're going to need a variety of platform specific logic to deal with things like file types, icons, app associations, etc. You'd need a real platform UI layer. So I don't think a single function to just find the path is likely to be useful to many.

Well, in our case it's a well known location where users have installed other (legacy)apps that we need to communicate with. Nothing to do with icons, file types,... Thanks, Jeff

Christian Holmquist wrote:

...

You might be better off calling SHGetFolderPath with a CSIDL value like

...

CSIDL_APPDATA. http://msdn.microsoft.com/en-us/library/bb762181.aspx http://msdn.microsoft.com/en-us/library/bb762494.aspx

I think CSIDL_APPDATA is not the correct flag, we're using the following args to SHGetFolderPath, and it appears to work

char path[MAX_PATH]; bool success = SUCCEEDED(SHGetFolderPath(NULL, CSIDL_PERSONAL | CSIDL_FLAG_CREATE, NULL, 0, path))));

CSIDL_PERSONAL corresponds to the user's 'My Documents' folder, whereas CSIDL_PROFILE corresponds to the user's profile directory which is more consistent with the POSIX 'HOME' directory. Also consensus from discussion on temp_directory_path was that the library should *not* create a directory if one doesn't exist. Hmm, the other drawback of this function is the lack of ability to get the buffer size, and no documented error code corresponding to insufficient buffer size. Is it guaranteed that the buffer size = MAX_PATH is always sufficient?

..and I think this would be a most useful addition to boost::filesystem.

Me too. I've needed this in projects for several different employers over the years. Jeff

Christian Holmquist wrote:

Ok, I've always thought of My Documents as the HOME directory on Windows. At my current location, My Documents points to: \\file01\private\chrhol\ while the environment variable "USERPROFILE" is set to C:\Documents And Settings\chrhol. The fact that some stuff is saved to local C:\ drive is to me annoying, I never fully understood this..

Is there a POSIX equivalent for My Documents?

Given that My Documents is nothing more than a common directory into or under which to deposit all user-produced content, it's only special in that Microsoft thought Windows users were too silly to decide for themselves where and how to organize their files. POSIX systems don't make that assumption, so there is no equivalent. Having said that, it is not uncommon to have a directory dedicated as a default location for files that don't have a more specific place. However, the choice of directory is user-specific.

enum os_path_t { path_temp, path_home, path_windows_my_documents, // Or is this too ugly? };

namespace filesystem { path get_os_path(os_path_t) }

os_path might not the best name, but you get the idea..

I was thinking of something similar: indicate which directory is wanted by a discriminator. As for names, I'd definitely avoid "windows." In your example, "path_windows_my_documents" could be "path_documents" and still serve just as well. Given that there is no specific My Documents equivalent on POSIX systems and no specific home directory on Windows, perhaps the better approach is to have calls for those throw an exception unless the application first defines their location. That is, Filesystem could offer a standard API for accessing such paths but require the actual path be specified -- once per run -- when there's no OS default. The advantage is that most code can use the common API and only initialization code must set the ambiguous paths according to some local policy and the current platform. Therefore: namespace filesystem { path set_os_path(os_path_t, path const &); path set_os_path_from_environment(os_path_t, std::string const &); } (It returns the previous path, if any.) What's missing is the ability to set a policy that indicates Filesystem should try some particular e-var and, if that's not set, check for a particular directory, and if that doesn't exist, throw an exception. That would be the most useful. Perhaps the right behavior is to simply install a function that Filesystem can call to provide the path when needed. It may be handy to provide a feature query function, too: namespace filesystem { bool is_os_path_available(os_path_t); } If get_os_path() calls a user-installed function, then is_os_path_available() would have to call the former in a try block, of course. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

Christian Holmquist wrote:

...

namespace filesystem { path set_os_path(os_path_t, path const &);

path set_os_path_from_environment(os_path_t, std::string const &); }

This I like, but do you need set_os_path_from_environment?

Maybe. I was thinking that set_os_path() might retain the path and upon query, via get_os_path(), validate the directory. Thus, the directory need not exist when set_os_path() is called; just when get_os_path() is called. (Filesystem could cache whether the path was validated to avoid doing so on each query.) Likewise, I was thinking that set_os_path_from_environment() might save the e-var and leave to get_os_path() the job of accessing the e-var and, if it exists, validating the pathname given by it. Thus, the program could modify the environment or create the "OS directory" in question after having called set_os_path*(). Is such laziness really needed? No. Would it be helpful or convenient? Perhaps. It was just a passing thought when I added set_os_path_from_environment(). However, to accommodate such behavior, the names really shouldn't begin with "set_os_path" as they wouldn't set the path so much as the policy for finding it.

set_os_path(X, boost::getenv("Y")) would suffice IMO, surely boost must somewhere have a string getenv(string) (?)

If that's all that set_os_path_from_environment() entails, I quite agree with you. If it were to do more, as suggested above, then no.

...

What's missing is the ability to set a policy that indicates Filesystem should try some particular e-var and, if that's not set, check for a particular directory, and if that doesn't exist, throw an exception. That would be the most useful. Perhaps the right behavior is to simply install a function that Filesystem can call to provide the path when needed.

with set_os_path I think users can do that themselves.

Sure. The reason to put such a thing in the library is just to remove the burden from the clients that would avail themselves of that behavior. Are there enough to justify it? I don't know.

Installing a callback function seems a little over-engineered, what would be the benefits?

That would allow initialization code to establish the policy in one place while querying code elsewhere benefits. Filesystem can just store a path against each possible query and leave all else to the client via an eager set_os_path(). That would, of course, require that the initialization code first create missing directories, etc., and then call set_os_path(). Delaying the work, via a callback, means that the reaction to missing directories or e-vars can be handled or reported using an otherwise fully initialized application, so the handling can, for example, make use of GUI facilities that might not be available during initialization. Again, I don't know whether that will prove useful to anyone, but that was the thought that passed through my head when I suggested the idea.

...

It may be handy to provide a feature query function, too:

namespace filesystem { bool is_os_path_available(os_path_t); }

If get_os_path() calls a user-installed function, then is_os_path_available() would have to call the former in a try block, of course.

I thought you were aiming at having filesystem configured at initialization with set_os_path.

There are other views of what initialization may do.

get_os_path() should IMO throw if there's no platform specific implementation for the discriminator and the user has not explicitly set the path.

Right, but the means of setting the path could be lazy or eager. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

Chad Nelson wrote:

"Stewart, Robert" <Robert.Stewart@sig.com> wrote:

...

Christian Holmquist wrote:

...

Given that My Documents is nothing more than a common directory into or under which to deposit all user-produced content, it's only special in that Microsoft thought Windows users were too silly to decide for themselves where and how to organize their files. [...]

Having seen how many (non-technical) people just dump all their files on their computer's desktop, I'm afraid I have to side with Microsoft on that one. :-)

I know, there are a great many such people. Yes, I think Microsoft was right to do that for their general population, but Filesystem cannot choose similarly for all clients.

...

Given that there is no specific My Documents equivalent on POSIX systems and no specific home directory on Windows, perhaps the better approach is to have calls for those throw an exception unless the application first defines their location. [...]

You said yourself, above, that My Documents on Windows is intended to be the repository of user-generated content. (As opposed to the "USERPROFILE" directory, which is presumably for program-generated files such as configuration files.) On *NIX systems, the user's home directory, or more often a user-selected subdirectory of it, serves the same purpose. So as Bjørn Roald said in another reply, why not just return the user's home directory for the document path?

Anything close to an equivalent on POSIX systems is likely to be a user-selected subdirectory of the user's home directory, so choosing the home directory is wrong. Indeed, the home directory should have little but subdirectories and "dot files," just as a Windows system's C:\ should have little more than subdirectories, choosing the home directory as the default location to dump documents is wrongheaded. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

Chad Nelson wrote:

Automatically saving user-generated content into it is wrong. Returning it from that function might not be, if there's no better logical candidate.

That distinction is interesting, but is it necessary that the function have a default on POSIX systems? We've suggested ideas on how that could be set by the client, according to some local policy without Filesystem having to impose an opinion on what is most appropriate.

Since the requested directory *is* explicitly for user-generated content, presumably most programs would use the returned path to pop up a load/save dialog to the user (who would then manually navigate to his preferred subdirectory for his documents), or would append their own subdirectory names to it before silently saving files.

Saving in or under My Documents is not so unlike saving in or under one's home directory. Perhaps I'm too stridently against content being put into the home directory thus not wanting to encourage that by making it the default.

So long as the behavior is documented, I don't think it would be a problem. As a developer it's what I'd want, because if I had to write OS-specific changes to my program to make it work, it would be a lot less useful to me.

Your notion of an app saving the user's last directory is reasonable, meaning that the My Documents directory, or its equivalent, would be used only when there is no user-driven default available. Since the home directory is at least the parent of the directory in which the user would want to put a file, its use in that scenario is not unreasonable. If Filesystem offered a means to override the path returned by home_directory_path() (or however it's spelled), then those that prefer something else can set it and those that prefer the home directory can leave the default. BTW, it would be better to refer to the result as the "documents directory" instead of the home directory since there isn't a good notion of home directory on Windows. Thus, documents_directory_path() or get_os_path(path_documents) (or something along those lines). _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

On Wed, Oct 20, 2010 at 6:44 PM, Chad Nelson <chad.thecomfychair@gmail.com> wrote:

On Wed, 20 Oct 2010 14:45:55 -0400 "Stewart, Robert" <Robert.Stewart@sig.com> wrote: wholeheartedly with your position. But the majority of Linux users are still power users, at the very least. If they save something in their home directory, it's because they explicitly want it there.

This isn't meant as a disagreement, since you do use "majority". However, not all Linux users are power users. My wife uses Linux, and I doubt she could tell you the path to her $HOME much less what it's supposed to be for. I have a friend who is a complete idiot, and he uses Linux.

...

I have no preference for either enum based or multiple function based solution, but I am curious to what the benifit of an enum based solution is.

--

I prefer to use enumerations when the interface allows, cause one can add string operators to them for debugging purposes and keep them around more easily in user-defined structs. Exceptions thrown from get_os_path() might include the discriminator, making things a little easier at the catch site.. namespace example { struct user_file_path { filesystem::os_path_t root; filesystem::path file_path; filesystem::path get() { return filesystem::get_os_path(root) / file_path; } }; void foo() { user_file_path p = {..., ....}; try { p.get(): } catch(const bad_os_path& p) { clog << "failed to get os path " << p.os_path(); } } } Christian

Bjørn Roald wrote:

...

On 10/19/2010 06:56 PM, Christian Holmquist wrote: method name                                POSIX  path        XP path -------------------------------------------------------------- ----------------------------------- temp_path                                     /tmp c:\TEMP user_(home|profile)_path             /home/bjorn c:\Documents and Settings\bjorn user_docs_path                             /home/bjorn c:\Documents and Settings\bjorn\My Documents

I disagree with your mappings.  I think leaving undefined what has no standard definition is superior. +1 The fact that I see it differently but understand some of the arguments being made here suggests to me that it should be left up to

On Wed, Oct 20, 2010 at 7:33 AM, Stewart, Robert <Robert.Stewart@sig.com> wrote: the application developers and not guess at it in Boost. For what it's worth I would see a correspondence something like: $TMPDIR - %TEMP% $HOME - %HOMEDRIVE%%HOMEPATH% $HOME/Documents - %HOMEDRIVE%%HOMEPATH%\My Documents And even so I'm very aware that even I don't even have a $HOME/Documents on some of the machines I use.

On 10/20/2010 01:33 PM, Stewart, Robert wrote:

Bjørn Roald wrote:

...

On 10/19/2010 06:56 PM, Christian Holmquist wrote:

...

Is there a POSIX equivalent for My Documents?

I think the users home directory is the closest you get as a general statement unless you start getting into details that are more specific for each POSIX system and/or distribution.

I disagree. The home directory is more like C:\ on Windows.

Wrong. I agree with statements in other postings that the home directory on Unix/Linux is not anything like C:\ on Windows.

As I mentioned in another post, there is no specific equivalent. Each user chooses to put documents in various subdirectories as suits their preference, so there isn't a good default on POSIX systems.

I assume you are talking about default for something that matches "My Documents" on Windows. I see absolutely no reason a boost::filesystem method that return the users My Documents folder on Windows should not return the users home directory on POSIX. What is so wrong with that? What problems should arise? As far as having a default behavior for these methods, assuming you with default mean something to fall back to when the primary method fails, I don't know if defaults are a good idea and I never suggested it. I think the proper strategy is to try a defined sequence of zero or more established techniques to find a valid existing directory covering a defined boost::filesystem concept for the given platform runtime environment. If that all fail it is probably best for the function to give up and and fail rather than guessing on some default directory and possibly trying to create the path.

Some desktop environments provide an equivalent, but as there is no standard even among those, it is still up to client code to determine the best equivalent, if wanted.

I still see no problems with just using $HOME on unix like systems as a match to My Documents. They do not have to be equivalent, they just need to be a place in the filesystem that serve the same concept for the software. An example is the concept of a directory under which the users private documents normally are organized.

...

method name POSIX path XP path -------------------------------------------------------------- ----------------------------------- temp_path /tmp c:\TEMP user_(home|profile)_path /home/bjorn c:\Documents and Settings\bjorn user_docs_path /home/bjorn c:\Documents and Settings\bjorn\My Documents

I disagree with your mappings. I think leaving undefined what has no standard definition is superior. Also, temp_path should be taken from the environment before using a fallback like /tmp.

I never sugested this table as a mapping. Immediately before my table I had text you have cut away, I quote myself:

...

A set of names of methods/enums and typical returned paths could be:

So please do not suggest I simply suggested a mapping.

...

user_appdata_path /home/bjorn

c:\Documents and Settings\bjorn\AppData appdata_path /etc c:\Documents and Settings\All Users\AppData user_desktop_path /home/bjorn/Desktop ?? c:\Documents and Settings\bjorn\Desktop desktop_path /etc/Desktop ?? c:\Documents and Settings\All Users\AppData etc..

This gets one into even more proprietary directions. A Desktop "folder" assumes a GUI which isn't necessarily the case on a POSIX system.

Agree, hence I put my question marks. Also, there is no standard way of defining the path to the desktop if there where a GUI. I do not even know if all GUI's have the concept of a desktop, or if they do if all of them map this to a filesystem directory.

The AppData "folder" on Windows has no equivalent on POSIX systems.

Why not? For the same concept as the Windows users AppData on POSIX systems the home directory works just fine. I have many directories in my Linux home directory containing application data for their respective applications. Many of these applications put their data in hidden files/directories to avoid polluting the home directory too much, but it is the standard solution as far as I can tell.

/etc is not appropriate as the user won't have write permission unless root.

That is why I suggested this to be similar to c:\Documents and Settings\All Users\AppData on Windows which typically require Administrator rights.

If such were to be supported by Filesystem, the best behavior remains leave undefined what has no standard definition.

The tricky part here is to come up with the desired concepts we want support for in boost::filesystem and then determine if there is a good solution on each relevant runtime platform. Looking for defined equivalence is the same as giving up. Then we will gain nothing. regards, Bjørn

On 10/21/2010 02:15 PM, Stewart, Robert wrote:

Bjørn Roald wrote:

...

On 10/20/2010 01:33 PM, Stewart, Robert wrote:

...

Bjørn Roald wrote:

...

On 10/19/2010 06:56 PM, Christian Holmquist wrote:

...

Is there a POSIX equivalent for My Documents?

I think the users home directory is the closest you get as a general statement unless you start getting into details that are more specific for each POSIX system and/or distribution.

...

...

As I mentioned in another post, there is no specific equivalent. Each user chooses to put documents in various subdirectories as suits their preference, so there isn't a good default on POSIX systems.

I assume you are talking about default for something that matches "My Documents" on Windows. I see absolutely no reason a boost::filesystem method that return the users My Documents folder on Windows should not return the users home directory on POSIX. What is so wrong with that? What problems should arise?

Since there is "absolutely no reason" that the home directory isn't appropriate, and I've enumerated various reasons against that mapping in other posts, there's no point in my answering your questions.

I said "I see no reason", I still do not see any. But I may be blind ;-)

...

As far as having a default behavior for these methods, assuming you with default mean something to fall back to when the primary method fails, I don't know if defaults are a good idea and I never suggested it.

Um, no. The idea of a default is what you get when you don't do anything to change the answer.

Right, the effect is the same. Technically it is most often simpler to set the default first and let it be overwritten - yes. Defaults too me means you always have a value, you never fail to provide one. Sure some value types may have a convention for an invalid value, if used as default you could signal failure trough it.

...

I think the proper strategy is to try a defined sequence of zero or more established techniques to find a valid existing directory covering a defined boost::filesystem concept for the given platform runtime environment.

I've been contending for the "zero" in "zero or more" because there's no really good mapping.

if you replace "because" with "if", then I agree with the above.

...

If that all fail it is probably best for the function to give up and and fail rather than guessing on some default directory and possibly trying to create the path.

You contend that the fallback, in this case, is the home directory. That's the default.

Absolutely not. There is no way you can be sure there is a valid home directory for the current user - so how can that be a default. You have to test a few thinks, if that fails to find a valid home directory, then the only sensible alternative is to return that fact to the caller.

...

I still see no problems with just using $HOME on unix like systems as a match to My Documents. They do not have to be equivalent, they just need to be a place in the filesystem that serve the same concept for the software.

What's the difference between being "equivalent" and serving "the same concept" as you've used those terms above? I find no difference between those, so I read the above as "they don't have to be equivalent, they just have to be equivalent."

for my notion of concepts here, the following apply: posix_home_dir is a user_doc_dir posix_home_dir is a user_profile_dir posix_home_dir is also other things that may be mapped to useful concepts user_doc_dir is not a user_profile_dir and vice versa hence neither of those two concepts are a posix_home_dir it follows that there is no equalence between user_doc_dir and posix_home_dir, but we still have that posix_home_dir is a user_doc_dir The point is that the posix_home_dir serves fine as a user_doc_dir without being the same thing by every letter of the law.

...

...

I disagree with your mappings. I think leaving undefined what has no standard definition is superior. Also, temp_path should be taken from the environment before using a fallback like /tmp.

I never sugested this table as a mapping. Immediately before my table I had text you have cut away, I quote myself:

...

...

A set of names of methods/enums and typical returned paths could be:

So please do not suggest I simply suggested a mapping.

If a given function returns X on platform A and it returns Y on platform B, then X and Y are equivalents between the two platforms A and B and therefore, one maps to the other. I have no idea what you read into the word "mapping" but your table clearly indicates equivalencies between directories on the two platforms.

They where examples of typically returned paths. You can argue that they are mapping of such - fine. But they are not mappings of suggested defaults.

...

...

The AppData "folder" on Windows has no equivalent on POSIX systems.

Why not? For the same concept as the Windows users AppData on POSIX systems the home directory works just fine. I have many directories in my Linux home directory containing application data for their respective applications. Many of these applications put their data in hidden files/directories to avoid polluting the home directory too much, but it is the standard solution as far as I can tell.

AppData is a rather structured, common location, away from other directories, that well-behaved applications are supposed to use. On POSIX systems, there is no common storage approach for data of that sort. Given that the manner in which such data is stored differs on the two platforms, there's no reason to suppose that the two directories will offer anything useful between the two platforms. Since everything about that data is platform specific, there is no point in Filesystem providing an accessor for such directories.

Hold on... I write application zzzz. I want to store application data for that application in a subdirectory called zzzz in a sensible place unique for each user on whatever platform my code runs on. I ask boost::filesystem::user_appdata_dir for that. If it return a usable path, then I create the zzzz subdirectory if it is not there already and store the data. Later I can read or write data in the same place with help of boost. I do not need to worry about how this could be different on a different platform. That is the whole point.

...

...

/etc is not appropriate as the user won't have write permission unless root.

That is why I suggested this to be similar to

c:\Documents and Settings\All Users\AppData

on Windows which typically require Administrator rights.

If you say so. I couldn't make sense of your table. By the time it arrived, it was munged beyond readability.

sorry about that - my bad. I did not realize my editor used a proportional font.

...

The tricky part here is to come up with the desired concepts we want support for in boost::filesystem and then determine if there is a good solution on each relevant runtime platform. Looking for defined equivalence is the same as giving up. Then we will gain nothing.

I have no idea what you're saying because that reads as, "find the desired concepts to support and look for a good default, but doing so is the same as giving up."

Where did you get "good default" from? I try to rephrase my intended message: Finding a perfect solution is impossible, so we should just give up. Or --- maybe, if we can find something that is very valuable and useful for boost::filesystem users, then we can accept a few scratches in the paint. -- Bjørn

Stewart, Robert wrote:

I have no experience with Macs, so I don't know how such data is managed on that platform, but I suspect it will be different in non-trivial ways from Windows and *nix.

Windows/Mac is my particular use case that spawned this thread. Mac(OSX) has the POSIX HOME environment variable defined to the directory /Users/<name> which is directly exposed in Finder with a house icon under the PLACES group. Home contains Applications, Desktop and Documents directories, among others. These appear to be stable location Win has the USERPROFILE (a composition of HOMEDRIVE, HOMEPATH) environment variable defined to the directory(at least on standard install of XP and Vista) C:\Documents and Settings\<name> which is *not* directly exposed in Explorer. Userprofile contains Desktop, My Documents directories, among others. There has been discussion concerning roaming issues. In both of these OS's there are no environment variables exposing Desktop or My Documents. Also the exact text is a function of the localization in effect. Both OS's do provide api's to access these paths accounting for localization. It appears based on this thread that *NIX's other than OSX do not have these concepts at all, other than maybe the concept of HOME which appears to be contentious in this thread as well. Is there room in boost for documents_directory_path, and desktop_directory_path, that return proper paths for (lack of a better categorization)GUI OS's? Jeff

Bjørn Roald wrote:

On 10/22/2010 01:11 PM, Stewart, Robert wrote:

...

Bjørn Roald wrote:

I have given reasons. You may say you find them insufficient, but even with the qualifier, "I see absolutely no reason," dismisses the rationale I've given out of hand.

Well, I am happy to settle with me finding the arguments insufficient and strike out "I see absolutely no reason" statement as it clearly is offending, something I did not intend - sorry.

Good. Apology accepted.

In any case I seriously think I may have missed points you have made in other posts. My statements was based om my understanding of the issue, and not a good understanding of all points made by others (including you) throughout this discussion.

OK. Please do try to account for all others have posted when writing or, at least, when another refers you to earlier arguments, have the courtesy to review the earlier posts to find the claimed arguments rather than expecting them to be reiterated.

Ok - we are discussing what a possible function returning the "My Documents" would return on Linux/UNIX and possibly also OSX platforms. Let us call this the user_doc_dir concept. This concept cover a natural place for the user to organize private documents. The directory is generally not shared, although most if not all systems will allow the users to make data public and accessible by others.

Good so far.

First of all I have not attempted to dig into any details, so there is likely to be variations on different Linux distributions and UNIX systems. Also there are many solutions for desktop environments which users to some extent can choose freely among, so this is not easy.

Agreed. That's where I foresee that there is not going to be a single policy that will satisfy all clients, meaning that hard coding one into Filesystem will be problematic from a justification and maintenance standpoint. To wit, a Trac issue is created asking, "Why isn't XYZ platform's configuration directory properly supported?" Then, Beman will need to learn about that platform, may well discover that its convention conflicts with that of another platform making distinguishing one from the other impossible. If that happens, what can he do but ostracize the requester -- and others using the XYZ platform -- by virtue of not being able to remove support for the platform already supported?

1. Let us assume that there exist some known solutions where user_doc_dir differ from the home directory. If there exist information about this boost::filesystem should check for this first, if a valid path is found with read/write/execute permissions, it should be used, else

OK

2. Parse /etc/passwd to find home directory of user, alternatively check pwd command output or $PWD value in a fresh shell or immediately after calling cd with no arguments, alternatively check $HOME value, if a valid path is found with read/write/execute permissions, it should be used, else

That's a lot of work for what's (soon-to-be) a header only library, isn't it? Still, it isn't unreasonable to expect.

3. signal failure

OK That's good. Previously, you were quite vague about what you thought should be included and simply argued against my use of the word, "default." Now I have a better understanding of your thoughts.

...

...

Hold on... I write application zzzz. I want to store application data for that application in a subdirectory called zzzz in a sensible place unique for each user on whatever platform my code runs on. I ask boost::filesystem::user_appdata_dir for that. If it return a usable path, then I create the zzzz subdirectory if it is not there already and store the data. Later I can read or write data in the same place with help of boost. I do not need to worry about how this could be different on a different platform. That is the whole point.

The convention on *nix systems is usually to create a .zzzz directory in the user's home directory, not a zzzz directory.

Right, so it make sense to do a simple test if we are on a POSIX system and make that dot caount :-)

You stated previously that you'd use the zzzz directory; I pointed out the need for it being called the .zzzz directory.

...

Furthermore, a Windows application will store a good deal of information in the registry which has no equivalent elsewhere. Consequently, the format, location, and layout of the configuration data will differ between the platforms.

Right, I think the registry is more associated with boost::program_options than boost::filesystem. Nevertheless I think complete and portable support of program_options concepts in boost is lacking, especially as far as Windows Registery as a database. It is not clear to me what advantages the repository provides over file based configurations except for ordered locations for data, simple consistent API, and possibly some expectations to well behaved applications.

I think there may be standards for data in the Windows registry supporting package management, windows installers, uninstallers and so forth. All modern OS/distros have similar databases for package management, so I think use of the registry for storing such package management data is orthogonal to use of registry to store application data.

You're suggesting that an application using Filesystem should be written with portability in mind so it will, of necessity, eschew proprietary things like the Windows registry whenever possible. That may be reasonable, but it also makes such applications less like the norm for a given platform, right? My point was that the directory name was different, the file formats were very likely different, some data that would be in the app data directory on POSIX systems would likely be in the Windows registry and, for all I know, somewhere else on OS X, etc., so there are a great many differences that call into question its providing real portability, and thus whether Filesystem should support such a thing.

...

Why then, would Filesystem try to offer something so much different as a general concept across platforms?

I have made many windows programs work without entries in registry that I am aware of. So I think it is mostly up to the application author how much if any registry is required. I for one would find a user_appdata_dir function useful.

OK. There are certainly some applications for which that is appropriate. The question is whether there are enough to make Filesystem support worthwhile. I can't answer that question, but it should be asked. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

Bjørn Roald wrote:

On 10/25/2010 01:44 PM, Stewart, Robert wrote:

...

Bjørn Roald wrote:

...

In any case I seriously think I may have missed points you have made in other posts. My statements was based om my understanding of the issue, and not a good understanding of all points made by others (including you) throughout this discussion.

OK. Please do try to account for all others have posted when writing or, at least, when another refers you to earlier arguments, have the courtesy to review the earlier posts to find the claimed arguments rather than expecting them to be reiterated.

Please stop this - this is not going anywhere. It is not that I did not read a lot before posting, I can not see where you got that impression from.

"I may have missed points you have made in other posts."

In this particular discussion I recall reading some of your posts in other threads on the subject of a portable "My Documents" folder concept. I just made the point that based on your reaction I thought there was a real chance I had missed something. It may also be the case that none of your arguments struck me as significant enough to make me change my position om the subject that a valid home directory is a fine solution on POSIX platforms. I do not intend to spend more time on this unless you point me to specifics.

Let me try making my points again, while incorporating subsequent ideas as well. My Documents is a default location for various kinds of documents. Windows users are notoriously naive because they come from so many walks of life, so it is common that everything be dumped in one place. Under earlier versions of Windows and DOS before that, users often dumped their files in the root directory of C:. Thus, My Documents is a way to force stupid^H^H^H^H^H^Hnaive users to dump their files in a less hazardous location. POSIX system users are typically less naive, though one reply mentioned the poster's wife as being in that class. Given the undesirable organizational results, choosing the home directory as the equivalent implies the need to find something better. To my way of thinking, one's home directory is very similar to C:\ on a Windows system -- not counting Vista and Win7, given their greater strictures on accessing the drive -- in that one usually puts little in the home directory and, instead, creates one or more arbitrary directories into which files are put. Now, with GUIs, there may well be an equivalent to My Documents, but only the applications written for that environment are likely to suggest it as a default location for saving a file. For example, running a GNOME app under KDE is likely to refer to the GNOME default rather than to the KDE default (assuming both have one). The foregoing suggests different locations for each GUI, plus arbitrary locations for other users. Now add the XDG stuff mentioned in another post and you have yet another variation for the location. Suppose you even try to account for the common GUI locations plus XDG. If you find more than one, which should be selected? Because of the above, the only common choice one can return on a POSIX system is the home directory and I find it a poor choice for a My Documents directory. Consequently, I'd rather Filesystem returned nothing than the home directory. Now, others have pointed out that most any location one would really want to use would be found under the home directory so, as a fallback, the home directory isn't so bad. Furthermore, it isn't Filesystem's job to babysit naive users; an application might choose to do something better. Does returning the home directory actually help provide portability? I don't think so. Given that Filesystem cannot decide among multiple choices when discovered (as described for multiple GUI equivalents plus XDG, all of which may exist), an application that calls a My Documents accessor won't know whether it is My Documents on a Windows system or the home directory to which some app-specific or vendor-specific subdirectory should be appended. Thus, I'd rather Filesystem return nothing when there is no obvious choice. Then, the application can learn that there is no path available from Filesystem, query for the home directory, and do the app-specific or vendor-specific work from that known point. If an application is written with generic portability in mind, then it would retrieve the home directory and add app-specific or vendor-specific subdirectories to locate the application-related documents, ignoring the My Documents directory on Windows. Such an app does not require an accessor for a My Documents type directory in Filesystem. My conclusion is that Filesystem should not provide a My Documents type directory unless there is an obviously good choice on the current system. IOW, it could include some complicated search for various environment-specific directories and, if there is a unique answer, return that but otherwise it should declare an ambiguity or failure to find an appropriate directory. Going back to my idea of calling a particular function to assert the desired convention, failure to find a conforming directory could result in creating the directory. It certainly would remove the decision Filesystem would otherwise be obliged to make of disambiguating among choices or forcing a potentially undesirable default. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

On 10/26/2010 05:37 PM, Stewart, Robert wrote:

Bjørn Roald wrote:

Please stop this - this is not going anywhere. It is not that

...

I did not read a lot before posting, I can not see where you got that impression from.

"I may have missed points you have made in other posts."

Note: Your quotations are above misleading, it looks like I wrote what you wrote and visa-versa. I see where you misinterpret me ;-) If I say I have missed something then I tried to find it, I most likely read it, but failed to see the significance, or relevance. In my book you can not miss something you are not looking for :-)

...

In this particular discussion I recall reading some of your posts in other threads on the subject of a portable "My Documents" folder concept. I just made the point that based on your reaction I thought there was a real chance I had missed something. It may also be the case that none of your arguments struck me as significant enough to make me change my position om the subject that a valid home directory is a fine solution on POSIX platforms. I do not intend to spend more time on this unless you point me to specifics.

Let me try making my points again, while incorporating subsequent ideas as well. My Documents is a default location for various kinds of documents. Windows users are notoriously naive because they come from so many walks of life, so it is common that everything be dumped in one place.

I see your point, but we may say the same about users of other operating systems as well. I don't think windows users are generally more naive than others.

Under earlier versions of Windows and DOS before that, users often dumped their files in the root directory of C:.

Sure, if nothing prevents it or guide them elsewhere.

Thus, My Documents is a way to force stupid^H^H^H^H^H^Hnaive users to dump their files in a less hazardous location.

I do not like calling users stupid. Maybe even calling them naive is to strong. Maybe it is more appropriate to state that programmers are clearly naive and stupid as they have so hard time figuring out how to make software the users can understand.

POSIX system users are typically less naive,

maybe or maybe not

though one reply mentioned the poster's wife as being in that class.

Given the undesirable organizational results, choosing the home directory as the equivalent implies the need to find something better.

Better in what way?

To my way of thinking, one's home directory is very similar to C:\ on a Windows system -- not counting Vista and Win7, given their greater strictures on accessing the drive -- in that one usually puts little in the home directory and, instead, creates one or more arbitrary directories into which files are put.

The main trait of the home directory is that it is personal, it belongs to one user. As in others are generally not welcome to use it. Whether security features exists or are set up to enforce that, is a different mather. Make note that nothing prevents me from opening up full access to my home directory to anybody logged on:

chmod 777 /home/bjornr

One can question how smart that is, but that did not change /home/bjornr in any way so it is now not my home directory. You are right that systems that does not know one user from another does generally not provide any help in organizing personal data either. Users on those machines have to do the job of organizing the filesystem all by themselves and the filesystem(s) are typically wide open and to their disposal - whether it is c:\ or anything else. That does not make the complete file system a home directory as long as we do not assume or know the file system is completely personal. We also know that the file system root most often is something else than the home directory, as it is the place the system data and programs live.

Now, with GUIs, there may well be an equivalent to My Documents, but only the applications written for that environment are likely to suggest it as a default location for saving a file. For example, running a GNOME app under KDE is likely to refer to the GNOME default rather than to the KDE default (assuming both have one).

They attempt to have a standard that differ as where the open desktop In fact both use the home directory.

The foregoing suggests different locations for each GUI

plus arbitrary locations for other users. Now add the XDG stuff mentioned in another post and you have yet another variation for the location. Suppose you even try to account for the common GUI locations plus XDG. If you find more than one, which should be selected?

I think you try to make this more complicated than it needs to be. For the concepts (if any) that boost::filesystem should support, we have the option to follow http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html and/or other similar systems depending on runtime and if that does not provide a solution, we can fall back to the home directory or something else that is an appropriate fall-back for the concept in question.

Because of the above, the only common choice one can return on a POSIX system is the home directory and I find it a poor choice for a My Documents directory. Consequently, I'd rather Filesystem returned nothing than the home directory.

Given that many have pointed out that the only appropriate use of a My Documents directory concept is as default directory for Save As / File Open dialogs or file explorer like applications, then I see little reason to discredit the feasibility of the home directory as fallback solution as strongly as you do.

Now, others have pointed out that most any location one would really want to use would be found under the home directory so, as a fallback, the home directory isn't so bad.

Ok, I guess we more or less agree then.

Furthermore, it isn't Filesystem's job to babysit naive users; an application might choose to do something better.

No its job is to babysit naive programmers and their stupid applications :-)

Does returning the home directory actually help provide portability? I don't think so. Given that Filesystem cannot decide among multiple choices when discovered (as described for multiple GUI equivalents plus XDG, all of which may exist), an application that calls a My Documents accessor won't know whether it is My Documents on a Windows system or the home directory to which some app-specific or vendor-specific subdirectory should be appended.

Applications should never add data to My Documents, only users shall.

Thus, I'd rather Filesystem return nothing when there is no obvious choice. Then, the application can learn that there is no path available from Filesystem, query for the home directory, and do the app-specific or vendor-specific work from that known point.

Ok, we are back where we started and as always anything is possible, but what known point are you referring to?

If an application is written with generic portability in mind, then it would retrieve the home directory and add app-specific or vendor-specific subdirectories to locate the application-related documents, ignoring the My Documents directory on Windows. Such an app does not require an accessor for a My Documents type directory in Filesystem.

Some apps may: File Open File Save As Explorer type apps Apps searching for and indexing user files, photos, music, etc. ... But you are right, there may be more important stuff to have in boost::filesystem.

My conclusion is that Filesystem should not provide a My Documents type directory unless there is an obviously good choice on the current system. IOW, it could include some complicated search for various environment-specific directories and, if there is a unique answer, return that but otherwise it should declare an ambiguity or failure to find an appropriate directory.

That is one possible solution, but it depend on your definition of the concept. You idea is to require the concept to represent the one and only unique user document directory for the current user. Another possibility is that the concept represent a directory that is appropriate as a document directory for the current user. Yet another, a list of appropriate directories. It all depend on what is needed.

Going back to my idea of calling a particular function to assert the desired convention, failure to find a conforming directory could result in creating the directory. It certainly would remove the decision Filesystem would otherwise be obliged to make of disambiguating among choices or forcing a potentially undesirable default.

This may be a good idea, I am just struggling a bit with this as it seems we are just pushing the difficult and tricky decision back on the application. How shall the application determine which convention to use? -- Bjørn

On 10/27/2010 02:51 PM, Stewart, Robert wrote:

Bjørn Roald wrote:

...

On 10/26/2010 05:37 PM, Stewart, Robert wrote:

...

To my way of thinking, one's home directory is very similar to C:\ on a Windows system -- not counting Vista and Win7, given their greater strictures on accessing the drive -- in that one usually puts little in the home directory and, instead, creates one or more arbitrary directories into which files are put.

The main trait of the home directory is that it is personal, it belongs to one user. As in others are generally not welcome to use it.

That was, traditionally, the same for C:\ on DOS/Windows.

except for the fact that the whole system lived in that directory, not only personal data and settings.

...

You are right that systems that does not know one user from another does generally not provide any help in organizing personal data either. Users on those machines have to do the job of organizing the filesystem all by themselves and the filesystem(s) are typically wide open and to their disposal - whether it is c:\ or anything else. That does not make the complete file system a home directory as long as we do not assume or know the file system is completely personal. We also know that the file system root most often is something else than the home directory, as it is the place the system data and programs live.

The analogy is by no means perfect, I admit. The point is that there are many things comingled in one's home directory. One has configuration files, application caches, scripts, possibly even some applications, so putting documents in the home directory itself is undesirable. Providing something akin to My Documents is desirable, if there is any possibility of doing so.

Agreed.

...

...

with GUIs, there may well be an equivalent to My Documents, but only the applications written for that environment are likely to suggest it as a default location for saving a file. For example, running a GNOME app under KDE is likely to refer to the GNOME default rather than to the KDE default (assuming both have one).

They attempt to have a standard that differ as where the open desktop In fact both use the home directory.

I can't parse that.

yes, I can not parse it either. But I know what was on my mind :-) It seems GNOME and KDE both attempt to use the XDG (freedesktop standard) for these things. My Kubuntu box has /etc/xdg/user-dirs.defaults with the following: # Default settings for user directories # # The values are relative pathnames from the home directory and # will be translated on a per-path-element basis into the users locale DESKTOP=Desktop DOWNLOAD=Downloads TEMPLATES=Templates PUBLICSHARE=Public DOCUMENTS=Documents MUSIC=Music PICTURES=Pictures VIDEOS=Videos # Another alternative is: #MUSIC=Documents/Music #PICTURES=Documents/Pictures #VIDEOS=Documents/Videos And these folders are created as subfolders of home for new users. I think GNOME on Ubuntu is the same, but I am not sure. To some extent I have clearly been guided into using the Desktop and the Documents folder. But in general most tools drops me as user in my home directory when I need to save something and I have my own long lived organization of data and have never paid attention to those other folders, hence they are empty. Also, the GUI desktop has a large "Home" widget displaying the content of the home directory. My conclusion is that except of the effects of files and folders dropped on the GUI desktop being mapped to the Desktop subdirectory of my home, the home directory itself is the top level concept for where the user is left alone. I guess the rest of the folders more or less are just suggestion to an organization.

...

...

The foregoing suggests different locations for each GUI plus arbitrary locations for other users. Now add the XDG stuff mentioned in another post and you have yet another variation for the location. Suppose you even try to account for the common GUI locations plus XDG. If you find more than one, which should be selected?

I think you try to make this more complicated than it needs to be.

For the concepts (if any) that boost::filesystem should support, we have the option to follow <http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html> and/or other similar systems depending on runtime and if that does not provide a solution, we can fall back to the home directory or something else that is an appropriate fall-back for the concept in question.

I have no concept of the authority of the spec you reference, but I do appreciate the idea of looking for something more specific and, if that fails, falling back on the home directory. For a My Documents equivalent, this may be adequate as in either case, the directory is usable, if not ideal. However, for an AppData equivalent, such an approach seems less useful because the user cannot be called upon to better direct the application. That is, the home directory may not be the desired location and, if the XDG directory is not found, how is the user to direct the application to the desired location?

...

...

Does returning the home directory actually help provide portability? I don't think so. Given that Filesystem cannot decide among multiple choices when discovered (as described for multiple GUI equivalents plus XDG, all of which may exist), an application that calls a My Documents accessor won't know whether it is My Documents on a Windows system or the home directory to which some app-specific or vendor-specific subdirectory should be appended.

Applications should never add data to My Documents, only users shall.

I had AppData in mind as well as My Documents, though I only mentioned the latter.

...

...

Thus, I'd rather Filesystem return nothing when there is no obvious choice. Then, the application can learn that there is no path available from Filesystem, query for the home directory, and do the app-specific or vendor-specific work from that known point.

Ok, we are back where we started and as always anything is possible, but what known point are you referring to?

If the app gets nothing from the My Documents query, then it branches into code that grabs the home directory and works from there. This leaves the fallback policy in the application rather than Filesystem. Of course, if the My Documents query returned the home directory, the application could query for the home directory, compare the two, and reach the same conclusion.

...

...

If an application is written with generic portability in mind, then it would retrieve the home directory and add app-specific or vendor-specific subdirectories to locate the application-related documents, ignoring the My Documents directory on Windows. Such an app does not require an accessor for a My Documents type directory in Filesystem.

Some apps may: File Open File Save As Explorer type apps Apps searching for and indexing user files, photos, music, etc. ...

I was really confusing matters, because I had AppData in mind when I wrote My Documents there.

...

...

Going back to my idea of calling a particular function to assert the desired convention, failure to find a conforming directory could result in creating the directory. It certainly would remove the decision Filesystem would otherwise be obliged to make of disambiguating among choices or forcing a potentially undesirable default.

This may be a good idea, I am just struggling a bit with this as it seems we are just pushing the difficult and tricky decision back on the application. How shall the application determine which convention to use?

First, you always want to push policy up to higher levels. Therefore, when there's a policy choice, Filesystem shouldn't make it. Filesystem can make selecting a given policy easy, but it shouldn't force one when there are choices available. That's just good design.

So far I agree if it means Filesystem select a default based on general knowledge and let the application override at will.

Second, the application can choose a convention based upon the choice of the programmer or vendor. IOW, if the app is supposed to target KDE, it will likely want to follow KDE's convention. If the app wants to be XDG compliant, it will choose that convention. If there is some more complicated, local convention supported by a particular developer or vendor, the application can write a function to effect that convention.

Filesystem could follow widely supported standards and, if the indicated directory does not exist, return the home directory. However, if instead it returned an empty path by default, clients would be forced to make a conscious choice of convention by setting the various directories using such functions as I've discussed so code querying those directories will get something useful. Arguably, just returning the home directory doesn't prevent clients from setting the directories to something more conventional, but by not encouraging selection of a convention, it leads to somewhat poorer results from lazy programmers.

I see your point, but I am not so sure this issue is hard enough for Filesystem to worry about the negative effects of lazy programmers, at least not to the extent you suggest. But if it is good reason to worry, and not feasible for Filesystem to make good policy choices, then I think your suggested approach should be considered. Our discussion has probably not been very helpful in clearing the ground for an implementation. There are clearly also some postings against adding any of this to boost at all, so it is worth reviewing if it make sense to add it. As usual someone have to step up and make a proposal covering at least Windows, Mac and Linux/Unix, and in this case with a reasonable set of runtime variants for each. Maybe this is a sensible idea for a GSoC task, but it may be to light weight academically for that, I do not know. Regrettably I can not help making a proposal, I would have liked to do that but I can not. -- Bjørn

2010/10/27 Patrick Horgan <phorgan1@gmail.com>:

I'm against the whole idea of home_directory_path or my_document_path because it calls for filesystem to make policy decisions that are better left to applications.  If you provide it, it will get used and silly things will happen.

I still can't understand, what things exactly? The only change worth doing is merging both functions into default_personal_files_path. It's neither a home directory, nor the "my documents" directory. Alexander Churanov

Bjørn Roald wrote:

On 10/22/2010 01:11 PM, Stewart, Robert wrote:

...

Bjørn Roald wrote:

As to the latter approach, note that Filesystem could even offer several functions a user can call to establish the values it vends. For example, there might be a function for POSIX systems that discovers the user's home directory, validates it, and then sets it as the Filesystem home directory, configuration directory, etc., according to your notions. There could be another that establishes some other common convention used on certain Linux distributions, for example. There could be a function that establishes WinXP conventions, another for Win7, and yet another for WinCE conventions. Thus, the user can take advantage of prepackaged code, if appropriate and available, or write custom code to set the Filesystem values according to some local convention.

Given that approach to establishing the values vended by Filesystem, all other user code can simply query Filesystem for the values when needed, being thus ignorant of what convention was set during initialization, while Filesystem avoids encouraging any given convention over another.

I am not sure I follow all this.

I'm suggesting that Filesystem's core merely provides a means to set and vend directories, such as the home directory or the app data directory. Most code will be written in terms of the vended values. If Filesystem is queried for a value that wasn't set, it throws an exception. Application initialization code, by contrast must set the directories for Filesystem to vend if other code is going to query them. To do so, that initialization code chooses some convention for discovering the desired directories and calls an appropriate function that will use that convention and then set the directories for Filesystem to vend. That approach means that Filesystem is open ended. Any library client can devise a different convention either by replacing or augmenting existing functions that set the directories.

How does this provide the ability to write simple, straight forward, and portable C++ in the spirit of boost? When I refer to giving up, I refer to those goals.

I'm not certain anyone else agreed to those goals specifically, but they are reasonable, so certainly worth addressing. The goal of simple code is achieved by making most code simply ask Filesystem for the desired directory. The initialization code is complicated, relative to a hardcoded scheme built into Filesystem's initialization logic, by the need to pick a directory initialization function and calling it before querying the directories. That's hardly difficult, but it is one step removed from the simplest interface possible. The goal of straightforward code is achieved by the simplicity. One calls one extra function that indicates the user directory convention desired. boost::filesystem::set_user_directories_default_convention(); boost::filesystem::set_user_directories_xdg_convention(); The default convention might encode some set of rules thought to be most generally applicable across all platforms -- what you might otherwise hard code into Filesystem -- but that aren't completely acceptable to all concerned, thus making not hard coding them preferable. The portability goal is achieved by the default convention, provided it is sufficient for a given client. The XDG convention clearly applies only to systems that support that convention, so conditional compilation would be likely needed for portability, or a client might test the environment to choose that versus another initialization function at runtime. Some other client may choose to write a custom function to set user directories according to local policies, which can be written with platform-specific logic inside the function, so that calling the one function during initialization accounts, portably, for the platform differences. I hope this makes more sense to you now. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

On Tuesday 19 October 2010 09:56:45 Christian Holmquist wrote:

Is there a POSIX equivalent for My Documents? Without a way to retrieve My Documents, I for one still needs my own wrapper for that..

On newer Linux systems, try $XDG_DOCUMENTS_DIR for an equivalent. Not sure whether it will make it into POSIX. Ravi

On Friday 22 October 2010 19:53:52 Patrick Horgan wrote:

On 10/19/2010 09:06 PM, Ravi wrote:

...

On Tuesday 19 October 2010 09:56:45 Christian Holmquist wrote:

...

Is there a POSIX equivalent for My Documents? Without a way to retrieve My Documents, I for one still needs my own wrapper for that..

On newer Linux systems, try $XDG_DOCUMENTS_DIR for an equivalent. Not sure whether it will make it into POSIX.

Data point, I'm on Ubuntu Meercat which is pretty new and doesn't define this. I only get:

XDG_SESSION_COOKIE=caae1d00b0a381de2973b0004748ac01-1287635546.203915-55787 7136 XDG_CONFIG_DIRS=/etc/xdg/xdg-gnome:/etc/xdg XDG_DATA_DIRS=/usr/share/gnome:/usr/local/share/:/usr/share/

One needs to use xdg-user-dirs to obtain the correct values: http://freedesktop.org/wiki/Software/xdg-user-dirs Both the premier Linux desktop environments KDE & Gnome support the spec; see, for example: http://norman.hooper.name/blog/post/32/changing-user-directory-names-in- gnome-and-kde/ As far as I can tell, other desktop environments offer partial support as well. For you case of Ubuntu: http://www.andrewbolster.info/blog/2010/06/customised-user-directories-in- ubuntu/ If you have Gnome installed, you will also have two packages, xdg-user-dirs and xdg-utils, installed on your system. The documentation accompanying them should provide complete details. Regards, Ravi

CSIDL_PERSONAL corresponds to the user's 'My Documents' folder, whereas CSIDL_PROFILE corresponds to the user's profile directory which is more consistent with the POSIX 'HOME' directory.

Checking my USERPROFILE folder it does contain log files, some .internal_folders_for_nix_like_applications and stuff that, well, belongs to my profile (Desktop, Favorites etc) I can't say that it resembles -more- a HOME path than does My Documents. USERPROFILE is still a mystery to me, I would not have my applications store anything there. :) Christian

Alexander Churanov wrote:

As a user of both Windows and UNIX (FreeBSD), I think that the discussion is overcomplicated. There several things an application needs to know about the directories:

1) Where to put/find user-specific configuration-data. 2) Where to find system-wide configuration-data. 3) Where to put user-specific caches. 4) Where to find user-generated content.

I fail to see how you have simplified the discussion with that enumeration. It may be more succinct or described in less proprietary terms, but it isn't a simplification.

Note, UNIX users typically store their files under ${HOME}, it is OK ***** to consider ${HOME} as an equivalent of "My Documents", though they are not the same.

They are definitely not the same. As I stated elsewhere, the user's home directory is equivalent to C:\ on a Windows system. The user shouldn't put "user-generated content" there. It should be in some subdirectory as preferred by the user, which you understood when you wrote "under" in the quoted sentence. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

Alexander Churanov wrote:

2010/10/20 Stewart, Robert <Robert.Stewart@sig.com>:

...

They are definitely not the same. As I stated elsewhere, the user's home directory is equivalent to C:\ on a Windows system. The user shouldn't put "user-generated content" there. It should be in some subdirectory as preferred by the user, which you understood when you wrote "under" in the quoted sentence.

I completely disagree on "C:\". The ${HOME} is user-specific, while "C:\" is not.

The latter is nearly true as Windows is only barely multi-user.

It is not uncommon to put documents (including textual) in different subdirectories under ${HOME}. For example, on my machine "lf ~" lists "examples" and "tasks" as different directories. To my mind, none of them entirely corresponds to "My Documents", but the whole ${HOME} does.

My Documents, being at once user specific and not the root directory of a drive maps well to a subdirectory of the user's home directory, not to the home directory itself. Again, it is poor practice to overpopulate one's home directory, just as over-populating C:\ is poor practice; Filesystem should not establish that as a default. The policy should be determined by the client, not by Filesystem as there is no agreement.

Another argument: some desktop environments place an icon on the desktop, which opens ${HOME}. Along with "System" and "Trash" this looks very related to default Windows icons.

That there's a button to access one's home directory in no way implies that the home directory must be the default location of user-generated content. Instead, it means the GUI doesn't know what the user might want to view, but that the home directory is the likely starting point for finding it. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

Alexander Churanov wrote:

2010/10/20 Stewart, Robert <Robert.Stewart@sig.com>:

...

The latter is nearly true as Windows is only barely multi-user.

Robert, this is surprising. What versions of Windows are to your opinion not multi-user? I think it is worth to examine XP/Vista/7 instead of 98/Me, which are dead.

Common knowledge. Windows is a single user system with multi-user functionality overlaid. All users have full access to C:\.

...

it is poor practice to overpopulate one's home directory

I read this for the first time. Could you, please, provide some links? The organization of user-generated files in home directory is up to the user.

I have no links. I'm sure I could find some if I tried, but this is wisdom passed down from others years past and from my own decades of experience. No directory should be overpopulated, as a rule, because directory searches take increasingly long as the number of files grow (though some filesystems are less hampered than others) and because directory listings become unwieldy to grok visually. The home directory is more important to not over-populate because it is already the location of dot files and all subdirectories leading to other content. Adding user-generated content to the necessary content makes it harder to use as the jump point to all user content, whether user-generated or locally installed applications/libraries. I hope these points help to convince you that Filesystem should not return $HOME by default. It is better to leave that decision to the client code and a convenience default will encourage use of that default, so it ought not to be a default that leads to poor practice. _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

On 20 October 2010 11:42, Alexander Churanov <alexanderchuranov@gmail.com>wrote:

2010/10/20 Stewart, Robert <Robert.Stewart@sig.com>:

...

Common knowledge. Windows is a single user system with multi-user functionality overlaid. All users have full access to C:\.

I'm afraid that isn't true. Search using google for "vista access to C drive".

If we disregard historical reasons for why things behave as they do on Windows, to me it's quite clear that C:\ (or the OS root drive) is not each user's $HOME path. They may or may not have write access outside their own My Documents (or Users) but that doesn't change anything. You can setup a NIX system that way too probably, allowing full access everywhere. $HOME would remain however. Are those people advocating for $HOME on Windows to be anything else than My Documents really Windows users? I've been trough development of fixing pre-Vista developed applications to work, and the times where one could just ignore access rights on Windows is long gone. IMO Boost.Filesystem should follow best practices for each supported platform, and using C:\ as default user folder or the obscure USERPROFILE is IMO not good practice on Windows. Christian

On Wed, Oct 20, 2010 at 7:11 PM, Christian Holmquist <c.holmquist@gmail.com> wrote:

On 20 October 2010 11:42, Alexander Churanov <alexanderchuranov@gmail.com>wrote:

Are those people advocating for $HOME on Windows to be anything else than My Documents really Windows users? I've been trough development of fixing pre-Vista developed applications to work, and the times where one could just ignore access rights on Windows is long gone. IMO Boost.Filesystem should follow best practices for each supported platform, and using C:\ as default user folder or the obscure USERPROFILE is IMO not good practice on Windows.

USERPROFILE on my Windows machine points to <root>\Users\<username>. This is pretty much the only dir by default where my normal user id has write permission. This is where Documents, Desktop, Favorites, Appdata user directories are. Is there any reason *not* to use USERPROFILE on Vista or later? -- gpd

On 20 October 2010 13:53, Giovanni Piero Deretta <gpderetta@gmail.com>wrote:

On Wed, Oct 20, 2010 at 7:11 PM, Christian Holmquist <c.holmquist@gmail.com> wrote:

...

On 20 October 2010 11:42, Alexander Churanov < alexanderchuranov@gmail.com>wrote:

Are those people advocating for $HOME on Windows to be anything else than My Documents really Windows users? I've been trough development of fixing pre-Vista developed applications to work, and the times where one could just ignore access rights on Windows is long gone. IMO Boost.Filesystem should follow best practices for each supported platform, and using C:\ as default user folder or the obscure USERPROFILE is IMO not good practice on Windows.

USERPROFILE on my Windows machine points to <root>\Users\<username>. This is pretty much the only dir by default where my normal user id has write permission. This is where Documents, Desktop, Favorites, Appdata user directories are. Is there any reason *not* to use USERPROFILE on Vista or later?

On Windows 7/Vista USERPROFILE seems definitely right, I apologize. At work on XP with some kind of networked domain storage thing for My Documents, C:\Documents and settings\<username> seems a little like bogus, but maybe the domain/installation is just ill-configured, and doesn't serve as a good example. Christian

On Wed, Oct 20, 2010 at 02:44:25PM +0200, Sebastian Redl wrote:

...

Jeff and others,

As a user of both Windows and UNIX (FreeBSD), I think that the discussion is overcomplicated. There several things an application needs to know about the directories:

1) Where to put/find user-specific configuration-data. 3) Where to put user-specific caches.

For Windows they are:

1) Application Data\Company Name\Product Name in user profile (CSIDL_APPDATA) 3) Application Data\Company Name\Product Name in user profile (CSIDL_APPDATA)

On 20.10.2010 14:09, Alexander Churanov wrote: 3) should be CSIDL_LOCAL_APPDATA, if available, since CSIDL_APPDATA is synced to the server for roaming profiles, which is a bad idea for caches.

Excuse me if it's posted before in this massive thread, but Raymond Chen says it best: http://blogs.msdn.com/b/oldnewthing/archive/2005/07/01/434647.aspx << What's the difference between My Documents and Application Data? >> As for the whole "C:\ is the user's home directory", that's just wrong. First of all, a regular user do not have write access anywhere on C:, as there's ACLs to Deny it in several places, not to mention that applications without decent manifests will be virtualized if they try. Second of all, who says there exists a C: at all? I've had lots of software break spectacularly when I had my OS on E:. -- Lars Viklund | zao@acc.umu.se

Ulrich Eckhardt wrote:

On Monday 18 October 2010 16:58:31 Jeff Flinn wrote:

...

There is the windows GetUserProfileDirectory api function also available since NT4. This is in the Userenv.lib/dll, obviously adding a link dependancy. [snip] How are these sorts of system link dependencies handled? Through bjam? Or is it acceptable/better to use #pragma comment(lib,"Userenv.lib")?

Using that #pragma is not portable, otherwise I would prefer it.

This would only be used on Windows systems. Isn't that pragma portable across Windows compilers? _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

Hi, Did a final decision has been taken for this feature? Is it abandonned? Joël Lamotte