Boost.Process article: Tutorial and request for comments

I've written an article about Boost.Process: http://www.highscore.de/cpp/process/ I've also fixed a few bugs in Boost.Process (which is now 0.31): http://www.highscore.de/boost/process.zip And the latest source code can also be found in the sandbox: https://svn.boost.org/svn/boost/sandbox/process/ For developers not familar with Boost.Process the article is a tutorial to get more easily started. For developers familiar with Boost.Process it is a request for comments. I've been looking at the big picture finally trying to find out where and how Boost.Process should be changed and in which direction it should move to become a complete library one day. As not everyone will read the entire article here are my thoughts: 1) Boost.Process is based on a few concepts like 'process' and 'context'. While those make sense there are other concepts where I wonder if they should be dropped: * 'executable' is the executable name which in practice must be a std::string. I think the original idea was to support numerous string types. This leads to the question how to support different Unicode encodings which I think is still unanswered in the Boost community. Other libraries, eg. Boost.Interprocess, simply stick to one string type (typically std::string). * 'handle' is the underlying native file handle of a stream (file descriptor on POSIX systems, handle on Windows). The concept is based on a class in the detail namespace of Boost.Process. There is just one class and it's not meant to be used by users of the library either. Thus I'm not sure what's the benefit of having an explicit 'handle' concept at all. 2) Environment variables must be sorted on Windows but not on POSIX systems. On Windows the type environment could be a typedef for std::map and on POSIX systems for std::unordered_map. Currently it's always a typedef for std::map. Is that ok? Should it be changed? Should a macro be provided in case developers want to use different typedefs? 3) Currently Boost.Process is really only useful for creating child processes. It's not possible for example to detect and iterate over other processes on a system. I guess Boost.Process should be enhanced here. Then one day the well-known utility 'ps' on POSIX systems could be implemented in Boost C++. Any comments? 4) When a process is created the standard streams (stdin, stdout and stderr) are closed by default. Is this a good choice? If it isn't what should happen otherwise? Inherting standard streams (which is the default behavior with fork on POSIX systems)? 5) Boost.Process requires to specify an absolute path to an executable which should be started. There is a helper function which returns the absolute path for an executable name (it uses the environment variable PATH). The question is if this should be done automatically if no absolute path is given? 6) When a stream of a child process is captured Boost.Process automatically creates a pipe. As pipes don't support asynchronous operations on Windows a named pipe is required (called FIFO on POSIX systems). Currently a macro must be defined to make Boost.Process use a named pipe on Windows. It might make more sense to enable developers to choose either a pipe or a FIFO explicitly - not only on Windows but on all platforms. Any comments? 7) It's possible to capture and access streams of a child process. Those streams are derived from std::istream and std::ostream and thus only support blocking I/O operations. Currently the underlying handle has to be fetched to initialize a generic I/O object from Boost.Asio to use asynchronous I/O operations. How should the library be changed to make it easier to use asynchronous I/O? 8) There is a method wait() which makes it possible to wait for another process to exit. Currently wait() is provided by the class child though. Shouldn't it really be defined by the class process (so you can wait not only for child processes to exit)? 9) As wait() blocks an asynchronous alternative should be provided. 10) The Windows-specific classes all start with win32_ (eg. win32_child). Wouldn't it make more sense to use windows_ instead (like windows_child) or a windows namespace? First other libraries use windows_ (see Boost.Interprocess). Second there should be no reason why a class like win32_child can't be used on a 64-bit Windows. Boris

Boris Schaeling escribió:
I've written an article about Boost.Process: http://www.highscore.de/cpp/process/
I've also fixed a few bugs in Boost.Process (which is now 0.31): http://www.highscore.de/boost/process.zip
And the latest source code can also be found in the sandbox: https://svn.boost.org/svn/boost/sandbox/process/
For developers not familar with Boost.Process the article is a tutorial to get more easily started. For developers familiar with Boost.Process it is a request for comments.
I think it is important that you state that Boost.Process is not a Boost library actually, at least currently. Referring to it as simply Boost.Process can be misleading, so a better alternative is to name it "Candidate Boost.Process" or something like that. Incidentally, given that the lib seems to be orphan, maybe you're in a position to adopt it and bring it to a state that it can be officially reviewed? Joaquín M López Muñoz Telefónica, Investigación y Desarrollo

Boris Schaeling wrote:
I've been looking at the big picture finally trying to find out where and how Boost.Process should be changed and in which direction it should move to become a complete library one day. As not everyone will read the entire article here are my thoughts:
1) Boost.Process is based on a few concepts like 'process' and 'context'. While those make sense there are other concepts where I wonder if they should be dropped:
Here are a few opinions:
* 'executable' is the executable name which in practice must be a std::string. I think the original idea was to support numerous string types. This leads to the question how to support different Unicode encodings which I think is still unanswered in the Boost community. Other libraries, eg. Boost.Interprocess, simply stick to one string type (typically std::string).
I assume the name of the executable should be in the locale character set. So std::string, or even const char*, appear suitable. Now, if you want to perform automatic translation from unicode to locale character set, why not, but that's another layer on top of that. An unicode string type should be convertible to the locale character set easily enough so that you don't need to that in your library, however. You could eventually add support for wide characters, though. While wide characters are Unicode on Windows, they are as far as C++ is concerned just another locale character set, and I think considering them as nothing more than that is best. As far as I know, POSIX systems simply don't have APIs for wide characters, however.
* 'handle' is the underlying native file handle of a stream (file descriptor on POSIX systems, handle on Windows). The concept is based on a class in the detail namespace of Boost.Process. There is just one class and it's not meant to be used by users of the library either. Thus I'm not sure what's the benefit of having an explicit 'handle' concept at all.
Isn't it in case Boost.Process doesn't implement the features I need, so that I may use them directly with the native functions?
3) Currently Boost.Process is really only useful for creating child processes. It's not possible for example to detect and iterate over other processes on a system. I guess Boost.Process should be enhanced here. Then one day the well-known utility 'ps' on POSIX systems could be implemented in Boost C++. Any comments?
I personally think this has pretty low priority.
8) There is a method wait() which makes it possible to wait for another process to exit. Currently wait() is provided by the class child though. Shouldn't it really be defined by the class process (so you can wait not only for child processes to exit)?
How can you wait for something else than your children, assuming there is no "ps"-like functionality?
9) As wait() blocks an asynchronous alternative should be provided.
Definitely.

On Tue, 21 Apr 2009 16:05:04 +0200, Mathias Gaunard
[...]I assume the name of the executable should be in the locale character set. So std::string, or even const char*, appear suitable.
Now, if you want to perform automatic translation from unicode to locale character set, why not, but that's another layer on top of that. An unicode string type should be convertible to the locale character set easily enough so that you don't need to that in your library, however.
You could eventually add support for wide characters, though. While wide characters are Unicode on Windows, they are as far as C++ is concerned just another locale character set, and I think considering them as nothing more than that is best.
As far as I know, POSIX systems simply don't have APIs for wide characters, however.
Another problem on Windows might be that new projects in VC++ 2008 automatically assume Unicode is used. Thus all Windows API functions expect wide character strings which again means for executable names std::wstring would need to be used. And I don't know if developers really think about it and not simply assume that they can use std::string in all cases. I think Ion faced a similar problem when he developed Boost.Interprocess which only accepts const char* strings (and according to http://www.boost.org/doc/libs/1_38_0/doc/html/interprocess/acknowledgements_... waits for string conversion utilities which would be the additional layer you mentioned).
* 'handle' is the underlying native file handle of a stream (file descriptor on POSIX systems, handle on Windows). The concept is based on a class in the detail namespace of Boost.Process. There is just one class and it's not meant to be used by users of the library either. Thus I'm not sure what's the benefit of having an explicit 'handle' concept at all.
Isn't it in case Boost.Process doesn't implement the features I need, so that I may use them directly with the native functions?
Before I had added a handle() function to the stream classes there was no way to access a handle. Indeed I had added handle() because I wanted to use the file descriptor/Windows HANDLE for asynchronous I/O. I think I agree with you that it should be possible to access native handles. However I'm not sure if a concept has to be defined. Maybe I understand concept wrong but if I hear concept I think about extensibility and different types. In that sense a concept for an executable name makes sense to me (if it wasn't so difficult to support it). But I'm not sure how it fits to a handle? Boris
[...]

on Tue Apr 21 2009, "Boris Schaeling"
I've written an article about Boost.Process: http://www.highscore.de/cpp/process/
Hi Boris, Looks like a really useful library, and the docs are very readable. Sorry, I know this will seem trivial, and I see that you took care to note it on your webpage, but when posting about a library that is not yet in Boost, I'd appreciate it if you could make that clear. Just starting with "I've written an article about the proposed Boost.Process" could be enough, though it's also be good to get an indicator in the subject line. Thanks! -- Dave Abrahams BoostPro Computing http://www.boostpro.com

On Fri, 29 May 2009 18:10:23 +0200, David Abrahams
[...]Looks like a really useful library, and the docs are very readable. Sorry, I know this will seem trivial, and I see that you took care to note it on your webpage, but when posting about a library that is not yet in Boost, I'd appreciate it if you could make that clear. Just starting with "I've written an article about the proposed Boost.Process" could be enough, though it's also be good to get an indicator in the subject line.
ok, no problem. There have been so many Boost.Process forks last year though that it took me quite some time to merge them. Unless someone else really plans to take over responsibility for Boost.Process and doesn't change only minor things (thus creating another fork) I hope that this proposed Boost.Process library remains the only one. There is no problem if a differently designed Boost.Process library is proposed. But in the past years all the Boost.Process libraries were forks and you didn't really know which one to pick (which lead me to merge them all). Boris
participants (4)
-
Boris Schaeling
-
David Abrahams
-
joaquin@tid.es
-
Mathias Gaunard