Re: [boost] Presentation of Boost.Process (SoC project)
----- Mensaje original ----- De: "Julio M. Merino Vidal" <jmmv84@gmail.com> Fecha: Miércoles, Agosto 16, 2006 5:31 pm Asunto: [boost] Presentation of Boost.Process (SoC project)
Hello,
During the past two-three months Jeff Garland (as mentor) and I have been working on the Boost Summer of Code project whose goal was to implement a library to spawn and manage child processes. A little introduction and a link to the source code can be found here:
https://www.boost-consulting.com:8443/trac/soc/wiki/process
But you'll probably want to go straight to the documentation which includes, among many other information, a couple of tutorials and multiple examples. I'm keeping an up-to-date snapshot at the following URL:
http://www.NetBSD.org/~jmmv/process/
If it wasn't for the imminent SoC deadline (next Monday), I wouldn't present it just yet because there are still a couple of important issues that I would like to see resolved as soon as possible:
First is the lack of a "design decisions" chapter in the documentation. This is not an extremely big issue because the Wiki page linked above already contains many discussions on the library's current design. In other words, that chapter will end up summarizing the information in there and adding few new things.
Second, and very important, is the "need" to integrate the library with Boost.Asio to allow asynchronous management of communication streams and process termination. Unfortunately it does not support such asynchronous events yet, and integrating with Boost.Asio sounds like a reasonable approach to cover this area.
Anyway, I wouldn't like this to discourage you from taking a look at the documentation and code. The library is already big enough to let you get a general idea on its focus and design. You can start criticising now! ;-) Sincerely, any constructive comments will be certainly appreciated.
Thank you.
Hello Julio, A cursory look at the docs shows this can be a very valuable contribution to Boost. Thank you for your effort! Although I plan to give a thorough review during the next week, let me advance a few issues: 1. The lib provides functionality to spawn child processes and examine them through launcher and child classes, respectively. Yet, it seems to lack functionality to examine the *current* process. In particular, it looks like one cannot get the handle_type of the current process (Win32:GetCurrentProcessID) in a manner compatible with child::get_handle. Would it be feasible to extend the lib to support this? In case it would, maybe it'd be a good idea then to rename "child" to a neutral "process", for instance, so that process classes are returned by laucher as well as by some potential "current_process" facility. 2. Would it be safe to guarantee that handle_type is an integral type? It is so at least for POSIX and Win32. This would allow the programmer to output handle_type's to std::ostreams's (for tracing purposes, for instance) and to use handle_type's in a number of scenarios, vg to store them in STL data structures. Joaquín M López Muñoz Telefónica, Investigación y Desarollo
On Fri, August 18, 2006 12:15 pm, JOAQUIN LOPEZ MU?Z wrote:
1. The lib provides functionality to spawn child processes and examine them through launcher and child classes, respectively. Yet, it seems to lack functionality to examine the *current* process. In particular, it looks like one cannot get the handle_type of the current process (Win32:GetCurrentProcessID) in a manner compatible with child::get_handle.
I agree. What I would also like to see is a good method of manipulating the environment variables of the current process. Sure, there's getenv(), but it's not exactly a great interface, and there is no portable way of setting one. But basically, when I pass environment variables to a child, I might want to make them a modified version of the current environment, and for that I'd expect the library to provide a way to access current variables.
2. Would it be safe to guarantee that handle_type is an integral type? It is so at least for POSIX and Win32. This would allow the programmer to output handle_type's to std::ostreams's (for tracing purposes, for instance) and to use handle_type's in a number of scenarios, vg to store them in STL data structures.
Win32's HANDLE is a pointer type. Does that count as integral? Excellent work on the library, all in all. Sebastian Redl
On 8/18/06, Sebastian Redl <sebastian.redl@getdesigned.at> wrote:
On Fri, August 18, 2006 12:15 pm, JOAQUIN LOPEZ MU?Z wrote:
1. The lib provides functionality to spawn child processes and examine them through launcher and child classes, respectively. Yet, it seems to lack functionality to examine the *current* process. In particular, it looks like one cannot get the handle_type of the current process (Win32:GetCurrentProcessID) in a manner compatible with child::get_handle.
I agree. What I would also like to see is a good method of manipulating the environment variables of the current process. Sure, there's getenv(), but it's not exactly a great interface, and there is no portable way of setting one.
Hmm... but does that belong to this library? It is easy to do by introducing a new class that represents the current process. Also, is it really a good idea to change the current process' environment? I like how Boost.Filesystem's "forbids" to change the current working directory very much, and environment variables seem very similar to this.
But basically, when I pass environment variables to a child, I might want to make them a modified version of the current environment, and for that I'd expect the library to provide a way to access current variables.
Some changes can be made to accomodate this. E.g., having a get_environment(var) method in the launcher. Not very elegant at first sight, though. Or... decouple the environment from the launcher, make it an independent (and public class) which you can instantiate (creating a snapshot of the current values), modify and pass to a new process: bp::environment e; e.set("var", e.get("var") + foo); bp::launcher l; l.set_environment(e); (Turning the launcher's start method into a free function and making the launcher simply the process' execution "context" as somebody else suggested gives a lot more room for improvement here.)
2. Would it be safe to guarantee that handle_type is an integral type? It is so at least for POSIX and Win32. This would allow the programmer to output handle_type's to std::ostreams's (for tracing purposes, for instance) and to use handle_type's in a number of scenarios, vg to store them in STL data structures.
Win32's HANDLE is a pointer type. Does that count as integral?
I don't think so. You cannot treat it as an integer in all cases without casts. But see my reply to Joaquín; HANDLEs are most probably used as identifiers when they really shouldn't be.
Excellent work on the library, all in all.
Thanks :) -- Julio M. Merino Vidal <jmmv84@gmail.com> The Julipedia - http://julipedia.blogspot.com/
On 8/18/06, JOAQUIN LOPEZ MU?Z <joaquin@tid.es> wrote:
Hello Julio,
A cursory look at the docs shows this can be a very valuable contribution to Boost. Thank you for your effort!
You're welcome :-)
Although I plan to give a thorough review during the next week, let me advance a few issues:
1. The lib provides functionality to spawn child processes and examine them through launcher and child classes, respectively. Yet, it seems to lack functionality to examine the *current* process. In particular, it looks like one cannot get the handle_type of the current process (Win32:GetCurrentProcessID) in a manner compatible with child::get_handle. Would it be feasible to extend the lib to support this? In case it would, maybe it'd be a good idea then to rename "child" to a neutral "process", for instance, so that process classes are returned by laucher as well as by some potential "current_process" facility.
I started the project thinking on a library to manage children processes as well as currently running ones (not only "self"). This turns out to be very, very complex portability-wise, and so Jeff convinced me to simplify it to child process management only. Most of the code used to support those ideas was not well thought and was removed recently. *However*, adding support for the specific feature you mention is relatively easy. I'd even say very easy. The idea is to introduce a new "process" class from which the existing "child" inherits. The parent could hold the identifier and provide additional functions such as "kill". This could let you do: bp::process self = bp::get_current_process(); bp::child c = l.start(cl); ... bp::process::handle_type a = c.get_id(); bp::process::handle_type b = self.get_id(); ... c.kill(); self.kill(); Note that the existing "child" cannot magically become "process" because there are some details not available for a non-child process: get_stdin et. al.
2. Would it be safe to guarantee that handle_type is an integral type? It is so at least for POSIX and Win32. This would allow the programmer to output handle_type's to std::ostreams's (for tracing purposes, for instance) and to use handle_type's in a number of scenarios, vg to store them in STL data structures.
Mmm... aren't Win32's HANDLEs pointers? There is something that makes me doubt though. Under Win32, processes have both an identifier and a handle. Based on what I outlined above and the GetCurrentProcessID call you mention, I'm afraid I'm incorrectly using the HANDLE as the process identifier. What I mean is: the new "process" class might need to hold the process identifier (which for a child process is the dwProcessId field) and the "child" class could additionally keep the child's HANDLE to allow interaction with it. Cheers, PS: On a fourth read of this mail, I'm now fairly sure my code is wrong. HANDLEs are used to access the system, but are specific to the process. This certainly defeats your desire to print identifiers to ostreams for debug purposes because a process' HANDLEs could not be compared with anything. PSS: If I didn't explain myself correctly enough to make the above understandable, please let me know ;-) -- Julio M. Merino Vidal <jmmv84@gmail.com> The Julipedia - http://julipedia.blogspot.com/
Julio M. Merino Vidal <jmmv84 <at> gmail.com> writes:
On 8/18/06, JOAQUIN LOPEZ MU?Z <joaquin <at> tid.es> wrote:
[...]
1. The lib provides functionality to spawn child processes and examine them through launcher and child classes, respectively. Yet, it seems to lack functionality to examine the *current* process. [...] I started the project thinking on a library to manage children processes as well as currently running ones (not only "self"). This turns out to be very, very complex portability-wise, and so Jeff convinced me to simplify it to child process management only. Most of the code used to support those ideas was not well thought and was removed recently.
*However*, adding support for the specific feature you mention is relatively easy. I'd even say very easy. The idea is to introduce a new "process" class from which the existing "child" inherits. The parent could hold the identifier and provide additional functions such as "kill". This could let you do:
bp::process self = bp::get_current_process(); bp::child c = l.start(cl); ... bp::process::handle_type a = c.get_id(); bp::process::handle_type b = self.get_id(); ... c.kill(); self.kill();
Note that the existing "child" cannot magically become "process" because there are some details not available for a non-child process: get_stdin et. al.
Why not? Aren't these obtainable from ::stdin, ::stdout and ::stderr or similar when the process is the current one?
2. Would it be safe to guarantee that handle_type is an integral type? It is so at least for POSIX and Win32. This would allow the programmer to output handle_type's to std::ostreams's (for tracing purposes, for instance) and to use handle_type's in a number of scenarios, vg to store them in STL data structures.
Mmm... aren't Win32's HANDLEs pointers?
There is something that makes me doubt though. Under Win32, processes have both an identifier and a handle. Based on what I outlined above and the GetCurrentProcessID call you mention, I'm afraid I'm incorrectly using the HANDLE as the process identifier.
What I mean is: the new "process" class might need to hold the process identifier (which for a child process is the dwProcessId field) and the "child" class could additionally keep the child's HANDLE to allow interaction with it.
If you hace a Win32 process ID, obtaining a HANDLE can be done via OpenProcess, so you can have a (id,handle) pair both for the child and the self cases. The only difference would be in how the relevant info is gathered (via GetCurrentProcessID and OpenProcess for self, via CreateProcess for child.)
Cheers,
PS: On a fourth read of this mail, I'm now fairly sure my code is wrong. HANDLEs are used to access the system, but are specific to the process. This certainly defeats your desire to print identifiers to ostreams for debug purposes because a process' HANDLEs could not be compared with anything.
So, if I'm getting you, you'll change your Win32 code so that handle_type is a DWORD and can then address my requirement that a guarantee is made about handle_type being always an integral type, right? Additionally, you can make the guarantee that handle_type objects are system-wide, which might have some application for interprocess identification. Best, Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
On 8/22/06, Joaquin M Lopez Munoz <joaquin@tid.es> wrote:
Julio M. Merino Vidal <jmmv84 <at> gmail.com> writes:
Note that the existing "child" cannot magically become "process" because there are some details not available for a non-child process: get_stdin et. al.
Why not? Aren't these obtainable from ::stdin, ::stdout and ::stderr or similar when the process is the current one?
Oh, sure. But I'd like to leave enough room for further expansion to "generic process management" (i.e., manage any running process). In that case, the "process" would be the most generic process representation, from which "child" could inherit. We'd then have a "current_process" class or something that inherits from "process" too.
So, if I'm getting you, you'll change your Win32 code so that handle_type is a DWORD and can then address my requirement that a guarantee is made about handle_type being always an integral type, right?
Yes, that's what I'm thinking.
Additionally, you can make the guarantee that handle_type objects are system-wide, which might have some application for interprocess identification.
Indeed. Thanks, -- Julio M. Merino Vidal <jmmv84@gmail.com> The Julipedia - http://julipedia.blogspot.com/
On Tue, August 22, 2006 10:36 am, Julio M. Merino Vidal wrote:
So, if I'm getting you, you'll change your Win32 code so that handle_type is a DWORD and can then address my requirement that a guarantee is made about handle_type being always an integral type, right?
Yes, that's what I'm thinking.
Wait a moment, does that mean that the internal code would cast between DWORD and HANDLE? That's absolutely invalid: they're not the same size on 64-bit systems. You'd have to use DWORD_PTR instead (an unsigned integral type guaranteed to be the same size as a pointer). Sebastian Redl
On 8/22/06, Sebastian Redl <sebastian.redl@getdesigned.at> wrote:
On Tue, August 22, 2006 10:36 am, Julio M. Merino Vidal wrote:
So, if I'm getting you, you'll change your Win32 code so that handle_type is a DWORD and can then address my requirement that a guarantee is made about handle_type being always an integral type, right?
Yes, that's what I'm thinking.
Wait a moment, does that mean that the internal code would cast between DWORD and HANDLE? That's absolutely invalid: they're not the same size on 64-bit systems. You'd have to use DWORD_PTR instead (an unsigned integral type guaranteed to be the same size as a pointer).
No. The code could expose identifiers (DWORDs) where it currently exposes HANDLEs, but no conversion could happen between them. Even more, it could expose both: class process { public: id_type get_id(); // DWORD in Win32, int in POSIX handle_type get_handle(); // HANDLE in Win32, int in POSIX }; These could return different values in Win32 but the same integer under POSIX. I believe it makes sense because these are different things conceptually even if the difference does not exist in Unix. -- Julio M. Merino Vidal <jmmv84@gmail.com> The Julipedia - http://julipedia.blogspot.com/
participants (4)
-
JOAQUIN LOPEZ MU?Z -
Joaquin M Lopez Munoz
-
Julio M. Merino Vidal -
Sebastian Redl