Hi boosters, My name is Moncef Mechri (aka "fireboot" on IRC). I'm a 20 years old french student in my final undergraduate year at the Faculté des Sciences de Luminy, Marseille, France. I would like to apply as a student for the Boost.Process SoC idea. My main domains of interest include C and C++ programming languages, system programming (especially on POSIX systems), parallel computing, generic programming, object-oriented design, software optimizations, graph theory, and UI development (using Qt). So far I've used a couple of boost librairies : Boost.Thread, Boost.Random, some features from Boost.Interprocess (principally atomic operations), Boost.SmartPointers, Boost.Foreach and Boost.Bind. They were all a pleasure to use and it's why I've just written a (french) paper about one of them : Boost.thread [1]. I'll would be glad to help improving the set of high quality libraries that Boost is. I have some ideas to help finishing Boost.Process and I'd be happy to hear your opinions about them: 1) Abstract the string type used to name executables. This could be done for example using a traits class 2) Move find_executable_in_path() and executable_to_progname() to Boost.Filesystem. I think their job is filesystem-related so maybe it would be wise to move them accordingly. 3) On POSIX systems, provide raise(3) and kill(3) functions to respectively send a signal to the current process and send a signal to a child. Sending signals to unrelated process should probably not take place in Boost.Process but rather in Boost.Interprocess 4) On POSIX systems, when a process dies it becomes a zombie process until wait() is called. Since wait() blocks, things may become tricky if there are a lot of child process. When a process die, the SIGCHLD signal is sent to his creator. It might be nice to use that by setting up a SIGCHLD signal handler which calls wait() when this signal is received. Since a similar problem exists on Windows, Boost.Process should provide a similar solution on this platform, but I've not yet thought about a possible design for it. It might be also useful to provide a way to activate/deactivate this feature, for example with a macro BOOST_MANAGES_DEAD_PROCESS 5) Right now, boost::process::wait_children() returns the first exit code which is not equal to EXIT_SUCCESS, and if there are more than one process which finished abnormally, return values of those child processes are ignored. I think we could improve that by returning for example a vector<boost::process::status> instead of just one boost::process::status. Doing that, we can retrieve all the status codes. It might be useful to define a way to toggle on/off 6) Actually in Boost.Process, if the developer wants to be able to do asynchronous operations on Windows he has to define the BOOST_PROCESS_WINDOWS_USE_NAMED_PIPE macro. By doing so, it makes Boost.Process on Windows create named pipe for all streams even if asynchronous operations are not used in all of them. First of all I propose to add a boost::process::anonymous_pipe class to Boost.Process, and a boost::process::named_pipe class to Boost.Interprocess (because named pipe can be used between unrelated process). After that the developer should be able to choose between anonymous pipe or named pipe on a per-stream basis (but of course asynchronous operations will only work on streams using named pipes). Thanks for reading, it'd be nice if someone accept to comment my ideas Moncef Mechri. [1] http://fz-corp.net/?p=505
On Thu, 25 Mar 2010 04:15:05 +0100, Moncef Mechri <moncef.mechri@gmail.com> wrote: Hi Moncef,
[...]I would like to apply as a student for the Boost.Process SoC idea.
you are definitely welcome!
[...]1) Abstract the string type used to name executables. This could be done for example using a traits class
I'm not really sure if it's worth doing: - There are currently so many other problems to think about that I wouldn't care about the Executable concept for now. - Other libraries like Boost.Interprocess simply stick to std::string, too. - The question how to support different string types and encodings should probably be answered by a different library anyway (something like Boost.Unicode).
2) Move find_executable_in_path() and executable_to_progname() to Boost.Filesystem. I think their job is filesystem-related so maybe it would be wise to move them accordingly.
In general I don't care where the code ends up as long as it's obvious to Boost.Process users that there is code in another library which they might want to use. For a possible GSoC project I think we should concentrate on Boost.Process only. Discussing with other library maintainers whether we can move code to their libraries can still be done later. Then we can concentrate on designing and implementing for now (to make sure this library is ever finished and reviewed :).
3) On POSIX systems, provide raise(3) and kill(3) functions to respectively send a signal to the current process and send a signal to a child. Sending signals to unrelated process should probably not take place in Boost.Process but rather in Boost.Interprocess
There is also some signal-related code in Boost.Test (although I have no idea if this is something we could use for Boost.Process).
4) On POSIX systems, when a process dies it becomes a zombie process until wait() is called. Since wait() blocks, things may become tricky if there are a lot of child process. When a process die, the SIGCHLD signal is sent to his creator. It might be nice to use that by setting up a SIGCHLD signal handler which calls wait() when this signal is received.
I think you describe an asynchronous wait() which is something we definitely need. :)
[...]5) Right now, boost::process::wait_children() returns the first exit code which is not equal to EXIT_SUCCESS, and if there are more than one process which finished abnormally, return values of those child processes are ignored. I think we could improve that by returning for example a vector<boost::process::status> instead of just one boost::process::status. Doing that, we can retrieve all the status codes. It might be useful to define a way to toggle on/off
No comment except that creating and managing pipelines is indeed another topic we have to think about.
6) Actually in Boost.Process, if the developer wants to be able to do asynchronous operations on Windows he has to define the BOOST_PROCESS_WINDOWS_USE_NAMED_PIPE macro. By doing so, it makes Boost.Process on Windows create named pipe for all streams even if asynchronous operations are not used in all of them.
I agree it must be changed (adding the macro was a quick fix as asynchronous I/O didn't work at all on Windows before). Boris
Philipe, thanks for your notification !
I'm not really sure if it's worth doing: - There are currently so many other problems to think about that I wouldn't care about the Executable concept for now. - Other libraries like Boost.Interprocess simply stick to std::string, too. - The question how to support different string types and encodings should probably be answered by a different library anyway (something like Boost.Unicode).
You're probably right, it's probably not the bigger problem to care about and it probably doesn't worth it.
3) On POSIX systems, provide raise(3) and kill(3) functions to
respectively send a signal to the current process and send a signal to a child. Sending signals to unrelated process should probably not take place in Boost.Process but rather in Boost.Interprocess
There is also some signal-related code in Boost.Test (although I have no idea if this is something we could use for Boost.Process).
Thanks for the info; I'm going to check what signal-related code Boost.test has. Moreover, I don't think Boost.Process should provide platform-independent signals mechanisms, since Windows only supports ANSI signals (which are very (very very) limited).
4) On POSIX systems, when a process dies it becomes a zombie process until
wait() is called. Since wait() blocks, things may become tricky if there are a lot of child process. When a process die, the SIGCHLD signal is sent to his creator. It might be nice to use that by setting up a SIGCHLD signal handler which calls wait() when this signal is received.
I think you describe an asynchronous wait() which is something we definitely need. :)
Yeah :). What do you think about the idea to set up an handler which calls wait() when a SIGCHLD signal is received ? Otherwise, what's your vision of this asynchronous wait() ?
[...]5) Right now, boost::process::wait_children() returns the first exit
code
which is not equal to EXIT_SUCCESS, and if there are more than one process which finished abnormally, return values of those child processes are ignored. I think we could improve that by returning for example a vector<boost::process::status> instead of just one boost::process::status. Doing that, we can retrieve all the status codes. It might be useful to define a way to toggle on/off
No comment except that creating and managing pipelines is indeed another topic we have to think about.
This means that you're ok with this proposition ?
6) Actually in Boost.Process, if the developer wants to be able to do
asynchronous operations on Windows he has to define the BOOST_PROCESS_WINDOWS_USE_NAMED_PIPE macro. By doing so, it makes Boost.Process on Windows create named pipe for all streams even if asynchronous operations are not used in all of them.
I agree it must be changed (adding the macro was a quick fix as asynchronous I/O didn't work at all on Windows before).
I'm going to thing about a concrete proposition to solve this issue Have you any other suggestions ? Thanks for your answer Boris :) Moncef Mechri
On Fri, 26 Mar 2010 03:58:57 +0100, Moncef Mechri <moncef.mechri@gmail.com> wrote:
[...]Yeah :). What do you think about the idea to set up an handler which calls wait() when a SIGCHLD signal is received ? Otherwise, what's your vision of this asynchronous wait() ?
From a user's point of view an asynchronous wait() ideally behaves like any other asynchronous operation supported by Boost.Asio. How the implementation looks like depends then on the platform (on POSIX systems I agree we could and maybe have to catch SIGCHLD).
[...]5) Right now, boost::process::wait_children() returns the first exit
code
which is not equal to EXIT_SUCCESS, and if there are more than one process which finished abnormally, return values of those child processes are ignored. I think we could improve that by returning for example a vector<boost::process::status> instead of just one boost::process::status. Doing that, we can retrieve all the status codes. It might be useful to define a way to toggle on/off
No comment except that creating and managing pipelines is indeed another topic we have to think about.
This means that you're ok with this proposition ?
Yes, no problem (mainly because I'm more concerned about overall design decisions for now :-).
6) Actually in Boost.Process, if the developer wants to be able to do
asynchronous operations on Windows he has to define the BOOST_PROCESS_WINDOWS_USE_NAMED_PIPE macro. By doing so, it makes Boost.Process on Windows create named pipe for all streams even if asynchronous operations are not used in all of them.
I agree it must be changed (adding the macro was a quick fix as asynchronous I/O didn't work at all on Windows before).
I'm going to thing about a concrete proposition to solve this issue
The problem here is that we need an I/O object which needs to be bound to an I/O service object (not sure how familiar you are with Boost.Asio?). Now what's the I/O object in Boost.Process? Shall we think of a process as an I/O object? Can we expect developers to prefer using asynchronous I/O just as they do with sockets for example? If we don't want to force developers to create an I/O service object whenever they work with Boost.Process we need to find another mechanism to turn a process object into an I/O object later (after it has been constructed). This again is something which hasn't been done yet in any Boost library (all other objects support asynchronous I/O right after construction). I think it all comes down whether we want to treat a process like any other I/O object. While noone complains about creating I/O service objects when you use sockets I'm not sure about processes? Developers might be surprised if Boost.Process is coupled with Boost.Asio? Boris
Hi, Boris Schaeling wrote:
On Fri, 26 Mar 2010 03:58:57 +0100, Moncef Mechri <moncef.mechri@gmail.com> wrote:
[...]Yeah :). What do you think about the idea to set up an handler which calls wait() when a SIGCHLD signal is received ? Otherwise, what's your vision of this asynchronous wait() ?
From a user's point of view an asynchronous wait() ideally behaves like any other asynchronous operation supported by Boost.Asio. How the implementation looks like depends then on the platform (on POSIX systems I agree we could and maybe have to catch SIGCHLD).
[...]5) Right now, boost::process::wait_children() returns the first exit
code
which is not equal to EXIT_SUCCESS, and if there are more than one process which finished abnormally, return values of those child processes are ignored. I think we could improve that by returning for example a vector<boost::process::status> instead of just one boost::process::status. Doing that, we can retrieve all the status codes. It might be useful to define a way to toggle on/off
No comment except that creating and managing pipelines is indeed another topic we have to think about.
This means that you're ok with this proposition ?
Yes, no problem (mainly because I'm more concerned about overall design decisions for now :-).
6) Actually in Boost.Process, if the developer wants to be able to do
asynchronous operations on Windows he has to define the BOOST_PROCESS_WINDOWS_USE_NAMED_PIPE macro. By doing so, it makes Boost.Process on Windows create named pipe for all streams even if asynchronous operations are not used in all of them.
I agree it must be changed (adding the macro was a quick fix as asynchronous I/O didn't work at all on Windows before).
I'm going to thing about a concrete proposition to solve this issue
The problem here is that we need an I/O object which needs to be bound to an I/O service object (not sure how familiar you are with Boost.Asio?). Now what's the I/O object in Boost.Process? Shall we think of a process as an I/O object? Can we expect developers to prefer using asynchronous I/O just as they do with sockets for example? If we don't want to force developers to create an I/O service object whenever they work with Boost.Process we need to find another mechanism to turn a process object into an I/O object later (after it has been constructed). This again is something which hasn't been done yet in any Boost library (all other objects support asynchronous I/O right after construction).
The IO Service for Boost.Process would have to create a thread to handle the signals. However, I think the Boost.ASIO implementation details already does something to block signals, this cannot be ignored! Anyway, see *sigemptyset*, *sigaddset* and *sigwait*. This API's allows us to handle signals in a sane manner :)
I think it all comes down whether we want to treat a process like any other I/O object. While noone complains about creating I/O service objects when you use sockets I'm not sure about processes? Developers might be surprised if Boost.Process is coupled with Boost.Asio?
Boris
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Fri, 26 Mar 2010 13:35:53 +0100, Bruno Santos <bsantos@av.it.pt> wrote:
[...]The IO Service for Boost.Process would have to create a thread to handle the signals. However, I think the Boost.ASIO implementation details already does something to block signals, this cannot be ignored! Anyway, see *sigemptyset*, *sigaddset* and *sigwait*. This API's allows us to handle signals in a sane manner :)
For the record: There is a feature request for a Boost.Asio extension and an implementation to handle signals at <https://svn.boost.org/trac/boost/ticket/2879>. Boris
The problem here is that we need an I/O object which needs to be bound to an I/O service object (not sure how familiar you are with Boost.Asio?). Now what's the I/O object in Boost.Process? Shall we think of a process as an I/O object? Can we expect developers to prefer using asynchronous I/O just as they do with sockets for example? If we don't want to force developers to create an I/O service object whenever they work with Boost.Process we need to find another mechanism to turn a process object into an I/O object later (after it has been constructed). This again is something which hasn't been done yet in any Boost library (all other objects support asynchronous I/O right after construction).
I think it all comes down whether we want to treat a process like any other I/O object. While noone complains about creating I/O service objects when you use sockets I'm not sure about processes? Developers might be surprised if Boost.Process is coupled with Boost.Asio?
I'm not yet familiar with boost.asio, I'm going to dig further into it today, thanks for the idea :) Moreover, Boris, I saw you've made a recent update to the handling signals proposal in asio. I'll check this out too.
On Mon, 29 Mar 2010 14:59:45 +0300, Moncef Mechri <moncef.mechri@gmail.com> wrote:
[...]Moreover, Boris, I saw you've made a recent update to the handling signals proposal in asio. I'll check this out too.
Unfortunately I didn't get an email two months ago when someone complained about the implementation. I hope the implementation is now correct. Boris
Boris Schaeling wrote:
On Mon, 29 Mar 2010 14:59:45 +0300, Moncef Mechri <moncef.mechri@gmail.com> wrote:
[...]Moreover, Boris, I saw you've made a recent update to the handling signals proposal in asio. I'll check this out too.
Unfortunately I didn't get an email two months ago when someone complained about the implementation. I hope the implementation is now correct.
That was me who commented two months ago. Just wonder, Boris, what made you reimplement my original proposal? You new version is no longer uses a mutex inside a signal handler. It still improperly handles errno, doesn't care about blocking inside a signal handler, doesn't care about setting FD_CLOEXEC, doesn't pass siginfo to the user, doesn't let the user to associate a signal with a file descriptor. Besides, it uses a thread, a mutex, a condvar, a shared_ptr, a weak_ptr, a deque and a map. None of this is required. You can have a look at this implementation http://dgoncharov.github.com/libunixsignal (online docs) and http://github.com/dgoncharov/libunixsignal and compare. Regards, Dmitry
On Mon, 29 Mar 2010 18:03:36 +0300, Dmitry Goncharov <dgoncharov@unison.com> wrote:
[...]Just wonder, Boris, what made you reimplement my original proposal?
As we probably should start a new thread to discuss the signal handler I try to keep my answer short. But the main reason why I reimplemented the signal handler was that a Boost.Asio extension should follow Boost.Asio patterns (assuming that the goal is to improve chances that Christopher ships a signal handler with Boost.Asio one day).
You new version is no longer uses a mutex inside a signal handler. It still improperly handles errno, doesn't care about blocking inside a signal handler, doesn't care about setting FD_CLOEXEC, doesn't pass siginfo to the user, doesn't let the user to associate a signal with a file descriptor. Besides, it uses a thread, a mutex, a condvar, a shared_ptr, a weak_ptr, a deque and a map. None of this is required.
I welcome test cases to improve the implementation (and of course patches). However your signal handler for example doesn't support a synchronous wait. As all other Boost.Asio I/O objects support both synchronous and asynchronous I/O there is something fundamental missing. While my signal handler's implementation can surely be improved your signal handler's interface doesn't even fulfill basic requirements of a Boost.Asio I/O object. Boris
Hi, just wanted to nice you that there's a typo in there: void foo(string&amp; str) Philippe
participants (5)
-
Boris Schaeling -
Bruno Santos -
Dmitry Goncharov -
Moncef Mechri -
Philippe Vaucher