I have built a C++ library using boost ASIO. The library needs to be both
thread-safe and fork-safe.
It has service scheduler thread, which calls "_io_service.run())". To
support fork-safety, I've registered pre_fork, post_fork_parent and
post_fork_child handlers. The pre_fork() handler, calls
"_io_service.notify_fork(boost::io_service:fork_prepare();",
post_fork_parent handler calls
"_io_service.notify_fork(boost::asio::io_service::fork_parent);" and
post_fork_child calls
"_io_service.notify_fork(boost::asio::io_service::fork_child);".
The problem I'm facing in, when the fork() happens, the service scheduler
thread might be in middle of some operation and might have acquired lock on
data members of io_service object. So, the child process seems them in the
same state and in the post_fork_child() when we call
"_io_service.notify_fork(boost::asio::io_service::fork_child);" it tries to
acquire the lock on the same object and hence get blocked indefinitely (as
there is no thread in child to release the lock).
The stack trace I see in the child process, which is blocked, is -
fffffd7ffed07577 lwp_park (0, 0, 0)
fffffd7ffecffc18 mutex_lock_internal () + 378
fffffd7ffecfffb2 mutex_lock_impl () + 112
fffffd7ffed0007b mutex_lock () + b
fffffd7fff26419d
__1cFboostEasioGdetailLscoped_lock4n0CLposix_mutex__2t5B6Mrn0D__v_ () + 1d
fffffd7fff2866a2
__1cFboostEasioGdetailQdev_poll_reactorMfork_service6Mn0BKio_serviceKfork_event__v_
() + 32
fffffd7fff278527
__1cFboostEasioGdetailQservice_registryLnotify_fork6Mn0BKio_serviceKfork_event__v_
() + 107
fffffd7fff27531c __1cDdesGtunnelQServiceSchedulerPpost_fork_child6M_v_ () +
1c
fffffd7fff29de24 post_fork_child () + 84
fffffd7ffec92188 _postfork_child_handler () + 38
fffffd7ffecf917d fork () + 12d
fffffd7ffec172d5 fork () + 45
fffffd7ffef94309 fork () + 9
000000000043299d main () + 67d
0000000000424b2c ???????? ()
Apparently the "dev_poll_reactor" is locked (because it seems to be
dispatching some pending events) in the service scheduler thread when the
fork has happened which is causing the problem.
I think to solve the problem, I need to ensure that service scheduler thread
is not in the middle of any processing when the fork happens and one way to
guarantee that would be to call "io_service.stop()" in pre_fork() handler
but that doesn't sound like a good solution. Could you please let me know
what is the right approach to make the library fork safe?
The code snippets looks something like this.
/**
* Combines Boost.ASIO with a thread for scheduling.
*/
class ServiceScheduler : private boost::noncopyable
{
public :
/// The actual thread used to perform work.
boost::shared_ptrboost::thread _service_thread;
/// Service used to manage async I/O events
boost::asio::io_service _io_service;
/// Work object to block the ioservice thread.
std::auto_ptrboost::asio::io_service::work _work;
...
};
/**
* CTOR
*/
ServiceScheduler::ServiceScheduler()
: _logger("ServiceScheduler"),
_io_service(),
_work(std::auto_ptrboost::asio::io_service::work(
new boost::asio::io_service::work(_io_service))),
_is_running(false)
{
}
/**
* Starts a thread to run async I/O service to process the scheduled work.
*/
void ServiceScheduler::start()
{
ScopedLock scheduler_lock(_mutex);
if (!_is_running) {
_is_running = true;
_service_thread = boost::shared_ptrboost::thread(
new boost::thread(boost::bind(
&ServiceScheduler::processServiceWork, this)));
}
}
/**
* Processes work passed to the ASIO service and handles uncaught
* exceptions
*/
void ServiceScheduler::processServiceWork()
{
try {
std::cout<<"Running io_service"<