[fiber] Fiber context switching during stack unwinding...
(Hopefully I'm not writing an essay for something that is a gcc/libstdc++/Linux bug) I've been using Boost.Context for a number of years now, to do something similar to what Boost.Fiber does. We recently upgraded our development target environment to Debian 9 (gcc 6.3), and moved up to Boost 1.62.0, so I've been looking at Boost.Fiber to replace how we're using Boost.Context (because it looks like it could greatly simplify our code, among other things). One thing I ran into with Boost.Context was that it interacted in some interesting ways with exception handling, and I haven't really seen any discussion about it. Specifically, switching contexts with an active exception (so during stack unwinding, or in a catch block) would result in weird behaviour in the other context (std::uncaught_exception() returning true, std::current_exception() returning the exception from the other context, try/catches in the new context causing new problems when switching back to the original context which is in the middle of stack unwinding, etc). I chalked that up to limitations of the context switching, though I haven't seen any mention of avoiding context switches with active exceptions; presumably this is some interaction with gcc's exception implementation, and would maybe need support from libunwind to save/restore the exception state (I don't recall if I checked if clang produced similar results, though I'll probably give it a go when I have the chance). So I wasn't surprised when I ran into similar problems with Boost.Fiber. I am wondering why this aspect of usage isn't discussed in any of the documentation; it would be nice if these libraries could work well with language features like exceptions, but I understand that the language spec doesn't really speak to the stack switching happening under-the-hood. It actually wouldn't surprise me that this wasn't even a problem in all environments, and just happens to be an artifact of the Linux implementation, but then we're either talking about unspecified or undefined behaviour, but would be nice to know. Even in the documents going through the C++ standards committee I don't see much mention of how things like coroutines should interact with something like std::current_exception() (perhaps because it is assumed to "just work" and doesn't need additional comment?) Anyone else ever run into issues related to this? Seems like any use of RAII for exception handling would easily run into problems. Mostly I've just found ways to work around it (e.g., not allowing the context switch if std::uncaught_exception() or std::current_exception() indicate it would be unsafe to do so, typically by throwing another exception) but this tends to make libraries harder to use correctly (e.g., can't really allow context switching in a destructor, or in a function called from a destructor, etc). Cheers Oliver
Hi, The documentation for boost context contains the following warning: "Do not jump from inside a catch block and then re-throw the exception in another continuation. " It's on this page: http://www.boost.org/doc/libs/1_66_0/libs/context/doc/html/context/cc.html Context switching with call/cc - 1.66.0 - boost.org<http://www.boost.org/doc/libs/1_66_0/libs/context/doc/html/context/cc.html> www.boost.org call/cc (call with current continuation) is a universal control operator (well-known from the programming language Scheme) that captures the current continuation as a ... ________________________________ From: Boost-users <boost-users-bounces@lists.boost.org> on behalf of Oliver Seiler via Boost-users <boost-users@lists.boost.org> Sent: Wednesday, February 28, 2018 6:45:16 AM To: boost-users@lists.boost.org Cc: Oliver Seiler Subject: [Boost-users] [fiber] Fiber context switching during stack unwinding... (Hopefully I'm not writing an essay for something that is a gcc/libstdc++/Linux bug) I've been using Boost.Context for a number of years now, to do something similar to what Boost.Fiber does. We recently upgraded our development target environment to Debian 9 (gcc 6.3), and moved up to Boost 1.62.0, so I've been looking at Boost.Fiber to replace how we're using Boost.Context (because it looks like it could greatly simplify our code, among other things). One thing I ran into with Boost.Context was that it interacted in some interesting ways with exception handling, and I haven't really seen any discussion about it. Specifically, switching contexts with an active exception (so during stack unwinding, or in a catch block) would result in weird behaviour in the other context (std::uncaught_exception() returning true, std::current_exception() returning the exception from the other context, try/catches in the new context causing new problems when switching back to the original context which is in the middle of stack unwinding, etc). I chalked that up to limitations of the context switching, though I haven't seen any mention of avoiding context switches with active exceptions; presumably this is some interaction with gcc's exception implementation, and would maybe need support from libunwind to save/restore the exception state (I don't recall if I checked if clang produced similar results, though I'll probably give it a go when I have the chance). So I wasn't surprised when I ran into similar problems with Boost.Fiber. I am wondering why this aspect of usage isn't discussed in any of the documentation; it would be nice if these libraries could work well with language features like exceptions, but I understand that the language spec doesn't really speak to the stack switching happening under-the-hood. It actually wouldn't surprise me that this wasn't even a problem in all environments, and just happens to be an artifact of the Linux implementation, but then we're either talking about unspecified or undefined behaviour, but would be nice to know. Even in the documents going through the C++ standards committee I don't see much mention of how things like coroutines should interact with something like std::current_exception() (perhaps because it is assumed to "just work" and doesn't need additional comment?) Anyone else ever run into issues related to this? Seems like any use of RAII for exception handling would easily run into problems. Mostly I've just found ways to work around it (e.g., not allowing the context switch if std::uncaught_exception() or std::current_exception() indicate it would be unsafe to do so, typically by throwing another exception) but this tends to make libraries harder to use correctly (e.g., can't really allow context switching in a destructor, or in a function called from a destructor, etc). Cheers Oliver
I had seen that line in the docs (though an explanation of why would be nice), but it really doesn't address the larger point that if you're writing good exception-safe code, then you *aren't* in a catch block most of the time. If I'm in a destructor, in particular, I can be there because the stack is being unwound and there is an active exception. If I then go and call, directly or indirectly, something that does a stack switch, then I'm in this (seemingly) undefined behaviour. Maybe you want to use ScopeExit in a fiber: void foo(channel<int>& c) { BOOST_SCOPE_EXIT_ALL(&c) { c.push(0); }; <other stuff that might throw> } Or maybe you are trying to compose a bunch of classes/functions that, unbeknownst to you (e.g., via a virtual function) wind up stack switching the current fiber, and these get called during stack unwinding. I don't see a good way to deal with this, at the present time, short of simply disabling exception entirely; what I find strange is that I see very little in the way of discussion around how things like std::uncaught_exception() or std::current_exception() do act and should act in conjunction with the various proposals for lightweight thread support (resumable functions, fibers, coroutines of various flavours, etc). I didn't really want to make my original e-mail even longer than it was, but beyond that one little warning in the Context documentation (but seemingly not in the Fiber documentation), the only other place I've seen mention of potential issues around this is in a pre-coroutine (resumable functions) paper that only says "The await operator shall not be invoked in a catch block of a try-statement", with a brief explanation in a footnote ( https://isocpp.org/files/papers/N4402.pdf); no mention restricting usage during stack unwinding, though. you could try to store the exeception in a std::exception_ptr ... leave the
catch clause and then use the std::exception_ptr to transport the exception to another context (via jump)
Again, I'm not really talking about being in the catch clause, or that I'd like to move the exception between contexts/fibers; I am concerned about APIs that don't work as expected in contexts (e.g., in a catch block, during stack unwinding with an active exception) that the calling code might not even be aware they are in. Imagine a logging API built on fibers written using one of the channel classes (e.g. log().push("my message"); ). Whether that is realistic or not, restricting usage outside of catch blocks, destructors, scope-guard APIs like ScopeExit, etc, seems pretty bad to me if it results in weird exception behaviour (and crashes, in my experience). On Tue, Feb 27, 2018 at 10:56 PM, Oliver Kowalke via Boost-users < boost-users@lists.boost.org> wrote:
Hi, I can attempt to answer "why" from a high-level point of view: I think is that it could be summarized as: the exception machinery is not reentrant. Throwing and catching exceptions performs a lot of messy manipulations with the data on the stack. During exception unwinding, you have to traverse the stack (a single-linked list) during which 1) a check is made at each level for whether the current stack frame has an exception handler, and if not 2) call destructors for local objects. Stack traversal is implemented by the runtime and even possibly relies on the OS' data structures (in case of Windows) and *might* use also thread-local storage (e.g., exception_ptr). Switching the stack aborts this process without informing the run-time exception machinery and.. what happens when you resume it? What if another exceptions has been thrown and handled on the same thread before resuming a suspended exception handler? A handler which is written with the assumption that only a single active invocation can exist on any particular thread? There are other concerns too: on Windows you can get asynchronous callbacks from the kernel (akin to unix signals, just nicer) and if you switch the stack when called from the kernel, you can really hose your process. Personally, I went the opposite route of yours: instead of disabling exceptions, I resisted all temptation to use fibers/stackful coroutines in production code as they seem too dangerous to use w/o full run-time support. Until that support is defined and implemented, they're just a cool hack (in a good sense of the word). ________________________________ From: Boost-users <boost-users-bounces@lists.boost.org> on behalf of Oliver Seiler via Boost-users <boost-users@lists.boost.org> Sent: Wednesday, February 28, 2018 9:27:51 PM To: boost-users@lists.boost.org Cc: Oliver Seiler Subject: Re: [Boost-users] [fiber] Fiber context switching during stack unwinding... I had seen that line in the docs (though an explanation of why would be nice), but it really doesn't address the larger point that if you're writing good exception-safe code, then you *aren't* in a catch block most of the time. If I'm in a destructor, in particular, I can be there because the stack is being unwound and there is an active exception. If I then go and call, directly or indirectly, something that does a stack switch, then I'm in this (seemingly) undefined behaviour. Maybe you want to use ScopeExit in a fiber: void foo(channel<int>& c) { BOOST_SCOPE_EXIT_ALL(&c) { c.push(0); }; <other stuff that might throw> } Or maybe you are trying to compose a bunch of classes/functions that, unbeknownst to you (e.g., via a virtual function) wind up stack switching the current fiber, and these get called during stack unwinding. I don't see a good way to deal with this, at the present time, short of simply disabling exception entirely; what I find strange is that I see very little in the way of discussion around how things like std::uncaught_exception() or std::current_exception() do act and should act in conjunction with the various proposals for lightweight thread support (resumable functions, fibers, coroutines of various flavours, etc). I didn't really want to make my original e-mail even longer than it was, but beyond that one little warning in the Context documentation (but seemingly not in the Fiber documentation), the only other place I've seen mention of potential issues around this is in a pre-coroutine (resumable functions) paper that only says "The await operator shall not be invoked in a catch block of a try-statement", with a brief explanation in a footnote (https://isocpp.org/files/papers/N4402.pdf); no mention restricting usage during stack unwinding, though. you could try to store the exeception in a std::exception_ptr ... leave the catch clause and then use the std::exception_ptr to transport the exception to another context (via jump) Again, I'm not really talking about being in the catch clause, or that I'd like to move the exception between contexts/fibers; I am concerned about APIs that don't work as expected in contexts (e.g., in a catch block, during stack unwinding with an active exception) that the calling code might not even be aware they are in. Imagine a logging API built on fibers written using one of the channel classes (e.g. log().push("my message"); ). Whether that is realistic or not, restricting usage outside of catch blocks, destructors, scope-guard APIs like ScopeExit, etc, seems pretty bad to me if it results in weird exception behaviour (and crashes, in my experience). On Tue, Feb 27, 2018 at 10:56 PM, Oliver Kowalke via Boost-users <boost-users@lists.boost.org<mailto:boost-users@lists.boost.org>> wrote: 2018-02-28 6:57 GMT+01:00 Stian Zeljko Vrba via Boost-users <boost-users@lists.boost.org<mailto:boost-users@lists.boost.org>>: Hi, The documentation for boost context contains the following warning: "Do not jump from inside a catch block and then re-throw the exception in another continuation. " It's on this page: http://www.boost.org/doc/libs/1_66_0/libs/context/doc/html/context/cc.html +1 ________________________________ From: Boost-users <boost-users-bounces@lists.boost.org<mailto:boost-users-bounces@lists.boost.org>> on behalf of Oliver Seiler via Boost-users <boost-users@lists.boost.org<mailto:boost-users@lists.boost.org>> Sent: Wednesday, February 28, 2018 6:45:16 AM To: boost-users@lists.boost.org<mailto:boost-users@lists.boost.org> Cc: Oliver Seiler Subject: [Boost-users] [fiber] Fiber context switching during stack unwinding... (Hopefully I'm not writing an essay for something that is a gcc/libstdc++/Linux bug) I've been using Boost.Context for a number of years now, to do something similar to what Boost.Fiber does. We recently upgraded our development target environment to Debian 9 (gcc 6.3), and moved up to Boost 1.62.0, so I've been looking at Boost.Fiber to replace how we're using Boost.Context (because it looks like it could greatly simplify our code, among other things). One thing I ran into with Boost.Context was that it interacted in some interesting ways with exception handling, and I haven't really seen any discussion about it. Specifically, switching contexts with an active exception (so during stack unwinding, or in a catch block) would result in weird behaviour in the other context (std::uncaught_exception() returning true, std::current_exception() returning the exception from the other context, try/catches in the new context causing new problems when switching back to the original context which is in the middle of stack unwinding, etc). I chalked that up to limitations of the context switching, though I haven't seen any mention of avoiding context switches with active exceptions; presumably this is some interaction with gcc's exception implementation, and would maybe need support from libunwind to save/restore the exception state (I don't recall if I checked if clang produced similar results, though I'll probably give it a go when I have the chance). So I wasn't surprised when I ran into similar problems with Boost.Fiber. I am wondering why this aspect of usage isn't discussed in any of the documentation; it would be nice if these libraries could work well with language features like exceptions, but I understand that the language spec doesn't really speak to the stack switching happening under-the-hood. It actually wouldn't surprise me that this wasn't even a problem in all environments, and just happens to be an artifact of the Linux implementation, but then we're either talking about unspecified or undefined behaviour, but would be nice to know. Even in the documents going through the C++ standards committee I don't see much mention of how things like coroutines should interact with something like std::current_exception() (perhaps because it is assumed to "just work" and doesn't need additional comment?) Anyone else ever run into issues related to this? Seems like any use of RAII for exception handling would easily run into problems. Mostly I've just found ways to work around it (e.g., not allowing the context switch if std::uncaught_exception() or std::current_exception() indicate it would be unsafe to do so, typically by throwing another exception) but this tends to make libraries harder to use correctly (e.g., can't really allow context switching in a destructor, or in a function called from a destructor, etc). Cheers Oliver you could try to store the exeception in a std::exception_ptr ... leave the catch clause and then use the std::exception_ptr to transport the exception to another context (via jump) _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org<mailto:Boost-users@lists.boost.org> https://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (3)
-
Oliver Kowalke -
Oliver Seiler -
Stian Zeljko Vrba