Non-thread-safe thread library (Boost.Function)
Dear All, In November Sergey Skorniakov wrote to this list asking about the non-thread-safety of Boost.Function and consequently of Boost.Thread itself. Steven Watanbee replied pointing out that the problem is resolved in the trunk. See: http://thread.gmane.org/gmane.comp.lib.boost.devel/167877 My thought at the time was "chocolate teapot". I put understanding this issue on my to-do list and it has finally got to the top. However, looking at the trac pages that Steven linked to, I am none the wiser; they contain patches but with no comments in the code and don't mention thread safety in the commit messages. So can someone please help me understand the scope of this problem? i.e. - Are all uses of Boost.Function affected, or only certain uses (e.g. member functions vs. free functions)? - What exactly is the race between? E.g. if I know that I'll never create two threads at the same time, do I still need to worry about other uses of Boost.Function that may be concurrent? Is the race between creating the function objects or between using them, or both? - Can I avoid the problem by using a new version of Boost.Function, without upgrading other libraries? (Is Boost.Function header-only?) - What would the symptom of failure be? I'm currently debugging a double-free somewhere below std::string::assign, which is probably unrelated. I'm also curious to know how much more serious than this a bug would need to be before a Boost release would be recalled, or a new point-release released, or an announcement made on the Boost web page (which links to bugs fixed in 1.34.1, but not bugs introduced). Surely there aren't many users of Boost.Thread who are happy to use a non-thread-safe thread library..... Many thanks, Phil.
Phil Endecott <spam_from_boost_dev <at> chezphil.org> writes:
In November Sergey Skorniakov wrote to this list asking about the non-thread-safety of Boost.Function and consequently of Boost.Thread itself. Steven Watanbee replied pointing out that the problem is resolved in the trunk.
Irrespective of any fixes applied to boost.Function, this has been fixed in the trunk --- boost.thread no longer depends on boost.function. Anthony
Anthony Williams wrote:
Phil Endecott <spam_from_boost_dev <at> chezphil.org> writes:
In November Sergey Skorniakov wrote to this list asking about the non-thread-safety of Boost.Function and consequently of Boost.Thread itself. Steven Watanbee replied pointing out that the problem is resolved in the trunk.
Irrespective of any fixes applied to boost.Function, this has been fixed in the trunk --- boost.thread no longer depends on boost.function.
It was fixed on the trunk for Boost.Function long ago. It's interesting that the thread library no longer depends on 'function' (at least, for launching threads); I see how the change was done (using virtual functions, which was once the implementation trick used in Function), but I'm curious as to why? Function has some storage optimizations that will tend to make it more efficient that the basic virtual function/templated derived class idiom. - Doug
Douglas Gregor <dgregor <at> osl.iu.edu> writes:
Anthony Williams wrote:
Irrespective of any fixes applied to boost.Function, this has been fixed in the trunk --- boost.thread no longer depends on boost.function.
It was fixed on the trunk for Boost.Function long ago. It's interesting that the thread library no longer depends on 'function' (at least, for launching threads); I see how the change was done (using virtual functions, which was once the implementation trick used in Function), but I'm curious as to why? Function has some storage optimizations that will tend to make it more efficient that the basic virtual function/templated derived class idiom.
I wanted the interface to match that being proposed for C++0x, which has the constructor being a template. I was considering using boost::function in the implementation, but I never got round to it. Anthony
Phil Endecott wrote:
Dear All,
In November Sergey Skorniakov wrote to this list asking about the non-thread-safety of Boost.Function and consequently of Boost.Thread itself. Steven Watanbee replied pointing out that the problem is resolved in the trunk. See: http://thread.gmane.org/gmane.comp.lib.boost.devel/167877
My thought at the time was "chocolate teapot". I put understanding this issue on my to-do list and it has finally got to the top. However, looking at the trac pages that Steven linked to, I am none the wiser; they contain patches but with no comments in the code and don't mention thread safety in the commit messages.
The root of the problem was some dynamic initialization in the vtable code, which is rarely (if ever?) thread-safe. The problem is that the compiler turns code like this: static T some_value = foo(); Into something like this: static bool some_value_initialized; static T some_value; if (!some_value_initialized) { some_value_initialized = true; some_value = foo(); }
So can someone please help me understand the scope of this problem? i.e.
- Are all uses of Boost.Function affected, or only certain uses (e.g. member functions vs. free functions)?
All uses.
- What exactly is the race between? E.g. if I know that I'll never create two threads at the same time, do I still need to worry about other uses of Boost.Function that may be concurrent? Is the race between creating the function objects or between using them, or both?
The race will occur if you construct or assign to two boost::function objects: - that have the same type, and - with target function objects of the same type, and - you have never completely constructed/assigned to a boost::function object of that type with that target function object type
- Can I avoid the problem by using a new version of Boost.Function, without upgrading other libraries? (Is Boost.Function header-only?)
Yes, you can grab function_base.hpp and function_template.hpp from the Boost trunk, overwrite the ones in 1.34.1, and you'll be fine. You should recompile anything that depends on Function (e.g., the threads library), but Function itself is header-only.
- What would the symptom of failure be? I'm currently debugging a double-free somewhere below std::string::assign, which is probably unrelated.
The symptom is a segmentation fault inside Boost.Function itself, because one thread's initialization of the vtable (a static) has claimed to be complete (e.g., some_value_initialized has been set to true) and another thread comes through and tries to dereference the NULL vtable pointer (which the other thread is filling in), then dies. If you've hit this problem, you'd know; it's very limited in scope.
I'm also curious to know how much more serious than this a bug would need to be before a Boost release would be recalled, or a new point-release released, or an announcement made on the Boost web page (which links to bugs fixed in 1.34.1, but not bugs introduced). Surely there aren't many users of Boost.Thread who are happy to use a non-thread-safe thread library.....
This should have gone on the web page, at the very least, and I'll do so now. - Doug
Douglas Gregor wrote:
Phil Endecott wrote:
- What exactly is the race between? E.g. if I know that I'll never create two threads at the same time, do I still need to worry about other uses of Boost.Function that may be concurrent? Is the race between creating the function objects or between using them, or both?
The race will occur if you construct or assign to two boost::function objects: - that have the same type, and - with target function objects of the same type, and - you have never completely constructed/assigned to a boost::function object of that type with that target function object type
OK, so the common case where I have a main thread that spawns per-operation threads: void handle_request(request_t req) { .... } void main() { while (1) { request_t r = receive_request(); thread t(bind(handle_request,r)); } } is safe, if this is the only place where threads are created. Another common case in my code is a per-object thread: class C { void run() { .... } thread run_thread; public: C(): run_thread(bind(C::run,this)) {} ~C() { run_thread.join(); } }; Because C is in the type, I only need to worry about concurrent creation of C objects; there's no race with threads in other classes.
- Can I avoid the problem by using a new version of Boost.Function, without upgrading other libraries? (Is Boost.Function header-only?)
[No, because]
You should recompile anything that depends on Function (e.g., the threads library)
This should have gone on the web page, at the very least, and I'll do so now.
Thanks. Phil.
Hi, Doug, I just following the message showing on boost.org: ----------- December 19, 2007 - Cricial Bug in Function Library * Boost.Function in Boost 1.34.x has a bug that affects the construction of Boost.Function objects in a multi-threaded context. The problem has been fixed in the Boost trunk and for the upcoming Boost 1.35.x. To patch your Boost 1.34.x, copy the files function_base.hpp and function_template.hpp into the Boost directory boost/function. ----------- and copied the function_base.hpp and function_template.hpp from the trunk and overwrite my boost 1.34.1, when I tried to compile it, it gives lots of errors, let me copy some of them, I am wondering how to solve it(I am using visual studio 2005): ------- .\boost/function/function_template.hpp(562) : error C2039: 'unary_function' : is not a member of 'std' .\boost/function/function_template.hpp(792) : see reference to class template instantiation 'boost::function1<R, T0,Allocator>' being compiled .\boost/function/function_template.hpp(562) : error C2504: 'unary_function' : base class undefined .\boost/function/function_template.hpp(562) : error C2143: syntax error : missing ',' before '<' .\boost/function/function_template.hpp(566) : error C2039: 'binary_function' : is not a member of 'std' .\boost/function/function_template.hpp(792) : see reference to class template instantiation 'boost::function2<R, T0,T1,Allocator>' being compiled .\boost/function/function_template.hpp(566) : error C2504: 'binary_function' : base class undefined .\boost/function/function_template.hpp(566) : error C2143: syntax error : missing ',' before '<' -------- Thanks, -Allen Douglas Gregor wrote:
Phil Endecott wrote:
Dear All,
In November Sergey Skorniakov wrote to this list asking about the non-thread-safety of Boost.Function and consequently of Boost.Thread itself. Steven Watanbee replied pointing out that the problem is resolved in the trunk. See: http://thread.gmane.org/gmane.comp.lib.boost.devel/167877
My thought at the time was "chocolate teapot". I put understanding this issue on my to-do list and it has finally got to the top. However, looking at the trac pages that Steven linked to, I am none the wiser; they contain patches but with no comments in the code and don't mention thread safety in the commit messages.
The root of the problem was some dynamic initialization in the vtable code, which is rarely (if ever?) thread-safe. The problem is that the compiler turns code like this:
static T some_value = foo();
Into something like this:
static bool some_value_initialized; static T some_value;
if (!some_value_initialized) { some_value_initialized = true; some_value = foo(); }
So can someone please help me understand the scope of this problem? i.e.
- Are all uses of Boost.Function affected, or only certain uses (e.g. member functions vs. free functions)?
All uses.
- What exactly is the race between? E.g. if I know that I'll never create two threads at the same time, do I still need to worry about other uses of Boost.Function that may be concurrent? Is the race between creating the function objects or between using them, or both?
The race will occur if you construct or assign to two boost::function objects: - that have the same type, and - with target function objects of the same type, and - you have never completely constructed/assigned to a boost::function object of that type with that target function object type
- Can I avoid the problem by using a new version of Boost.Function, without upgrading other libraries? (Is Boost.Function header-only?)
Yes, you can grab function_base.hpp and function_template.hpp from the Boost trunk, overwrite the ones in 1.34.1, and you'll be fine. You should recompile anything that depends on Function (e.g., the threads library), but Function itself is header-only.
- What would the symptom of failure be? I'm currently debugging a double-free somewhere below std::string::assign, which is probably unrelated.
The symptom is a segmentation fault inside Boost.Function itself, because one thread's initialization of the vtable (a static) has claimed to be complete (e.g., some_value_initialized has been set to true) and another thread comes through and tries to dereference the NULL vtable pointer (which the other thread is filling in), then dies. If you've hit this problem, you'd know; it's very limited in scope.
I'm also curious to know how much more serious than this a bug would need to be before a Boost release would be recalled, or a new point-release released, or an announcement made on the Boost web page (which links to bugs fixed in 1.34.1, but not bugs introduced). Surely there aren't many users of Boost.Thread who are happy to use a non-thread-safe thread library.....
This should have gone on the web page, at the very least, and I'll do so now.
- Doug _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Thu, 2007-12-20 at 16:07 -0800, Allen wrote:
Hi, Doug,
I just following the message showing on boost.org: ----------- December 19, 2007 - Cricial Bug in Function Library
* Boost.Function in Boost 1.34.x has a bug that affects the construction of Boost.Function objects in a multi-threaded context. The problem has been fixed in the Boost trunk and for the upcoming Boost 1.35.x. To patch your Boost 1.34.x, copy the files function_base.hpp and function_template.hpp into the Boost directory boost/function. -----------
and copied the function_base.hpp and function_template.hpp from the trunk and overwrite my boost 1.34.1, when I tried to compile it, it gives lots of errors, let me copy some of them, I am wondering how to solve it(I am using visual studio 2005):
Thanks for reporting this. There is an updated function_base.hpp at that same location, which properly includes the <functional> header. - Doug
participants (4)
-
Allen -
Anthony Williams -
Douglas Gregor -
Phil Endecott