Is Boost.Xpressive's sregex::compile() thread-safe for an object only used in that thread?
I have a case where I am compiling a dynamic regex based on client code input, and it works except for when I run my multi-threaded unit test 100-1000 times. When I run it with 1 thread, it never fails. When I run it with 2 or 4 threads it fails at this rare frequency. Except on my build server, where it seems to fail most of the time, which is how I learned about the issue to begin with. It's not crashing though, it's just very rarely returning the wrong results, and not always in the same place in the unit test. So I'm trying to understand Xpressive's thread safety. I understand it as far as not using a regex object in multiple threads until it's finished compiling and not sharing a match_results across threads, but what about compiling different regex objects for each thread: if two threads are compiling two different regexes (or even the same const regex string but from different threads, which is possible with my test code), is that ok? I'm seeing this failure on MSVC 12 but I think I also got it with VC10, just not as reliably on the build server (but the build server had to be upgraded to Windows 2012 to install VC12, so maybe that OS change had an effect as well). Thanks, -Matt
On 12/11/2014 2:35 PM, Chambers, Matthew wrote:
I have a case where I am compiling a dynamic regex based on client code input, and it works except for when I run my multi-threaded unit test 100-1000 times.
When I run it with 1 thread, it never fails. When I run it with 2 or 4 threads it fails at this rare frequency. Except on my build server, where it seems to fail most of the time, which is how I learned about the issue to begin with. It's not crashing though, it's just very rarely returning the wrong results, and not always in the same place in the unit test.
So I'm trying to understand Xpressive's thread safety. I understand it as far as not using a regex object in multiple threads until it's finished compiling and not sharing a match_results across threads, but what about compiling different regex objects for each thread: if two threads are compiling two different regexes (or even the same const regex string but from different threads, which is possible with my test code), is that ok?
I'm seeing this failure on MSVC 12 but I think I also got it with VC10, just not as reliably on the build server (but the build server had to be upgraded to Windows 2012 to install VC12, so maybe that OS change had an effect as well).
Are your regex objects referring to other regex objects? That is, are you defining a grammar? See this caveat in the docs: http://www.boost.org/doc/libs/1_57_0/doc/html/xpressive/user_s_guide.html#bo... The docs only mentions static regexes, but it's true for dynamic regexes. If you're not building grammars where some of the regexes are shared, then no, there are no known thread-safety problems with constructing a regex on a single thread. After the regex object is constructed, it is safe to use from multiple threads, since it's not mutated during pattern matching. HTH, Eric
On 12/12/2014 2:09 AM, Eric Niebler wrote:
On 12/11/2014 2:35 PM, Chambers, Matthew wrote:
I have a case where I am compiling a dynamic regex based on client code input, and it works except for when I run my multi-threaded unit test 100-1000 times.
When I run it with 1 thread, it never fails. When I run it with 2 or 4 threads it fails at this rare frequency. Except on my build server, where it seems to fail most of the time, which is how I learned about the issue to begin with. It's not crashing though, it's just very rarely returning the wrong results, and not always in the same place in the unit test.
So I'm trying to understand Xpressive's thread safety. I understand it as far as not using a regex object in multiple threads until it's finished compiling and not sharing a match_results across threads, but what about compiling different regex objects for each thread: if two threads are compiling two different regexes (or even the same const regex string but from different threads, which is possible with my test code), is that ok?
I'm seeing this failure on MSVC 12 but I think I also got it with VC10, just not as reliably on the build server (but the build server had to be upgraded to Windows 2012 to install VC12, so maybe that OS change had an effect as well). Are your regex objects referring to other regex objects? That is, are you defining a grammar? See this caveat in the docs:
http://www.boost.org/doc/libs/1_57_0/doc/html/xpressive/user_s_guide.html#bo...
The docs only mentions static regexes, but it's true for dynamic regexes.
If you're not building grammars where some of the regexes are shared, then no, there are no known thread-safety problems with constructing a regex on a single thread.
After the regex object is constructed, it is safe to use from multiple threads, since it's not mutated during pattern matching.
No, I'm using them as plain boost::regex-style regexes. You say "no thread-safety problems with constructing a regex on a single thread" - but I'm constructing a regex on multiple threads (it may be the same regex pattern or a different one, depending on how the threads are executing). If it's not a construction problem (a problem I may have ruled out by moving construction to a singleton which mutexes the compilation and caches the compiled patterns), then any other ideas why I would be seeing these very rare aberrant results? Thanks, -Matt
participants (2)
-
Chambers, Matthew -
Eric Niebler