Hello Hartmut Using multiple threads, wave crashes randomly after a few seconds of running. The crash is actually an assert in test.exe!std::list<boost::wave::cpplexer::lex_token<boost::wave::util::file_position<boost::wave::util::flex_string<char,std::char_traits<char>,std::allocator<char>,boost::wave::util::CowString<boost::wave::util::AllocatorStringStorage<char,std::allocator<char> >,char *> > > >,boost::fast_pool_allocator<boost::wave::cpplexer::lex_token<boost::wave::util::file_position<boost::wave::util::flex_string<char,std::char_traits<char>,std::allocator<char>,boost::wave::util::CowString<boost::wave::util::AllocatorStringStorage<char,std::allocator<char> >,char *> > > >,boost::default_user_allocator_new_delete,boost::details::pool::win32_mutex,32> >::_Const_iterator<1>::operator*() Line 218 + 0x17 bytes C++ indicating that the iterator is not dereferencable. I am using an VC 9 x86 debug build on Vista x64. Here's the code: #include <fstream> #include <string> //Wave does not like the new interface :( //#define BOOST_FILESYSTEM_NO_DEPRECATED #include <boost/thread.hpp> #include <boost/wave.hpp> #include <boost/wave/cpplexer/cpp_lex_token.hpp> #include <boost/wave/cpplexer/cpp_lex_iterator.hpp> std::string file; const char* file_name="C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v5.0\\Include\\Windows.h"; void test() { for(;;) { typedef boost::wave::cpplexer::lex_token<> token_type; typedef boost::wave::cpplexer::lex_iterator<token_type> lex_iterator_type; typedef boost::wave::context<std::string::iterator, lex_iterator_type> context_type; context_type ctx (file.begin(), file.end(), file_name); ctx.set_language(static_cast<boost::wave::language_support>(boost::wave::support_cpp|boost::wave::support_option_variadics|boost::wave::support_option_include_guard_detection)); ctx.add_sysinclude_path("C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v5.0\\Include\\"); ctx.add_sysinclude_path("C:\\Program Files (x86)\\Microsoft Visual Studio 9.0\\VC\\include\\"); ctx.add_macro_definition("_CPPRTTI",true); ctx.add_macro_definition("_CPPUNWIND",true); ctx.add_macro_definition("_MT",true); ctx.add_macro_definition("_MSC_VER=1600",true); ctx.add_macro_definition("_M_IX86=600",true); ctx.add_macro_definition("_WIN32=600",true); context_type::iterator_type first = ctx.begin(); context_type::iterator_type last = ctx.end(); while (first != last) { ++first; } } } int main() { std::ifstream instream(file_name); file = std::string(std::istreambuf_iterator<char>(instream.rdbuf()),std::istreambuf_iterator<char>()); instream.close(); for(int i=0;i<10;++i) { boost::thread t(test); } test(); } Thank you
Using multiple threads, wave crashes randomly after a few seconds of running. The crash is actually an assert in
test.exe!std::list<boost::wave::cpplexer::lex_token<boost::wave::util:: file_position<boost::wave::util::flex_string<char,std::char_traits<char
,std::allocator<char>,boost::wave::util::CowString<boost::wave::util:: AllocatorStringStorage<char,std::allocator<char> >,char *> > > ,boost::fast_pool_allocator<boost::wave::cpplexer::lex_token<boost::wa ve::util::file_position<boost::wave::util::flex_string<char,std::char_t raits<char>,std::allocator<char>,boost::wave::util::CowString<boost::wa ve::util::AllocatorStringStorage<char,std::allocator<char> >,char *> >
,boost::default_user_allocator_new_delete,boost::details::pool::win32_ mutex,32> >::_Const_iterator<1>::operator*() Line 218 + 0x17 bytes C++
indicating that the iterator is not dereferencable. I am using an VC 9 x86 debug build on Vista x64. Here's the code:
#include <fstream> #include <string>
//Wave does not like the new interface :( //#define BOOST_FILESYSTEM_NO_DEPRECATED #include <boost/thread.hpp> #include <boost/wave.hpp> #include <boost/wave/cpplexer/cpp_lex_token.hpp> #include <boost/wave/cpplexer/cpp_lex_iterator.hpp>
std::string file; const char* file_name="C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v5.0\\Include\\Windows.h";
void test() { for(;;) { typedef boost::wave::cpplexer::lex_token<> token_type; typedef boost::wave::cpplexer::lex_iterator<token_type> lex_iterator_type; typedef boost::wave::context<std::string::iterator, lex_iterator_type> context_type;
context_type ctx (file.begin(), file.end(), file_name);
ctx.set_language(static_cast<boost::wave::language_support>(boost::wave ::support_cpp|boost::wave::support_option_variadics|boost::wave::suppor t_option_include_guard_detection)); ctx.add_sysinclude_path("C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v5.0\\Include\\"); ctx.add_sysinclude_path("C:\\Program Files (x86)\\Microsoft Visual Studio 9.0\\VC\\include\\");
ctx.add_macro_definition("_CPPRTTI",true); ctx.add_macro_definition("_CPPUNWIND",true); ctx.add_macro_definition("_MT",true); ctx.add_macro_definition("_MSC_VER=1600",true); ctx.add_macro_definition("_M_IX86=600",true); ctx.add_macro_definition("_WIN32=600",true);
context_type::iterator_type first = ctx.begin(); context_type::iterator_type last = ctx.end(); while (first != last) { ++first; } } }
int main() { std::ifstream instream(file_name); file = std::string(std::istreambuf_iterator<char>(instream.rdbuf()),std::istre ambuf_iterator<char>()); instream.close();
for(int i=0;i<10;++i) { boost::thread t(test); }
test(); }
Thanks! Will investigate. Regards Hartmut
On Mon, Mar 9, 2009 at 11:21 PM, Hartmut Kaiser <hartmut.kaiser@gmail.com>wrote:
Thanks! Will investigate. Regards Hartmut
Hello *, I can also reproduce it on my WinXP (32bit) & MSVC 9 machine, but only when no debugger is attached :( At least waiting with attached debugger (release compiled build) for about 10 minutes did not cause this app to crash. Last time I coded some C++ source parser with Wave (almost 2 years ago) it worked fine in MT environment with pretty much heavy load, but that were every time different files. What I found out for now is that wave::context can't deal with std::string::const_iterator. Why? I get a compiler error. If it uses non-const iterator, is it changing the string? Luckily making a local copy of the file string in each thread causes the app to crash almost immediately in the token_data::dtor. But just a question, is token_data only wave::context related or is it used accross contexts? My modified source: #include <fstream> #include <string> //Wave does not like the new interface :( //#define BOOST_FILESYSTEM_NO_DEPRECATED #include <boost/thread.hpp> #include <boost/wave.hpp> #include <boost/wave/cpplexer/cpp_lex_token.hpp> #include <boost/wave/cpplexer/cpp_lex_iterator.hpp> const char* file_name="C:\\Program Files\\Microsoft SDKs\\Windows\\v5.0\\Include\\Windows.h"; std::string file_contents() { std::ifstream instream(file_name); return std::string(std::istreambuf_iterator<char>(instream.rdbuf()),std::istreambuf_iterator<char>()); } const std::string file=file_contents(); void test() { std::string local=file; for(;;) { typedef boost::wave::cpplexer::lex_token<> token_type; typedef boost::wave::cpplexer::lex_iterator<token_type> lex_iterator_type; typedef boost::wave::context<std::string::iterator, lex_iterator_type> context_type; context_type ctx (local.begin(), local.end(), file_name); ctx.set_language(static_cast<boost::wave::language_support>(boost::wave::support_cpp|boost::wave::support_option_variadics|boost::wave::support_option_include_guard_detection)); ctx.add_sysinclude_path("C:\\Program Files\\Microsoft SDKs\\Windows\\v5.0\\Include\\"); ctx.add_sysinclude_path("C:\\Program Files\\Microsoft Visual Studio 9.0\\VC\\include\\"); ctx.add_macro_definition("_CPPRTTI",true); ctx.add_macro_definition("_CPPUNWIND",true); ctx.add_macro_definition("_MT",true); ctx.add_macro_definition("_MSC_VER=1600",true); ctx.add_macro_definition("_M_IX86=600",true); ctx.add_macro_definition("_WIN32=600",true); context_type::iterator_type first = ctx.begin(); context_type::iterator_type last = ctx.end(); while (first != last) { ++first; } } } int main() { for(int i=0;i<10;++i) { boost::thread t(test); } test(); } Greetings, Ovanes
Ovanes,
I can also reproduce it on my WinXP (32bit) & MSVC 9 machine, but only when no debugger is attached :( At least waiting with attached debugger (release compiled build) for about 10 minutes did not cause this app to crash.
Last time I coded some C++ source parser with Wave (almost 2 years ago) it worked fine in MT environment with pretty much heavy load, but that were every time different files. What I found out for now is that wave::context can't deal with std::string::const_iterator. Why? I get a compiler error. If it uses non-const iterator, is it changing the string?
Could you provide a small test reproducing your problem? Regards Hartmut
Ein Held wrote:
{ std::ifstream instream(file_name); file = std::string(std::istreambuf_iterator<char>(instream.rdbuf()),std::istreambuf_iterator<char>()); instream.close();
for(int i=0;i<10;++i) { boost::thread t(test); }
test(); }
I assume you are using the latest Boost. At a quick glance, it seems that it is possible for one of your 10 threads to still be running when the main program exits. I wonder if this is the problem. You might want to use thread_group and join_all to make sure. Specifically: for(...) { // destructor calls detach() -> thread keeps running boost::thread t(test); } test(); // possible to finish before one of the above threads Try changing to: boost::thread_group group; for(...) { group.add_thread(new boost::thread(test)); } test(); group.join_all(); Good luck! -- Sohail Somani http://uint32t.blogspot.com
Sohail Somani wrote:
Ein Held wrote:
{ std::ifstream instream(file_name); file = std::string(std::istreambuf_iterator<char>(instream.rdbuf()),std::istreambuf_iterator<char>()); instream.close();
for(int i=0;i<10;++i) { boost::thread t(test); }
test(); }
I assume you are using the latest Boost.
At a quick glance, it seems that it is possible for one of your 10 threads to still be running when the main program exits. I wonder if this is the problem.
That quick glance killed it... That is an infinite loop in test(). My bad! -- Sohail Somani http://uint32t.blogspot.com
participants (4)
-
Ein Held -
Hartmut Kaiser -
Ovanes Markarian -
Sohail Somani