My test program was as follows: [Code] // fly.cpp : main project file. #include "stdafx.h" #include <string> #include <iostream> #include <vector> #include <fstream> #include <iterator> #include <algorithm> #include <boost/progress.hpp> #include <boost/flyweight.hpp> using namespace std; vector<boost::flyweight<string> > vec; //vector<string> vec; struct Insert { void operator()(string s) { vec.push_back(boost::flyweight<string>(s)); //vec.push_back(s); } }; void Test() { fstream f("d:\\bible12.txt", ios_base::in); // A large text file! { boost::progress_timer t; for_each(istream_iterator<string>(f), istream_iterator<string>(), Insert()); } } int main(int argc, char* argv[]) { Test(); cout << vec.size() << endl; getchar(); copy(vec.begin(), vec.begin() + 200, ostream_iterator<string, char>(cout, " ")); getchar(); return 0; } [/Code] Using the boost_1_35_0 svn library download and Visual C++ 2008 Express edition. Originally compiled as a CLR program. Recompiling using native Win32 still gives a crash during program termination - in atomic_count_win32.hpp in operator-- (BOOST_INTERLOCKED_DECREMENT) Unhandled exception at 0x010752b9 in fly2.exe: 0xC0000005: Access violation writing location 0x00779040. There is still the anomaly of release version being slower than debug. Compiled as release - the exception occurs in xmemory in _Destroy_range. I'll experiment using set_factory in Borland - although I have largely abandoned it's use, just can't compile enough of Boost to be useful now... Martin
On Wed, Jan 30, 2008 at 01:41:15PM +0000, Martin Fisher wrote:
Using the boost_1_35_0 svn library download and Visual C++ 2008 Express edition. Originally compiled as a CLR program. Recompiling using native Win32 still gives a crash during program termination - in atomic_count_win32.hpp in operator-- (BOOST_INTERLOCKED_DECREMENT) Unhandled exception at 0x010752b9 in fly2.exe: 0xC0000005: Access violation writing location 0x00779040.
There is indeed a bug somewhere. I could reproduce it on Linux which doesn't suffer from CLR/libc/... mess. I just compiled it (code from trunk) and it crashes. valgrind reports: ==25155== Invalid read of size 4 ==25155== at 0x8049678: boost::detail::atomic_count::atomic_exchange_and_add(int*, int) (atomic_count_gcc_x86.hpp:74) ==25155== by 0x80496A2: boost::detail::atomic_count::operator--() (atomic_count_gcc_x86.hpp:42) ==25155== by 0x8049835: boost::flyweights::detail::refcounted_value<std::string>::release() const (refcounted.hpp:61) ==25155== by 0x804BEF6: boost::flyweights::detail::refcounted_handle<boost::flyweights::detail::refcounted_value<std::string> const*, boost::flyweights::detail::flyweight_core_tracking_helper<std::string, mpl_::na, boost::flyweights::refcounted, boost::flyweights::hashed_factory<mpl_::na, mpl_::na, mpl_::na, 0>, boost::flyweights::simple_locking, boost::flyweights::static_holder> >::~refcounted_handle() (refcounted.hpp:91) ==25155== by 0x804BF20: boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::~flyweight() (flyweight.hpp:64) ==25155== by 0x804BF34: void std::_Destroy<boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_> >(boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>*) (stl_construct.h:107) ==25155== by 0x804BF4A: void std::__destroy_aux<boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>*>(boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>*, boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>*, std::__false_type) (stl_construct.h:122) ==25155== by 0x804BF79: void std::_Destroy<boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>*>(boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>*, boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>*) (stl_construct.h:155) ==25155== by 0x804BF93: void std::_Destroy<boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>*, boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_> >(boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>*, boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>*, std::allocator<boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_> >) (stl_construct.h:182) ==25155== by 0x804BFD4: std::vector<boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>, std::allocator<boost::flyweights::flyweight<std::string, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_> > >::~vector() (stl_vector.h:268) ==25155== by 0x80492B1: __tcf_2 (test.cpp:12) ==25155== by 0x4870F13: exit (in /lib/i686/cmov/libc-2.7.so) ==25155== Address 0x6fd1374 is not stack'd, malloc'd or (recently) free'd ==25155== ==25155== Invalid read of size 4 ==25155== at 0x8049678: boost::detail::atomic_count::atomic_exchange_and_add(int*, int) (atomic_count_gcc_x86.hpp:74) ==25155== by 0x80496C0: boost::detail::atomic_count::operator long() const (atomic_count_gcc_x86.hpp:47) ==25155== by 0x8049BD3: boost::flyweights::detail::refcounted_value<std::string>::count() const (refcounted.hpp:59) ==25155== by 0x804BABA: boost::flyweights::detail::refcounted_handle<boost::flyweights::detail::refcounted_value<std::string> const*, [snip] Jens
Jens Seidel ha escrito:
On Wed, Jan 30, 2008 at 01:41:15PM +0000, Martin Fisher wrote:
Using the boost_1_35_0 svn library download and Visual C++ 2008 Express edition. Originally compiled as a CLR program. Recompiling using native Win32 still gives a crash during program termination - in atomic_count_win32.hpp in operator-- (BOOST_INTERLOCKED_DECREMENT) Unhandled exception at 0x010752b9 in fly2.exe: 0xC0000005: Access violation writing location 0x00779040.
There is indeed a bug somewhere. I could reproduce it on Linux which doesn't suffer from CLR/libc/... mess. I just compiled it (code from trunk) and it crashes. valgrind reports:
[...] Jens, Martin, I could also reproduce it here, and found the reason for the crash. Please follow me: The static initialization of flyweight<string>'s internal factory (explained in detail at http://tinyurl.com/2rwwug ) is done automatically before the first use of flyweight<string> in the program, or before main() at the very latest. But now let's take a look again at Martin's program: vector<boost::flyweight<string> > vec; ... int main(int argc, char* argv[]) { ... } The static initialization of vec invokes vector<boost::flyweight<string> > ctor, but this ctor *does not use boost::flyweight<string>*: no flyweight objects are created during the default initialization of the vector. So, boost::flyweight<string>'s factory is not guaranteed to be initialized before vec, only before main()! Consequently, the following is a valid sequence of static initializations and destructions (which go in reverse order as the corresponding initializations): // initialziation of vec vector<boost::flyweight<string> > vec; // (automatic) initialization of boost::flyweight<string>'s factory ... int main(int argc, char* argv[]) { ... } // automatic destruction of boost::flyweight<string>'s factory // automatic destruction of vec And this is the problem: when the factory is destroyed, there are still flyweight objects around (those in vec), when there should be none, and on vec's destruction time the flyweight objects point to deallocated memory, hence the crash. This is solved by forcing boost::flyweight<string> static initialization before that of vec: boost::flyweights::flyweight<string>::initializer init; vector<boost::flyweights::flyweight<string> > vec; So that static initialization order is correct. Could you please try the fix? This is a fundamental issue for which I see no easy *automatic* fix upon first reflection, other than documenting very clearly the situations in which it can arise. Comments and suggestions welcome. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
On Wed, Jan 30, 2008 at 06:38:34PM +0100, Joaquín Mª López Muñoz wrote:
Jens Seidel ha escrito:
On Wed, Jan 30, 2008 at 01:41:15PM +0000, Martin Fisher wrote:
Using the boost_1_35_0 svn library download and Visual C++ 2008 Express edition. Originally compiled as a CLR program. Recompiling using native Win32 still gives a crash during program termination - in atomic_count_win32.hpp in operator-- (BOOST_INTERLOCKED_DECREMENT) Unhandled exception at 0x010752b9 in fly2.exe: 0xC0000005: Access violation writing location 0x00779040.
There is indeed a bug somewhere. I could reproduce it on Linux which doesn't suffer from CLR/libc/... mess. I just compiled it (code from trunk) and it crashes. valgrind reports:
I could also reproduce it here, and found the reason for the crash. Please follow me:
And this is the problem: when the factory is destroyed, there are still flyweight objects around (those in vec), when there should be none, and on vec's destruction time the flyweight objects point to deallocated memory, hence the crash.
This is solved by forcing boost::flyweight<string> static initialization before that of vec:
boost::flyweights::flyweight<string>::initializer init; vector<boost::flyweights::flyweight<string> > vec;
So that static initialization order is correct. Could you please try the fix?
Right. valgrind does no longer complain.
This is a fundamental issue for which I see no easy *automatic* fix upon first reflection, other than documenting very clearly the situations in which it can arise. Comments and suggestions welcome.
Maybe it is possible to add an assert somewhere, so that at least debugging is simplified? Not everyone reads documentation carefully. Thanks, Jens
Jens Seidel ha escrito:
On Wed, Jan 30, 2008 at 06:38:34PM +0100, Joaquín Mª López Muñoz wrote:
This is solved by forcing boost::flyweight<string> static initialization before that of vec:
boost::flyweights::flyweight<string>::initializer init; vector<boost::flyweights::flyweight<string> > vec;
So that static initialization order is correct. Could you please try the fix?
Right. valgrind does no longer complain.
This is a fundamental issue for which I see no easy *automatic* fix upon first reflection, other than documenting very clearly the situations in which it can arise. Comments and suggestions welcome.
Maybe it is possible to add an assert somewhere, so that at least debugging is simplified? Not everyone reads documentation carefully.
Yep, this is an excellent suggestion. the refcounted tracking policiy can detect such illegal situations (an entry is destroyed with a refcount!=0), so I can add a BOOST_ASSERT to check it. Put in my todo list. Have you been able to rerun your tests? Still getting weird performance figures? Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
participants (3)
-
Jens Seidel -
Joaquín Mª López Muñoz -
Martin Fisher