Threads and Smart Ptrs

Greetings, everyone. I'm currently having some trouble with a heavily multithreaded app which uses objects shared between threads via smart pointers. It's a multi-user application, where each user has a thread, and when one user sends a message to another (or broadcasts to all), a message object is created, and a smart pointer to that message is put into a queue for each thread. I'm using pthread_mutex for locking the queues for each thread. This program will sometimes work flawlessly for hours, and other times it will die within moments of starting. Pretty much every time it dies, it's because the message object has been freed while there are still smart pointers pointing to it. I know that the recent version of the smart pointer library was supposed to have added thread-safety to it, but I'm wondering if anyone else is having similar troubles. I'm using gcc 3.0.3 on Linux, and boost 1.27. I also discovered some things in the gcc documentation having to do with threads, (eg: when you compile the compiler, you have to tell it to build multi-threaded programs). And there are various threading flags that can be used during a compile, and I'm not sure which to use (or all of them?): -D_PTHREADS -D_REENTRANT -pthread : I'm using all those right now. Is that wrong? Any help would be greatly appreciated. It's driving me mad! :) Regards, Colin -- Colin Fox CF Consulting Inc. cfox@telus.net

From: "Colin Fox" <cfox@crystalcherry.com>
Greetings, everyone.
I'm currently having some trouble with a heavily multithreaded app which uses objects shared between threads via smart pointers.
It's a multi-user application, where each user has a thread, and when one user sends a message to another (or broadcasts to all), a message object is created, and a smart pointer to that message is put into a queue for each thread.
I'm using pthread_mutex for locking the queues for each thread.
This program will sometimes work flawlessly for hours, and other times it will die within moments of starting. Pretty much every time it dies, it's because the message object has been freed while there are still smart pointers pointing to it.
I know that the recent version of the smart pointer library was supposed to have added thread-safety to it, but I'm wondering if anyone else is having similar troubles.
I'm using gcc 3.0.3 on Linux, and boost 1.27.
Can you simplify the program to the bare minimum that still has the problem? Also, does the CVS version of shared_ptr work? (It now uses a pthread_mutex on Linux and not atomic intructions.)

On Fri, 2002-02-15 at 11:31, Peter Dimov wrote:
From: "Colin Fox" <cfox@crystalcherry.com> <..>
I'm using gcc 3.0.3 on Linux, and boost 1.27.
Can you simplify the program to the bare minimum that still has the problem?
Not very easily, but if I can't make any progress, I'll do that.
Also, does the CVS version of shared_ptr work? (It now uses a pthread_mutex on Linux and not atomic intructions.)
Thanks, I'll give it a shot. I noticed that one time that it died, it died within the atomic code, so maybe this will help. (Crossing fingers). -- Colin Fox CF Consulting Inc. cfox@crystalcherry.com

On Fri, 2002-02-15 at 11:31, Peter Dimov wrote:
From: "Colin Fox" <cfox@crystalcherry.com> <..>
I'm using gcc 3.0.3 on Linux, and boost 1.27.
Can you simplify the program to the bare minimum that still has the
From: "Colin Fox" <cfox@crystalcherry.com> problem?
Not very easily, but if I can't make any progress, I'll do that.
OK, let me explain the exact level of thread safety provided by shared_ptr; it might be of help. shared_ptr (in 1.27+) is (should be) thread neutral. A class X is thread neutral when: * Accessing two different objects of class X is safe, even when they are equivalent copies; * Accessing the same object is safe only when all accesses are read accesses; otherwise the behavior is undefined. For example, under the pthreads memory model all C types are thread neutral. This means that: shared_ptr p; // thread 1 shared_ptr q(p); // read p // thread 2 shared_ptr r(p); // read p is safe. // thread 1 q.reset(); // write q // thread 2 r.reset(); // write r is safe. // thread 1 shared_ptr s(p); // read p // thread 2 p.reset(); // write p is undefined behavior. If your program doesn't read/write or write/write (to) the same shared_ptr simultaneously, and still misbehaves, then there is a bug in shared_ptr that we'll have to track down.

On Sat, 2002-02-16 at 03:37, Peter Dimov wrote: <..>
OK, let me explain the exact level of thread safety provided by shared_ptr; it might be of help.
shared_ptr (in 1.27+) is (should be) thread neutral. A class X is thread neutral when:
* Accessing two different objects of class X is safe, even when they are equivalent copies; * Accessing the same object is safe only when all accesses are read accesses; otherwise the behavior is undefined. <..> If your program doesn't read/write or write/write (to) the same shared_ptr simultaneously, and still misbehaves, then there is a bug in shared_ptr that we'll have to track down.
The problem I'm having is simply that one thread destroys it's smart pointer to an object, and the object gets destroyed while another thread is still using that object. Once these shared objects are created, all threads only read them, so it's not that kind of read/write conflict. I just need the object to hang around until no more threads need it. I got the CVS version of boost, which you suggested, and had my program running all night last night. No problems so far. Perhaps this solves it. -- Colin Fox CF Consulting Inc. cfox@crystalcherry.com

From: "Colin Fox" <cfox@crystalcherry.com>
I got the CVS version of boost, which you suggested, and had my program running all night last night. No problems so far. Perhaps this solves it.
Let's hope so. I presume you're on a multiprocessor? Could you please also try my latest attempt, that uses lwm_linux.hpp on linux instead of the generic lwm_pthreads.hpp? It's much faster than the pthreads version when there's no contention but there's no point being faster if it's incorrect.

On Sat, 2002-02-16 at 10:50, Peter Dimov wrote:
From: "Colin Fox" <cfox@crystalcherry.com>
I got the CVS version of boost, which you suggested, and had my program running all night last night. No problems so far. Perhaps this solves it.
Let's hope so. I presume you're on a multiprocessor? Could you please also try my latest attempt, that uses lwm_linux.hpp on linux instead of the generic lwm_pthreads.hpp? It's much faster than the pthreads version when there's no contention but there's no point being faster if it's incorrect.
Yes, I forgot to mention that - I'm developing on a dual processor box. How do I try the latest attempt? Is that what's in CVS, or do I need something else? Speed will become important, because eventually this is going to have to handle a fairly high thread & activity load. But I completely agree that correctness comes first. :) -- Colin Fox CF Consulting Inc. cfox@crystalcherry.com

On Sat, 2002-02-16 at 10:50, Peter Dimov wrote:
From: "Colin Fox" <cfox@crystalcherry.com>
I got the CVS version of boost, which you suggested, and had my
From: "Colin Fox" <cfox@crystalcherry.com> program
running all night last night. No problems so far. Perhaps this solves it.
Let's hope so. I presume you're on a multiprocessor? Could you please also try my latest attempt, that uses lwm_linux.hpp on linux instead of the generic lwm_pthreads.hpp? It's much faster than the pthreads version when there's no contention but there's no point being faster if it's incorrect.
Yes, I forgot to mention that - I'm developing on a dual processor box.
I kind of suspected that. This probably means that the atomic_* functions don't synchronize memory (they are supposed to, but...)
How do I try the latest attempt? Is that what's in CVS, or do I need something else?
Yes, it's in CVS. I've also added a test for thread safety, libs/smart_ptr/shared_ptr_mt_test.cpp.
Speed will become important, because eventually this is going to have to handle a fairly high thread & activity load. But I completely agree that correctness comes first. :)
Copying a shared_ptr is about 3.4 times faster with the specialized version, but it depends on atomic_inc and atomic_dec_and_test as well, so it probably won't work. Oh well.

Hi Colin, I had soimilar problem using boost 1.25.8. Here is my own modified implementation of smart pointers based on the non thread safe boost one. Hope this will help you, Seb. #ifndef BOOST_SMART_PTR_HPP #define BOOST_SMART_PTR_HPP #include <boost/config.hpp> // for broken compiler workarounds #include <cstddef> // for std::size_t #include <memory> // for std::auto_ptr #include <algorithm> // for std::swap #include <boost/utility.hpp> // for boost::noncopyable, checked_delete, checked_array_delete #include <functional> // for std::less #include <boost/static_assert.hpp> // for BOOST_STATIC_ASSERT // Added by seb #include <boost/thread/mutex.hpp> #ifdef BOOST_MSVC // moved here to work around VC++ compiler crash # pragma warning(push) # pragma warning(disable:4284) // return type for 'identifier::operator->' is not a UDT or reference to a UDT. Will produce errors if applied using infix notation #endif namespace boost { // scoped_ptr --------------------------------------------------------------// // scoped_ptr mimics a built-in pointer except that it guarantees deletion // of the object pointed to, either on destruction of the scoped_ptr or via // an explicit reset(). scoped_ptr is a simple solution for simple needs; // see shared_ptr (below) or std::auto_ptr if your needs are more complex. template<typename T> class scoped_ptr : noncopyable { T* ptr; public: typedef T element_type; explicit scoped_ptr( T* p=0 ) : ptr(p) {} // never throws ~scoped_ptr() { checked_delete(ptr); } void reset( T* p=0 ) { if ( ptr != p ) { checked_delete(ptr); ptr = p; } } T& operator*() const { return *ptr; } // never throws T* operator->() const { return ptr; } // never throws T* get() const { return ptr; } // never throws #ifdef BOOST_SMART_PTR_CONVERSION // get() is safer! Define BOOST_SMART_PTR_CONVERSION at your own risk! operator T*() const { return ptr; } // never throws #endif }; // scoped_ptr // scoped_array ------------------------------------------------------------// // scoped_array extends scoped_ptr to arrays. Deletion of the array pointed to // is guaranteed, either on destruction of the scoped_array or via an explicit // reset(). See shared_array or std::vector if your needs are more complex. template<typename T> class scoped_array : noncopyable { T* ptr; public: typedef T element_type; explicit scoped_array( T* p=0 ) : ptr(p) {} // never throws ~scoped_array() { checked_array_delete(ptr); } void reset( T* p=0 ) { if ( ptr != p ) {checked_array_delete(ptr); ptr=p;} } T* get() const { return ptr; } // never throws #ifdef BOOST_SMART_PTR_CONVERSION // get() is safer! Define BOOST_SMART_PTR_CONVERSION at your own risk! operator T*() const { return ptr; } // never throws #else T& operator[](std::size_t i) const { return ptr[i]; } // never throws #endif }; // scoped_array // shared_ptr --------------------------------------------------------------// // An enhanced relative of scoped_ptr with reference counted copy semantics. // The object pointed to is deleted when the last shared_ptr pointing to it // is destroyed or reset. template<typename T> class shared_ptr { public: typedef T element_type; explicit shared_ptr(T* p =0) : px(p) { try { // Added by seb to be able to delete pn is new throws pn = NULL; pn = new long(1); // Added by seb. pm = new mutex(); } // fix: prevent leak if new throws catch (...) { checked_delete(p); // Added by seb: delete mutex tom prevent memory leaks checked_delete(pm); throw; } } ~shared_ptr() { dispose(); } #if !defined( BOOST_NO_MEMBER_TEMPLATES ) || defined (BOOST_MSVC6_MEMBER_TEMPLATES) template<typename Y> shared_ptr(const shared_ptr<Y>& r) : px(r.px) { // never throws // Added by seb: Take the shared mutex here to be safe when // we are incrementing the reference count. // First share the same mutex. pm = r.pm; mutex::scoped_lock l(*pm); // End of seb's modif ++*(pn = r.pn); } #ifndef BOOST_NO_AUTO_PTR template<typename Y> explicit shared_ptr(std::auto_ptr<Y>& r) { pn = new long(1); // may throw // Added by seb. pm = new mutex(); // may throw px = r.release(); // fix: moved here to stop leak if new throws } #endif template<typename Y> shared_ptr& operator=(const shared_ptr<Y>& r) { share(r.px,r.pn, r.pm); return *this; } #ifndef BOOST_NO_AUTO_PTR template<typename Y> shared_ptr& operator=(std::auto_ptr<Y>& r) { // code choice driven by guarantee of "no effect if new throws" // Added by seb - synchronize all this thing mutex::scoped_lock l(*pm); if (*pn == 1) { checked_delete(px); } else { // allocate new reference counter long * tmp = new long(1); // may throw --*pn; // only decrement once danger of new throwing is past pn = tmp; } // allocate new reference counter px = r.release(); // fix: moved here so doesn't leak if new throws return *this; } #endif #else #ifndef BOOST_NO_AUTO_PTR explicit shared_ptr(std::auto_ptr<T>& r) { pn = new long(1); // may throw // Added by seb - Instanciate the mutex pm = new mutex(); px = r.release(); // fix: moved here to stop leak if new throws } shared_ptr& operator=(std::auto_ptr<T>& r) { // code choice driven by guarantee of "no effect if new throws" mutex::scoped_lock l(*pm); if (*pn == 1) { checked_delete(px); } else { // allocate new reference counter long * tmp = new long(1); // may throw --*pn; // only decrement once danger of new throwing is past pn = tmp; } // allocate new reference counter px = r.release(); // fix: moved here so doesn't leak if new throws return *this; } #endif #endif // Added by seb to be able to downcast a shared_ptr template<class Derived> void downcasted_copy_to(shared_ptr<Derived>& q) const { Derived* rawq = dynamic_cast<Derived*>(px); if(!rawq || !px) { q = shared_ptr<Derived>(); } else { shared_ptr<Derived> ptmp; ptmp.px = rawq; { mutex::scoped_lock l(*pm); ++*pn; } ptmp.pn = pn; ptmp.pm = pm; q.swap(ptmp); } } // End seb's addition // The assignment operator and the copy constructor must come after // the templated versions for MSVC6 to work. (Gary Powell) shared_ptr(const shared_ptr& r) : px(r.px), // Added by seb pm(r.pm) { // Added by seb mutex::scoped_lock l(*pm); ++*(pn = r.pn); } // never throws shared_ptr& operator=(const shared_ptr& r) { share(r.px,r.pn, r.pm); return *this; } void reset(T* p=0) { if ( px == p ) return; // fix: self-assignment safe // Added by seb mutex::scoped_lock l(*pm); if (--*pn == 0) { checked_delete(px); } else { // allocate new reference counter try { // Added by seb pm = NULL; pn = new long; // Added by seb pm = new mutex(); } // fix: prevent leak if new throws catch (...) { ++*pn; // undo effect of --*pn above to meet effects guarantee checked_delete(p); // Added by seb checked_delete(pm); throw; } // catch } // allocate new reference counter // If an other smart pointer had a reference of the pointer, // then we have taken the common mutex. // In the other case, our pointer is not yet shared, therefore // we don't need to take a lock here. *pn = 1; px = p; } // reset T& operator*() const { return *px; } // never throws T* operator->() const { return px; } // never throws T* get() const { return px; } // never throws #ifdef BOOST_SMART_PTR_CONVERSION // get() is safer! Define BOOST_SMART_PTR_CONVERSION at your own risk! operator T*() const { return px; } // never throws #endif long use_count() const { // Added by Seb mutex::scoped_lock l(*pm); return *pn; } // never throws bool unique() const { mutex::scoped_lock l(*pm); return *pn == 1; } // never throws void swap(shared_ptr<T>& other) // never throws { std::swap(px,other.px); std::swap(pn,other.pn); // Added by seb std::swap(pm, other.pm); } // Tasteless as this may seem, making all members public allows member templates // to work in the absence of member template friends. (Matthew Langston) // Don't split this line into two; that causes problems for some GCC 2.95.2 builds #if ( defined(BOOST_NO_MEMBER_TEMPLATES) && !defined(BOOST_MSVC6_MEMBER_TEMPLATES) ) || !defined( BOOST_NO_MEMBER_TEMPLATE_FRIENDS ) private: #endif T* px; // contained pointer long* pn; // ptr to reference counter // Added by seb mutex* pm; // Don't split this line into two; that causes problems for some GCC 2.95.2 builds #if !defined( BOOST_NO_MEMBER_TEMPLATES ) && !defined( BOOST_NO_MEMBER_TEMPLATE_FRIENDS ) template<typename Y> friend class shared_ptr; #endif void dispose() { long ref; { mutex::scoped_lock l(*pm); ref = --*pn; } if (ref == 0) { checked_delete(px); delete pn; // Added by seb delete pm; } } // rpm added by seb void share(T* rpx, long* rpn, mutex* rpm) { mutex::scoped_lock l(*rpm); if (pn != rpn) { // Q: why not px != rpx? A: fails when both == 0 ++*rpn; // done before dispose() in case rpn transitively // dependent on *this (bug reported by Ken Johnson) dispose(); px = rpx; pn = rpn; pm = rpm; } } // share }; // shared_ptr template<typename T, typename U> inline bool operator==(const shared_ptr<T>& a, const shared_ptr<U>& b) { return a.get() == b.get(); } template<typename T, typename U> inline bool operator!=(const shared_ptr<T>& a, const shared_ptr<U>& b) { return a.get() != b.get(); } // shared_array ------------------------------------------------------------// // shared_array extends shared_ptr to arrays. // The array pointed to is deleted when the last shared_array pointing to it // is destroyed or reset. template<typename T> class shared_array { public: typedef T element_type; explicit shared_array(T* p =0) : px(p) { try { pn = new long(1); } // fix: prevent leak if new throws catch (...) { checked_array_delete(p); throw; } } shared_array(const shared_array& r) : px(r.px) // never throws { ++*(pn = r.pn); } ~shared_array() { dispose(); } shared_array& operator=(const shared_array& r) { if (pn != r.pn) { // Q: why not px != r.px? A: fails when both px == 0 ++*r.pn; // done before dispose() in case r.pn transitively // dependent on *this (bug reported by Ken Johnson) dispose(); px = r.px; pn = r.pn; } return *this; } // operator= void reset(T* p=0) { if ( px == p ) return; // fix: self-assignment safe if (--*pn == 0) { checked_array_delete(px); } else { // allocate new reference counter try { pn = new long; } // fix: prevent leak if new throws catch (...) { ++*pn; // undo effect of --*pn above to meet effects guarantee checked_array_delete(p); throw; } // catch } // allocate new reference counter *pn = 1; px = p; } // reset T* get() const { return px; } // never throws #ifdef BOOST_SMART_PTR_CONVERSION // get() is safer! Define BOOST_SMART_PTR_CONVERSION at your own risk! operator T*() const { return px; } // never throws #else T& operator[](std::size_t i) const { return px[i]; } // never throws #endif long use_count() const { return *pn; } // never throws bool unique() const { return *pn == 1; } // never throws void swap(shared_array<T>& other) // never throws { std::swap(px,other.px); std::swap(pn,other.pn); } private: T* px; // contained pointer long* pn; // ptr to reference counter void dispose() { if (--*pn == 0) { checked_array_delete(px); delete pn; } } }; // shared_array template<typename T> inline bool operator==(const shared_array<T>& a, const shared_array<T>& b) { return a.get() == b.get(); } template<typename T> inline bool operator!=(const shared_array<T>& a, const shared_array<T>& b) { return a.get() != b.get(); } // Added by seb for downcasting capability template<class Tout, class Tin> shared_ptr<Tout> do_dynamic_cast(const shared_ptr<Tin>& p) { shared_ptr<Tout> pout; p.downcasted_copy_to(pout); return pout; } } // namespace boost // specializations for things in namespace std -----------------------------// #ifndef BOOST_NO_TEMPLATE_PARTIAL_SPECIALIZATION namespace std { // Specialize std::swap to use the fast, non-throwing swap that's provided // as a member function instead of using the default algorithm which creates // a temporary and uses assignment. template<typename T> inline void swap(boost::shared_ptr<T>& a, boost::shared_ptr<T>& b) { a.swap(b); } template<typename T> inline void swap(boost::shared_array<T>& a, boost::shared_array<T>& b) { a.swap(b); } // Specialize std::less so we can use shared pointers and arrays as keys in // associative collections. // It's still a controversial question whether this is better than supplying // a full range of comparison operators (<, >, <=,
=).
template<typename T> struct less< boost::shared_ptr<T> > : binary_function<boost::shared_ptr<T>, boost::shared_ptr<T>, bool> { bool operator()(const boost::shared_ptr<T>& a, const boost::shared_ptr<T>& b) const { return less<T*>()(a.get(),b.get()); } }; template<typename T> struct less< boost::shared_array<T> > : binary_function<boost::shared_array<T>, boost::shared_array<T>, bool> { bool operator()(const boost::shared_array<T>& a, const boost::shared_array<T>& b) const { return less<T*>()(a.get(),b.get()); } }; } // namespace std #endif // ifndef BOOST_NO_TEMPLATE_PARTIAL_SPECIALIZATION #ifdef BOOST_MSVC # pragma warning(pop) #endif #endif // BOOST_SMART_PTR_HPP __________________________________________________ Do You Yahoo!? Got something to say? Say it better with Yahoo! Video Mail http://mail.yahoo.com
participants (3)
-
Colin Fox
-
Peter Dimov
-
Sebastien Marc