Does boost::thread_group::create_thread have memory leak?

pete＠mu.org

18 Jan 2007 18 Jan '07

6:33 a.m.

I've been trying to track down memory leak being reported by valgrind. There are others out there who appear to have the same problem, but I can't find any reference to it in the boost list archives. I know valgrind can be inaccurate, but it seems pretty accurate every where else in my program. I've narrowed it down to something like this: ----------------- #include #include #include <iostream> void fn(){ return; } int main() { boost::thread_group thrd; thrd.create_thread(boost::bind(&fn)); thrd.join_all(); return 0; } ----------------- Valgrind is showing this: ... by 0x4053D76: boost::thread_group::create_thread(boost::function0boost::function_base > const&) (in /usr/lib/libboost_thread-mt.so.1.33.1) LEAK SUMMARY: ==22004== definitely lost: 0 bytes in 0 blocks. ==22004== possibly lost: 144 bytes in 1 blocks. ==22004== still reachable: 0 bytes in 0 blocks. ==22004== suppressed: 0 bytes in 0 blocks. Any clues? Thoughts? Thanks, Pete

Show replies by date

Roland Schwarz

19 Jan 19 Jan

10:26 a.m.

New subject: Does boost::thread_group::create_thread have memory leak?

pete@mu.org wrote:

...

I've been trying to track down memory leak being reported by valgrind. <snipped> /usr/lib/libboost_thread-mt.so.1.33.1)

Version 1.33 is know to have a memory leak, but I am surprised that it shows up on linux. Can you repeat this with the 1.34 CVS version? Roland

pete＠mu.org

10:15 p.m.

New subject: Does boost::thread_group::create_thread have memory leak?

On Fri, Jan 19, 2007 at 11:26:50AM +0100, Roland Schwarz wrote:

...

pete@mu.org wrote:

...
I've been trying to track down memory leak being reported by valgrind. <snipped> /usr/lib/libboost_thread-mt.so.1.33.1)

Version 1.33 is know to have a memory leak, but I am surprised that it shows up on linux. Can you repeat this with the 1.34 CVS version?

I complied against the latest cvs version and still got the leak. boost::thread_group::create_thread(boost::function0boost::function_base > const&) (thread.cpp:338) It appears to come from this line in boost/lib/src/pthread.cpp: std::auto_ptr<thread> thrd(new thread(threadfunc)); Could it be that the leak is being created by using the tmp object in the thread creation? I've read advise about NOT doing that? I realized after reading your email again that 1.34 is what you specified, but I used 1.35. Would that be a problem? Thanks, Pete

Roland Schwarz

20 Jan 20 Jan

12:02 p.m.

New subject: Does boost::thread_group::create_thread have memory leak?

pete@mu.org wrote:

...

It appears to come from this line in boost/lib/src/pthread.cpp:

std::auto_ptr<thread> thrd(new thread(threadfunc));

I suspect you mean file: boost/libs/thread/src/thread.cpp.

...

Could it be that the leak is being created by using the tmp object in the thread creation? I've read advise about NOT doing that?

I am not sure if I understand you correctly. Could you please try to be more explicit? Which tmp object are you refering to? Where did you read what? I guess your original sample exhibits the leak? I'll try to reproduce locally: In order to get equivalent test conditions, could you post the exact steps you used to create and invoke the test?

...

I realized after reading your email again that 1.34 is what you specified, but I used 1.35. Would that be a problem?

No I don't expect so. They should currently be the same. Roland

pete＠mu.org

21 Jan 21 Jan

4:14 a.m.

New subject: Does boost::thread_group::create_thread have memory leak?

On Sat, Jan 20, 2007 at 01:02:56PM +0100, Roland Schwarz wrote:

...

pete@mu.org wrote:

...
It appears to come from this line in boost/lib/src/pthread.cpp: I suspect you mean file: boost/libs/thread/src/thread.cpp.

Doh! Sorry, you are correct.

...

...
Could it be that the leak is being created by using the tmp object in the thread creation? I've read advise about NOT doing that?

I am not sure if I understand you correctly. Could you please try to be more explicit?

Which tmp object are you refering to? Where did you read what?

Sorry again, as I was not correct when I said that. I spoke before I should have. I was not paying attention to the auto_ptr, it is not a tmp. However, It looks like thread_group::create_thread is really just calling pthread_create in my case. ie., Linux OS. Is is possible that this could be an issue with new/delete vs malloc/free? I'm pretty sure that pthread_create uses malloc, and on line 331 thread_group::~thread_group there is a delete taking place for all those pointers. thread_group::~thread_group() { // We shouldn't have to scoped_lock here, since referencing this object // from another thread while we're deleting it in the current thread is // going to lead to undefined behavior any way. for (std::list::iterator it = m_threads.begin(); it != m_threads.end(); ++it) { delete (*it); } } I'm still relatively new to c++ and even newer to boost, so maybe I'm way off. However, I'm mentioning it because it seems to raise a flag with Effective C++'s recommendation. Also, I don't think even free'ing a joined pthread is necessary in the first place, so I'm kinda confused about what's going on here or what the effect might be. Any other ideas? Thanks, Pete

...

I guess your original sample exhibits the leak?

I'll try to reproduce locally: In order to get equivalent test conditions, could you post the exact steps you used to create and invoke the test?

...
I realized after reading your email again that 1.34 is what you specified, but I used 1.35. Would that be a problem?

No I don't expect so. They should currently be the same.

Roland

...

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Ovanes Markarian

20 Jan 20 Jan

1:43 p.m.

New subject: Does boost::thread_group::create_thread havememory leak?

Pete, I yesterday faced the bug as well. Funny thing is, that even Boost Unit Testing Framework reports the memory leak. It is really simple to reproduce: struct some_thread_function { void operator()()const { //really simple code here, to make some calculations in a thread ;) } } void test_threading_lib() { { boost::thread_group grp; for(int i=0; i<100; ++i) //create 100 threads grp.create_thread(some_thread_function()); grp.join_all(); } BOOST_MESSAGE( "Memory leak should be identified..." ); } This already produces a mem leak in unit testing framework: test_suite* init_unit_test_suite( int argc, char* argv[] ) { test_suite* test = BOOST_TEST_SUITE( "Test for memory leak" ); test->add( BOOST_TEST_CASE( &test_threading_lib ) ); return test; } Thanks, Ovanes -----Original Message----- From: pete@mu.org [mailto:pete@mu.org] Sent: Friday, January 19, 2007 11:15 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Does boost::thread_group::create_thread havememory leak? On Fri, Jan 19, 2007 at 11:26:50AM +0100, Roland Schwarz wrote:

...

pete@mu.org wrote:

...
I've been trying to track down memory leak being reported by valgrind. <snipped> /usr/lib/libboost_thread-mt.so.1.33.1)

Version 1.33 is know to have a memory leak, but I am surprised that it shows up on linux. Can you repeat this with the 1.34 CVS version?

Sohail Somani

4:47 p.m.

New subject: Does boost::thread_group::create_threadhavememory leak?

I had the same issues, but only on Windows. I had expected that was the unfreed critical section /singleton (?) that has been the topic of such discussions previously. -----Original Message----- From: boost-users-bounces@lists.boost.org on behalf of Ovanes Markarian Sent: Sat 1/20/2007 5:43 AM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Does boost::thread_group::create_threadhavememory leak? Pete, I yesterday faced the bug as well. Funny thing is, that even Boost Unit Testing Framework reports the memory leak. It is really simple to reproduce:

Roland Schwarz

7:06 p.m.

New subject: Does boost::thread_group::create_thread have memory leak?

Pete, could you please post the output from your valgrind run? Thanky you, Roland

pete＠mu.org

21 Jan 21 Jan

3:53 a.m.

New subject: Does boost::thread_group::create_thread have memory leak?

Sorry, I've been out of town and I just got back in. Here you go: $ valgrind -v threadgroup ==19432== Memcheck, a memory error detector. ==19432== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al. ==19432== Using LibVEX rev 1606, a library for dynamic binary translation. ==19432== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. ==19432== Using valgrind-3.2.0, a dynamic binary instrumentation framework. ==19432== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al. ==19432== --19432-- Command line --19432-- threadgroup --19432-- Startup, with flags: --19432-- --memcheck:leak-check=full --19432-- --memcheck:error-limit=no --19432-- --memcheck:num-callers=20 --19432-- --memcheck:leak-resolution=high --19432-- --memcheck:show-reachable=yes --19432-- --memcheck:show-below-main=yes --19432-- --memcheck:trace-children=yes --19432-- --memcheck:gen-suppressions=no --19432-- -v --19432-- Contents of /proc/version: --19432-- Linux version 2.6.18-gentoo-r3 () (gcc version 4.1.1 (Gentoo 4.1.1-r1)) #7 SMP Fri Dec 8 15:47:16 CST 2006 --19432-- Arch and hwcaps: X86, x86-sse1-sse2 --19432-- Valgrind library directory: /usr/lib/valgrind --19432-- Reading syms from /lib/ld-2.4.so (0x4000000) --19432-- Reading syms from /home/pete/src/C_tests/boost_threads_group/threadgroup (0x8048000) --19432-- Reading syms from /usr/lib/valgrind/x86-linux/memcheck (0x38000000) --19432-- object doesn't have a symbol table --19432-- object doesn't have a dynamic symbol table --19432-- Reading suppressions file: /usr/lib/valgrind/default.supp --19432-- REDIR: 0x4014610 (index) redirected to 0x3801CC0B (???) --19432-- Reading syms from /usr/lib/valgrind/x86-linux/vgpreload_core.so (0x401C000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so (0x401E000) --19432-- object doesn't have a symbol table ==19432== WARNING: new redirection conflicts with existing -- ignoring it --19432-- new: 0x04014610 (index ) R-> 0x04020DCA index --19432-- Reading syms from /home/pete/lib/libboost_thread-gcc41-mt-d-1_35.so.1.35.0 (0x4023000) --19432-- Reading syms from /usr/lib/gcc/i686-pc-linux-gnu/4.1.1/libstdc++.so.6.0.8 (0x4085000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /lib/libm-2.4.so (0x415E000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /usr/lib/gcc/i686-pc-linux-gnu/4.1.1/libgcc_s.so.1 (0x4182000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /lib/libc-2.4.so (0x418D000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /lib/librt-2.4.so (0x42A4000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /lib/libpthread-2.4.so (0x42AD000) --19432-- REDIR: 0x41F4E50 (memset) redirected to 0x402135B (memset) --19432-- REDIR: 0x41F52F0 (memcpy) redirected to 0x4021106 (memcpy) --19432-- REDIR: 0x41F4000 (rindex) redirected to 0x4020D1A (rindex) --19432-- REDIR: 0x4131C80 (operator new(unsigned)) redirected to 0x4020872 (operator new(unsigned)) --19432-- REDIR: 0x41F0CE9 (calloc) redirected to 0x401F626 (calloc) --19432-- REDIR: 0x4130908 (operator delete(void*)) redirected to 0x401FB95 (operator delete(void*)) --19432-- REDIR: 0x41EF06E (free) redirected to 0x401FE3D (free) ==19432== ==19432== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 25 from 1) --19432-- --19432-- supp: 25 Fedora-Core-5-hack3-ld24 ==19432== malloc/free: in use at exit: 144 bytes in 1 blocks. ==19432== malloc/free: 3 allocs, 2 frees, 164 bytes allocated. ==19432== ==19432== searching for pointers to 1 not-freed blocks. ==19432== checked 8,515,228 bytes. ==19432== ==19432== 144 bytes in 1 blocks are possibly lost in loss record 1 of 1 ==19432== at 0x401F6A5: calloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==19432== by 0x401017A: allocate_dtv (in /lib/ld-2.4.so) ==19432== by 0x4010224: _dl_allocate_tls (in /lib/ld-2.4.so) ==19432== by 0x42B289F: pthread_create@@GLIBC_2.1 (in /lib/libpthread-2.4.so) ==19432== by 0x4050213: boost::thread::thread(boost::function0boost::function_base > const&) (thread.cpp:171) ==19432== by 0x40502F3: boost::thread_group::create_thread(boost::function0boost::function_base > const&) (thread.cpp:338) ==19432== by 0x8048B49: main (threadgroup.cpp:12) ==19432== by 0x41A2863: (below main) (in /lib/libc-2.4.so) ==19432== by 0x8048A08: (within /home/pete/src/C_tests/boost_threads_group/threadgroup) ==19432== ==19432== LEAK SUMMARY: ==19432== definitely lost: 0 bytes in 0 blocks. ==19432== possibly lost: 144 bytes in 1 blocks. ==19432== still reachable: 0 bytes in 0 blocks. ==19432== suppressed: 0 bytes in 0 blocks. --19432-- memcheck: sanity checks: 3 cheap, 1 expensive --19432-- memcheck: auxmaps: 0 auxmap entries (0k, 0M) in use --19432-- memcheck: auxmaps: 0 searches, 0 comparisons --19432-- memcheck: SMs: n_issued = 14 (224k, 0M) --19432-- memcheck: SMs: n_deissued = 0 (0k, 0M) --19432-- memcheck: SMs: max_noaccess = 65535 (1048560k, 1023M) --19432-- memcheck: SMs: max_undefined = 0 (0k, 0M) --19432-- memcheck: SMs: max_defined = 162 (2592k, 2M) --19432-- memcheck: SMs: max_non_DSM = 14 (224k, 0M) --19432-- memcheck: max sec V bit nodes: 0 (0k, 0M) --19432-- memcheck: set_sec_vbits8 calls: 0 (new: 0, updates: 0) --19432-- memcheck: max shadow mem size: 528k, 0M --19432-- translate: fast SP updates identified: 4,395 ( 92.4%) --19432-- translate: generic_known SP updates identified: 261 ( 5.4%) --19432-- translate: generic_unknown SP updates identified: 100 ( 2.1%) --19432-- tt/tc: 6,275 tt lookups requiring 6,452 probes --19432-- tt/tc: 6,275 fast-cache updates, 2 flushes --19432-- transtab: new 3,126 (67,048 -> 1,127,737; ratio 168:10) [0 scs] --19432-- transtab: dumped 0 (0 -> ??) --19432-- transtab: discarded 0 (0 -> ??) --19432-- scheduler: 395,790 jumps (bb entries). --19432-- scheduler: 3/3,301 major/minor sched events. --19432-- sanity: 4 cheap, 1 expensive checks. --19432-- exectx: 30,011 lists, 13 contexts (avg 0 per list) --19432-- exectx: 30 searches, 17 full compares (566 per 1000) --19432-- exectx: 0 cmp2, 67 cmp4, 0 cmpAll Here's the code: #include #include #include <iostream> void fn(void){ } int main() { boost::thread_group thrd; thrd.create_thread(&fn); thrd.join_all(); } On Sat, Jan 20, 2007 at 08:06:45PM +0100, Roland Schwarz wrote:

...

Pete,

could you please post the output from your valgrind run?

Thanky you, Roland

...

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Peter Dimov

1:57 p.m.

New subject: Does boost::thread_group::create_threadhave memory leak?

pete@mu.org wrote:

...

==19432== 144 bytes in 1 blocks are possibly lost in loss record 1 of 1 ==19432== at 0x401F6A5: calloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==19432== by 0x401017A: allocate_dtv (in /lib/ld-2.4.so) ==19432== by 0x4010224: _dl_allocate_tls (in /lib/ld-2.4.so) ==19432== by 0x42B289F: pthread_create@@GLIBC_2.1 (in /lib/libpthread-2.4.so) ==19432== by 0x4050213: boost::thread::thread(boost::function0boost::function_base > const&) (thread.cpp:171)

As you can see, the block has been allocated by pthread_create, not by Boost.Threads.

Ovanes Markarian

4:23 p.m.

New subject: Doesboost::thread_group::create_threadhave memory leak?

Ok, But why do I get a mem leak with VC 8 Express on windows if I run Unit Tests? I will try to investigate more on this issue. Thanks, Ovanes -----Original Message----- From: Peter Dimov [mailto:pdimov@mmltd.net] Sent: Sunday, January 21, 2007 2:57 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Doesboost::thread_group::create_threadhave memory leak? pete@mu.org wrote:

...

==19432== 144 bytes in 1 blocks are possibly lost in loss record 1 of 1 ==19432== at 0x401F6A5: calloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==19432== by 0x401017A: allocate_dtv (in /lib/ld-2.4.so) ==19432== by 0x4010224: _dl_allocate_tls (in /lib/ld-2.4.so) ==19432== by 0x42B289F: pthread_create@@GLIBC_2.1 (in /lib/libpthread-2.4.so) ==19432== by 0x4050213: boost::thread::thread(boost::function0boost::function_base > const&) (thread.cpp:171)

As you can see, the block has been allocated by pthread_create, not by Boost.Threads. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Roland Schwarz

6:17 p.m.

New subject: Does boost::thread_group::create_threadhave memory leak?

Peter Dimov wrote:

...

As you can see, the block has been allocated by pthread_create, not by Boost.Threads.

Pete, as already pointed out by Peter, the leak seems to be caused by the pthread library, not boost. Most likely it isn't harmful. It looks independent of the number of threads created. I guess it is related to thread local storage management. You might try the below code snippet, which gives me the same valgrind report than the example using Boost.Thread: #include void* fn(void* p) { return NULL; } int main() { pthread_t th; pthread_create(&th, NULL, fn, NULL); pthread_join(th, NULL); return 0; } Regards, Roland

Sohail Somani

7:01 p.m.

New subject: [Boost-users] Does boost::thread_group::create_threadhave memory leak?

-----Original Message----- From: boost-users-bounces@lists.boost.org on behalf of Roland Schwarz Peter Dimov wrote:

...

As you can see, the block has been allocated by pthread_create, not by Boost.Threads.

Pete, as already pointed out by Peter, the leak seems to be caused by the pthread library, not boost. Most likely it isn't harmful. It looks independent of the number of threads created. I guess it is related to thread local storage management. ----- I ran the OPs tests and I agree with your statement, except that it *does* depend on the number of threads. If I created two threads -> I got two leaks.

Peter Dimov

9:30 p.m.

New subject: Does boost::thread_group::create_thread have memory leak?

Sohail Somani wrote:

...

I ran the OPs tests and I agree with your statement, except that it *does* depend on the number of threads. If I created two threads -> I got two leaks.

Try joining the first thread before creating a second one; it's likely that the block will be reused by libpthread (something that can't happen with a leak).

Sohail Somani

10:03 p.m.

New subject: Does boost::thread_group::create_thread havememory leak?

-----Original Message----- From: boost-users-bounces@lists.boost.org on behalf of Peter Dimov Sent: Sun 1/21/2007 1:30 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Does boost::thread_group::create_thread havememory leak? Sohail Somani wrote:

...

I ran the OPs tests and I agree with your statement, except that it *does* depend on the number of threads. If I created two threads -> I got two leaks.

Try joining the first thread before creating a second one; it's likely that the block will be reused by libpthread (something that can't happen with a leak). ----- This is correct. The OP should put in some valgrind suppression rules I suppose (i.e., not a leak). Would it make sense to address this and the usual "boost threads has a leak on windows" question in the boost threads documentation? Perhaps under the "Implementation Notes" section?

pete＠mu.org

22 Jan 22 Jan

12:36 a.m.

New subject: [Boost-users] Does boost::thread_group::create_threadhave memory leak?

On Sun, Jan 21, 2007 at 07:17:42PM +0100, Roland Schwarz wrote:

...

You might try the below code snippet, which gives me the same valgrind report than the example using Boost.Thread:

#include

void* fn(void* p) { return NULL; }

int main() { pthread_t th; pthread_create(&th, NULL, fn, NULL); pthread_join(th, NULL); return 0; }

I noticed with the straight pthread code that if I compiled it as ANSI C, the leak was smaller. It sure seems like this should be a well known issue. Here is the ANSI C (gcc) valgrind report: ==5584== LEAK SUMMARY: ==5584== definitely lost: 0 bytes in 0 blocks. ==5584== possibly lost: 136 bytes in 1 blocks. ==5584== still reachable: 0 bytes in 0 blocks. ==5584== suppressed: 0 bytes in 0 blocks. Here is the same code but compiled with C++ (g++): ==5568== LEAK SUMMARY: ==5568== definitely lost: 0 bytes in 0 blocks. ==5568== possibly lost: 144 bytes in 1 blocks. ==5568== still reachable: 0 bytes in 0 blocks. Thanks, I guess it's not a boost issue at all. Pete

...

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

6505

Age (days ago)

6509

Last active (days ago)

List overview

Download

15 comments

5 participants

participants (5)

Ovanes Markarian
pete＠mu.org
Peter Dimov
Roland Schwarz
Sohail Somani