Re: Memory leak reported using threads library
I don't mind any one off small leaks. They may not be nice but no one is going to notice them. What bothers me is when memory keeps leaking for each iteration of the loop. I don't bother with the microsoft crt leak stuff. It's rarely that much use in complex projects and even if it does detect a leak it rarely can tell you where it is. As it turns out the crt leak detect stuff doesn't seem to show any leak (at least on my system), but lets say I put in my own my own leak of say new char[10] or something then the leak messages pour out. Use windows performance counters however and it's a different story, watch the working set rocket up. And it seems to increase at different rates depending on what the thread is doing. Eventually you will run out of memory entirely. Use the crt dll in your main program however and the working set is dead flat.
What I don't understand is your mentioning of the boost::function lib. Where do you think it is getting used in your example?
boost::function is being used in the constructor of thread i believe. Has anyone been able to confirm this leak or is it just me? Naturally boost::threads is a dll only (at least v 1.30 which i'm currently using) and it itself can either link dynamically or statically to the crt. It doesn't make any difference which though. I have to link dynamically to crt in my program to avoid the leak. Why should this make a difference? I don't particular want to upgrade to 1.32 for my project as it's rather late in the day. I don't want any other issues creeping in and I've resolved the issue by using the dll. However I thought I should report my findings and see if anyone knows anything about it. Do we know if there is no leak in 1.32 or is it still there? Thanks. Alan
alansa@ati-uk.com wrote:
I don't mind any one off small leaks. They may not be nice but no one is going to notice them. What bothers me is when memory keeps leaking for each iteration of the loop. <snip> Has anyone been able to confirm this leak or is it just me?
I think most people have learned from bitter experience to avoid trouble by using the DLL configuration of the CRT.
Naturally boost::threads is a dll only (at least v 1.30 which i'm currently using) and it itself can either link dynamically or statically to the crt. It doesn't make any difference which though. I have to link dynamically to crt in my program to avoid the leak. Why should this make a difference? <snip>
Multithreaded configurations of MSVCRT allocates some per-thread memory as needed, e.g. for the object that localtime returns a pointer to. To ensure that this memory is freed at thread exit, you must either use a DLL configuration of the CRT or ensure that all threads that call into it are started using the _beginthread or _beginthreadex function. The Boost.Thread DLL is not (and cannot be) linked to the same CRT that is statically linked into your executable, so neither of these conditions is satisfied. If I remember correctly, any memory allocated in your executable cannot be deallocated in the Boost.Thread DLL, or vice versa, since they have separate heaps. I would expect an attempt to do so to result in a crash, but it might in some cases be silently ignored, resulting in a leak. Ben.
<alansa <at> ati-uk.com> writes:
I don't mind any one off small leaks. They may not be nice but no one is going to notice them. What bothers me is when memory keeps leaking for each iteration of the loop.
I don't bother with the microsoft crt leak stuff. It's rarely that much use in complex projects and even if it does detect a leak it rarely can tell you where it is. As it turns out the crt leak detect stuff doesn't seem to show any leak (at least on my system), but lets say I put in my own my own leak of say new char[10] or something then the leak messages pour out.
Well I totally disagree. I am working now on a project, which solution consists of 73 vc-projects (more than 2500 files) and it does not leak memory in its main loop, thanks to the debug crt of m$. We have overwritten the new and delete operator like m$ in MFC does so we get accurate memory leak reporting. If you take this business serious one can not allow any memory leak reported by the debug crt. And of course if you use rare fct's like VirtualAlloc or GlobalAlloc, they are not reported, but there is really no need to use them. Even the speed advantage can be argued about. Wkr, me
Naturally boost::threads is a dll only (at least v 1.30 which i'm currently using) and it itself can either link dynamically or statically to the crt. It doesn't make any difference which though. I have to link dynamically to crt in my program to avoid the leak. Why should this make a difference?
That's your problem: there must be one single rtl shared both by your code and the Boost.Threads dll: and that means using the dynamic C runtime. Frankly I'm surprised your code didn't just crash if you were mixing runtimes :-( John.
John Maddock wrote:
That's your problem: there must be one single rtl shared both by your code and the Boost.Threads dll: and that means using the dynamic C runtime.
That surprises me somewhat. Explicitely: mydll depending on dynamic CRT. myapp depending on static CRT and depending on mydll. You think this is not possible? Just tried it. Found no obvious problem. Are there hidden traps? If this would not be possible you never could write a self contained dll ! But I think I didn't understand your argument corectly, could you elaborate please? Hmm, when I rethink the issue the following might happen: Writing a app that uses static CRT, then requesting for the DLL version of a boost lib. Could it be that the autolink feature pulls in dll versions of CRT? If this is the case this points out a problem with the autolink, and has nothing to do with the unfeasability of the above scenario. Roland
Roland Schwarz wrote:
John Maddock wrote:
That's your problem: there must be one single rtl shared both by your code and the Boost.Threads dll: and that means using the dynamic C runtime.
That surprises me somewhat. Explicitely:
mydll depending on dynamic CRT.
myapp depending on static CRT and depending on mydll.
You think this is not possible? Just tried it. Found no obvious problem. Are there hidden traps?
It works as long as you don' allocate in one and delete in the other. C-style interfaces are fine, but passing a std::string from the EXE to the DLL (or vice versa) will fail. shared_ptr will work, though :-) unless you unload the DLL that has created it, of course. Deleting an object with a virtual destructor works, too.
Peter Dimov wrote:
It works as long as you don' allocate in one and delete in the other. C-style interfaces are fine, but passing a std::string from the EXE to the DLL (or vice versa) will fail. shared_ptr will work, though :-) unless you unload the DLL that has created it, of course. Deleting an object with a virtual destructor works, too.
That makes sense to me. A very convincing argument! I would suggest to add this into the comment section of autolink.hpp: #if (defined(_DLL) || defined(_RTLDLL)) && defined(BOOST_DYN_LINK) # define BOOST_LIB_PREFIX #elif defined(BOOST_DYN_LINK) # error "Mixing a dll boost library with a static runtime is a really bad idea..." #else # define BOOST_LIB_PREFIX "lib" #endif I really did not figure out from the message why this was a "really bad idea..." (Since there is no obvious technical reason.) BTW.: I would even prefer this being a warning, since as long as beeing careful there are usages where it could be of advantage. Roland
I inadvertently replied to the wrong posting. John Maddock wrote:
That's your problem: there must be one single rtl shared both by your code and the Boost.Threads dll: and that means using the dynamic C runtime.
That surprises me somewhat. Explicitely: mydll depending on dynamic CRT. myapp depending on static CRT and depending on mydll. You think this is not possible? Just tried it. Found no obvious problem. Are there hidden traps? If this would not be possible you never could write a self contained dll ! But I think I didn't understand your argument corectly, could you elaborate please? Hmm, when I rethink the issue the following might happen: Writing a app that uses static CRT, then requesting for the DLL version of a boost lib. Could it be that the autolink feature pulls in dll versions of CRT? If this is the case this points out a problem with the autolink, and has nothing to do with the unfeasability of the above scenario. Roland
There are traps in the MS world. Some CRT functions (think strtok - yuk) use static data. If you build with the statically linked version of the library, each executable (.exe AND .dll) has it's own copy of that static data and they don't know about each other. All executables built with the dynamically linked version share the same copy of that static data. If you use the dynamically linked version then you should (on Win NT and up systems - not Win 9x) share the same copy of the code with all other apps in the system that use the CRT. HTH - Richard -----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Roland Schwarz Sent: 25 November 2004 10:45 To: boost-users@lists.boost.org Subject: Re: [Boost-users] Re: Memory leak reported using threads library I inadvertently replied to the wrong posting. John Maddock wrote:
That's your problem: there must be one single rtl shared both by your code and the Boost.Threads dll: and that means using the dynamic C runtime.
That surprises me somewhat. Explicitely: mydll depending on dynamic CRT. myapp depending on static CRT and depending on mydll. You think this is not possible? Just tried it. Found no obvious problem. Are there hidden traps? If this would not be possible you never could write a self contained dll ! But I think I didn't understand your argument corectly, could you elaborate please? Hmm, when I rethink the issue the following might happen: Writing a app that uses static CRT, then requesting for the DLL version of a boost lib. Could it be that the autolink feature pulls in dll versions of CRT? If this is the case this points out a problem with the autolink, and has nothing to do with the unfeasability of the above scenario. Roland _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users --- Incoming mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.797 / Virus Database: 541 - Release Date: 15/11/2004 --- Outgoing mail is Virus checked. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.797 / Virus Database: 541 - Release Date: 15/11/2004
Richard Howells wrote:
There are traps in the MS world. Some CRT functions (think strtok - yuk) use static data. If you build with the statically linked version of the library, each executable (.exe AND .dll) has it's own copy of that static data and they don't know about each other.
All executables built with the dynamically linked version share the same copy of that static data.
If you use the dynamically linked version then you should (on Win NT and up systems - not Win 9x) share the same copy of the code with all other apps in the system that use the CRT.
I can understand this. But I cannot see why this will be a problem. Case 1): Singgle threaded: As long as one isn't calling functions that modify this data in between no harm is done, whether two instances of static data or not. Case 2): Multi threaded: Every thread has its own local copy of static data in TLS storage. No harm again. But perhaps I am still simply not beeing able to see the trap? BTW.: Having a separate copy of the data in the (boostified) DLL will improve robustness, not making it worse. Altough it was recommended using the _same_ CRT which gives rise to the shared static data issue. Roland
I think the classic example would be errno. If a call in your DLL set errno and you attempted to check errno back in your EXE then: - if you link BOTH with the DLL version you will both share the same errno and will get the error code correctly - if EITEHR is linked with the static library you will be using DIFFERENT errno variables and will get the wrong error code. Does that help? - R -----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Roland Schwarz Sent: 25 November 2004 11:59 To: boost-users@lists.boost.org Subject: Re: [Boost-users] Re: Memory leak reported using threads library Richard Howells wrote:
There are traps in the MS world. Some CRT functions (think strtok - yuk) use static data. If you build with the statically linked version of the library, each executable (.exe AND .dll) has it's own copy of that static data and they don't know about each other.
All executables built with the dynamically linked version share the same copy of that static data.
If you use the dynamically linked version then you should (on Win NT and up systems - not Win 9x) share the same copy of the code with all other apps in the system that use the CRT.
I can understand this. But I cannot see why this will be a problem. Case 1): Singgle threaded: As long as one isn't calling functions that modify this data in between no harm is done, whether two instances of static data or not. Case 2): Multi threaded: Every thread has its own local copy of static data in TLS storage. No harm again. But perhaps I am still simply not beeing able to see the trap? BTW.: Having a separate copy of the data in the (boostified) DLL will improve robustness, not making it worse. Altough it was recommended using the _same_ CRT which gives rise to the shared static data issue. Roland _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users --- Incoming mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.797 / Virus Database: 541 - Release Date: 15/11/2004 --- Outgoing mail is Virus checked. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.797 / Virus Database: 541 - Release Date: 15/11/2004
Roland Schwarz <roland.schwarz <at> chello.at> writes:
I inadvertently replied to the wrong posting.
John Maddock wrote:
That's your problem: there must be one single rtl shared both by your code and the Boost.Threads dll: and that means using the dynamic C runtime.
That surprises me somewhat. Explicitely:
mydll depending on dynamic CRT.
myapp depending on static CRT and depending on mydll.
You think this is not possible? Just tried it. Found no obvious problem. Are there hidden traps?
This is an old-time problem MSVC programmers keep stumbling upon. As John points out, the only safe alternative is that every executable (EXE or DLL) in an app uses the same *dynamic* version of the CRT: all of them debug dynamic or all of them release dynamic. Every other option will casue you trouble. The biggest source of incompatibilites has to do with memory allocation. MSVC malloc/free (and new/delete) implementation uses an internal heap allocated at CRT initialization time. Every instance of the CRT has its own internal heap. Now, if module (EXE or DLL) A allocates some memory and module B tries to free it through another copy of the CRT, an instant crash is guranteed, because the memory does not belong in B's internal heap. Try it: select static CRT for every module, allocate some memory in your EXE and let some DLL try to free that memory. When everybody selects dynamic CRT, there's only one copy of it across the app, and you'll find no problem. If you restrict yourself to C-style calls where no ownership of memory is transferred, then everything is OK, but this is hardly possible in Boost libraries where C++ objects are passed around. In the case where the boundary between the EXE and the DLL is crossed only by object *pointers*, the problem still remains if a module tries to delete a pointer not allocated within its CRT, but in this case there's a simple workaround: if the class being pointed to has a virtual destructor, then it is safe to delete a pointer everywhere, since the destruction code, being virtual, will get executed within the module where the object was created (as the vtable will be pointing to that module). Hope this sheds some light on the issue. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo
participants (8)
-
alansa＠ati-uk.com -
Ben Hutchings -
gast128 -
Joaquin M Lopez Munoz -
John Maddock -
Peter Dimov -
Richard Howells -
Roland Schwarz