Memoryleaks in boost::pool_allocator
Some time ago i had noticed some strange behavior in the boost pool_allocator regarding its memory usage. Now that i've found these great mailinglists ;) i've reproduced the error and with some additional analysis i can post the thing here.
My environment:
- VS2005
- Windows XP
- Boost 1_34_1 with BOOST_ALL_DYN_LINK set (if it makes a difference)
- Running release build
The test code:
{
std::pair<int, int> pairStart(m_spLeakDetection->ReportLeaks(_T("boost test 1a")));
{
std::vector<int> v;
std::pair<int, int> pairInterim2(m_spLeakDetection->ReportLeaks(_T("boost test 1b"))); //0 bytes (only stack memory used)
for (int i = 0; i < 1000; ++i) { v.push_back(5); }
std::pair<int, int> pairInterim3(m_spLeakDetection->ReportLeaks(_T("boost test 1c"))); //4264 bytes in 1 block
v.clear();
}
std::pair<int, int> pairInterim4(m_spLeakDetection->ReportLeaks(_T("boost test 1d"))); //0 bytes
}
{
std::pair<int, int> pairStart(m_spLeakDetection->ReportLeaks(_T("boost test 2a")));
{
std::vector<int, boost::pool_allocator<int> > v;
std::pair<int, int> pairInterim2(m_spLeakDetection->ReportLeaks(_T("boost test 2b"))); //0 bytes (only stack memory used)
for (int i = 0; i < 1000; ++i) { v.push_back(5); }
std::pair<int, int> pairInterim3(m_spLeakDetection->ReportLeaks(_T("boost test 2c"))); //16312 bytes over 7 blocks; 136, 264, 520, 1032, 2056, 4104, 8200
v.clear();
}
std::pair<int, int> pairInterim4(m_spLeakDetection->ReportLeaks(_T("boost test 2d"))); //still 16312 bytes over 7 blocks
boost::singleton_pool<boost::pool_allocator_tag, sizeof(int)>::release_memory();
std::pair<int, int> pairEnd(m_spLeakDetection->ReportLeaks(_T("boost test 2e"))); //8112 over 6 blocks; 136, 264, 520, 1032, 2056, 4104
{
std::vector<int, boost::pool_allocator<int> > v;
for (int i = 0; i < 10; ++i) { v.push_back(5); }
std::pair<int, int> pairInterim3(m_spLeakDetection->ReportLeaks(_T("boost test 2f"))); //8112 over 6 blocks; 136, 264, 520, 1032, 2056, 4104
v.clear();
}
{
std::vector<int, boost::pool_allocator<int> > v;
for (int i = 0; i < 1000; ++i) { v.push_back(5); }
std::pair<int, int> pairInterim3(m_spLeakDetection->ReportLeaks(_T("boost test 2g"))); //24504 bytes over 7 blocks; 136, 264, 520, 1032, 2056, 4104, 16392
v.clear();
}
}
>From these tests;
- why does the boost::pool_allocator reserve 16312 bytes for 1000 int's while the default stl vector consumes (only) 4264 bytes?
- why does a call to release_memory( ) not remove all the memory allocations done by the pool_allocator? But only the last block?
- why does the size of each new block gets duplicated in size (even after release_memory)?
Does this make sense to any of you?
Best regards,
Paul
_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
Hi Paul, On Wed, Aug 27, 2008 at 2:44 AM, Paul Bormans <pbor1234@hotmail.com> wrote: > Some time ago i had noticed some strange behavior in the boost pool_allocator regarding its memory usage. Now that i've found these great mailinglists ;) i've reproduced the error and with some additional analysis i can post the thing here. > I have the same experience, although with Linux and GCC 4.1.2. More precisely: [snip] > > >From these tests; > - why does the boost::pool_allocator reserve 16312 bytes for 1000 int's while the default stl vector consumes (only) 4264 bytes? I think it has something to do with the way the pool_allocator grows, which is by doubling the available pool source every time it "runs out of available free contiguous space". > - why does a call to release_memory( ) not remove all the memory allocations done by the pool_allocator? But only the last block? I think this is the major bug that gets exacerbated in "high data churn applications" -- where lots of small containers acquire/release memory often, and the pool_allocator just doesn't know about the blocks that are supposedly "free" and "available" again. > - why does the size of each new block gets duplicated in size (even after release_memory)? > This I think is the intended behavior (which I think you mean is that the size of the whole memory block gets doubled every time). > Does this make sense to any of you? No, but it would be nice to know if indeed this is a bug and whether there should be additional tests to expose the bug. One thing I notice is that there's an interface to pool_allocator<...>::purge_memory() -- or something similar. Is this by any means part of the standard allocator interface and shouldn't this be called whenever a release is called as well? HTH -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
Dean Michael, I've just ran with purge_memory as well, and much to my surprise, all memory gets free then... Documentation explains that purge_memory invalidates each pointer to that block (while release_memory does only free memory that is no longer referenced) which makes me believe that there is a ref-leak of some kind. The memory cannot be referenced any more since the vector is out of scope, right? Even though purge_memory releases all memory previously allocated, i still see a problem in a scenario where the (singleton) pool is used, then purged/released frequently, it will quickly run out of memory, as shown below in the sample. To my opinion, the release_memory doesn't do what it is supposed to do and in addition i have doubts if the size of the next block is correct. What would be the best way to get this solved? (as i'm only a boost user and not a developer (yet ;))? In code with some results added in comments: ....continued from previous tests (see original post).... std::pair<int, int> pairInterim4(m_spLeakDetection->ReportLeaks(_T("boost test 2d"))); //still 16312 bytes over 7 blocks boost::singleton_pool<boost::pool_allocator_tag, sizeof(int)>::release_memory(); std::pair<int, int> pairInterim5(m_spLeakDetection->ReportLeaks(_T("boost test 2e1"))); //8112 over 6 blocks; 136, 264, 520, 1032, 2056, 4104 boost::singleton_pool<boost::pool_allocator_tag, sizeof(int)>::purge_memory(); std::pair<int, int> pairEnd(m_spLeakDetection->ReportLeaks(_T("boost test 2e2"))); //0 bytes { std::vector<int, boost::pool_allocator<int> > v; for (int i = 0; i < 10; ++i) { v.push_back(5); } std::pair<int, int> pairInterim3(m_spLeakDetection->ReportLeaks(_T("boost test 2f"))); //16392 bytes over 1 block v.clear(); } for(int count=0; count<15; ++count) { { std::vector<int, boost::pool_allocator<int> > v; for (int i = 0; i < 1000; ++i) { v.push_back(5); } std::pair<int, int> pairTest1(m_spLeakDetection->ReportLeaks(_T("boost test 2x1"))); //giving 16392, 32776, ..... 268435464 v.clear(); boost::singleton_pool<boost::pool_allocator_tag, sizeof(int)>::release_memory(); } boost::singleton_pool<boost::pool_allocator_tag, sizeof(int)>::purge_memory(); std::pair<int, int> pairTest2(m_spLeakDetection->ReportLeaks(_T("boost test 2x2"))); }
Date: Wed, 27 Aug 2008 16:49:51 +0800> From: mikhailberis@gmail.com> To: boost@lists.boost.org> Subject: Re: [boost] Memoryleaks in boost::pool_allocator> > Hi Paul,> > On Wed, Aug 27, 2008 at 2:44 AM, Paul Bormans <pbor1234@hotmail.com> wrote:> > Some time ago i had noticed some strange behavior in the boost pool_allocator regarding its memory usage. Now that i've found these great mailinglists ;) i've reproduced the error and with some additional analysis i can post the thing here.> >> > I have the same experience, although with Linux and GCC 4.1.2. More precisely:> > [snip]> >> > >From these tests;> > - why does the boost::pool_allocator reserve 16312 bytes for 1000 int's while the default stl vector consumes (only) 4264 bytes?> > I think it has something to do with the way the pool_allocator grows,> which is by doubling the available pool source every time it "runs out> of available free contiguous space".> > > - why does a call to release_memory( ) not remove all the memory allocations done by the pool_allocator? But only the last block?> > I think this is the major bug that gets exacerbated in "high data> churn applications" -- where lots of small containers acquire/release> memory often, and the pool_allocator just doesn't know about the> blocks that are supposedly "free" and "available" again.> > > - why does the size of each new block gets duplicated in size (even after release_memory)?> >> > This I think is the intended behavior (which I think you mean is that> the size of the whole memory block gets doubled every time).> > > Does this make sense to any of you?> > No, but it would be nice to know if indeed this is a bug and whether> there should be additional tests to expose the bug.> > One thing I notice is that there's an interface to> pool_allocator<...>::purge_memory() -- or something similar. Is this> by any means part of the standard allocator interface and shouldn't> this be called whenever a release is called as well?> > HTH> > -- > Dean Michael C. Berris> Software Engineer, Friendster, Inc.> _______________________________________________> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
Does anyone have hints/tips for me to get properly formatted time zone (%Q) for local time without providing the time zone myself? _________________________________________________________________ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
participants (2)
-
Dean Michael Berris -
Paul Bormans