Re: [Boost-users] [boost::pool] using many pools in the application
Hi,
Since little help is forthcoming, here are some pointers:
I did look into the pool library some time back (boost version 1.34.1) in search of a solution to some severe performance and memory management issues. In short, I was rather disappointed, for various reasons:
1. memory blocks allocated by boost::fast_pool tended to never get freed again until the end of the program, because the release mechanism was (is?) disabled. This meant my memory management problems got even worse!
2. modifying the code to enable the release mechanism didn't help - for some reason it never got called!
3. the basic method of adjusting the pool size in case it ran out of memory, was doubling it's size. This aggravated the above problems through allocating a lot more memory than was needed most of the time. In fact, when I tested it, fast_pool eventually ran out of memory at about the same time the the local member variable storing the memory increment size sufferd an overflow!
I managed to fix this locally by deriving my own pool class from the existing one and overriding a method. In effect I restricted the maximum memory block size to a couple of MB. It worked for me, but wasn't sufficiently general as a fix for the library, especially as I still didn't understand all of the inner workings at that time.
4. support for the pool library seems to have been dropped several years ago, although someone did a few minor fixes a couple of months ago (I believe they were incorporated in version 1.35).
Regarding your question, the general idea of the boost::pool classes are singleton pool objects that each serve any object of either a particular type (object_pool) or a particular size (all others if I understood them correctly), so depending on the kind of pool you use you get exactly one pool object per type or object size, not multiple.
To fix the issues you mentioned, you (or someone else) would have to do several things:
1. fix the memory allocation so it is limited to some reasonable size (I'd suggest to add this maximumum blocksize to the parameter list either of the pool constructors or the templates)
2. fix the release mechanism. What I would do is add a usage counter to each memory block that allows for easy control of emptied blocks. They should work similar to shared pointer usage semantics.
3. Add a mechanism to monitor fill status and do some kind of garbage collection. This is where it's starting to get tricky: If you find a memory block that is almost 'empty' then you might want to shove the remaining data somewhere else - but you can't do so if someone outside your control is pointing to it! It would be possible though if you never hand out raw pointers to pooled objects, but shared pointers. But then you would need to retain a copy of each shared pointer as well. And this would be overkill for objects that take up only a few bytes each!
I could go on some more - I did think about it for quite some time. But I think I already gave you a notion of what to expect.
Conclusion: consider what you would need a pool for in terms of individual object size, total size, fluctuation, and so on. Then check out the boost documentation on the four pool types it provides and check whether any of the four types would match your requirements in theory. Once you're there, think again, especially in light of the missing deallocation problem: can you afford to effectively keep a chunk of memory till the end of the program, a chunk that is big enough to store the maximum amount of pooled objects and probably quite a bit more? If you can, then give it a try. If not, ...
HTH,
Stefan
-----Ursprüngliche Nachricht-----
Date: Wed, 29 Oct 2008 15:34:53 +0200
From: "Roman Shmelev"
participants (1)
-
Lang Stefan