Boost.Pool - Are you using this library, please can we have your input?
Boost.Pool is the subject of a 'make-over' by Boost.Guild. (This is some of a test-bed for a process of revision of older, perhaps orphaned, libraries using the relative safety of the sandbox). In order to permit any modification, the original (hand-written?) html documentation has been converted to Quickbook (and somewhat revised) at http://svn.boost.org/svn/boost/sandbox/guild/pool/libs/pool/doc/ You can build the html docs using the jamfile.v2 in the normal way, but you will probably find it more convenient to go direct to the pdf version http://svn.boost.org/svn/boost/sandbox/guild/pool/libs/pool/doc/pool.pdf This is fairly fully hyperlinked, and includes a reference section using Doxygen. Doxygen comments have been added to the header files (from the original html docs). (If you are a Doxygen fan, there is also a Doxyfile for generating Doxygen-style docs in the /doc/Doxygen/ folder). We are now considering some better tests, and perhaps some enhancements to the code. We'd like input from users about: 1 Things missing or wrong from the docs. More up to date references? 2 Things missing (or don't work well) from Pool. 3 New things that you would like (this is not a promise!). 4 Examples of your use. Perhaps you built some during your development that could be used? 5 Better tests. Thanks. Paul PS Don't panic - the release version will not be changed without a mini-review. --- Paul A. Bristow, Prizet Farmhouse, Kendal LA8 8AB UK +44 1539 561830 07714330204 pbristow@hetp.u-net.com
On Tue, 18 Jan 2011, Paul A. Bristow wrote:
Boost.Pool is the subject of a 'make-over' by Boost.Guild.
(This is some of a test-bed for a process of revision of older, perhaps orphaned, libraries using the relative safety of the sandbox).
(snip)
We are now considering some better tests, and perhaps some enhancements to the code.
We'd like input from users about:
1 Things missing or wrong from the docs. More up to date references?
2 Things missing (or don't work well) from Pool.
Using pools with small allocation chunks (to get behavior like malloc/new) is very slow. It might be nice to have a type of pool that keeps variable-sized blocks, like current malloc implementations do.
3 New things that you would like (this is not a promise!).
The ability to store the pool metadata outside the pool itself. The case where one would use this is when the pool is being used to allocate a special kind of memory, and the pool metadata (headers and free list) doesn't need to be in that kind of memory. A shared_ptr deleter for objects allocated from pools. A pool UserAllocator that gets its memory from another pool (to allow a pool of large blocks to be broken into smaller pieces to store individual objects). A thread-safe pool (preferably including lock-free or semi-lock-free algorithms when possible).
4 Examples of your use. Perhaps you built some during your development that could be used?
I was using a pool (simple_segregated_storage) to manage pages of memory that are registered with GASNet for message passing. In its fastest mode, GASNet requires a contiguous block of memory to be registered when the program starts, and I used a pool to allocate data structures in this block. Additionally, a pool is a safe way to allocate memory in a signal context (since it is possible to lock accesses to the pool), while malloc/... are not signal-safe. -- Jeremiah Willcock
One of my pet peeves is having a embedded friendly pool that register its memeory block in some stack allcoated huge array. While doing shenanigans on the Cell processor i often wanted to have a pool pre-requesting 200 of the 256Kb of the main memory and lemme allocate in this while havign some hard error if i stomped over the limit instead of silently doing crap. Other feature I would like is having some way to say: here is some contiguous memory range, please use it as a pool. So i can have a pool i can feed the data bank afterward. On other side, I really wish the pool_allocator to be under heavy test scurtiny. I used it once and got some strange cannot reproduce bugs with it. My use case was using a pool of float/double to allocate a block for holding some temporary matrix resulting from a call of GEMM int he middle of a proto AST.
On 18/01/2011 12:58, Paul A. Bristow wrote:
We'd like input from users about:
1 Things missing or wrong from the docs. More up to date references?
BOOST_PREVENT_MACRO_SUBSTITUTION should probably not appear in the docs. BOOST_STATIC_CONSTANT should probably be expanded. new[] appears as new\[\] in the docs. Page 30, item 11, the description of pool::free seems to be wrong. Also, did you really mean for protected member functions to appear in the docs? Some directly refer to types in the details namespace. The documentation of next_of is clearly lacking. What does it do? Is it a mechanism to iterate the pool?
2 Things missing (or don't work well) from Pool.
I've never used Boost.Pool before, but I'm considering using it now in a compiler project for the management of the nodes of the AST, which are all of the same type (a boost::variant). I'm a bit puzzled at one thing though: why is object_pool::free O(n)? Couldn't it be O(1) like for the other interfaces? I suppose that's because it tries to be order preserving, but I don't see any reason why that should be forced on users.
3 New things that you would like (this is not a promise!).
Ability to choose the boundary used when reclaiming memory from the system.
AMDG On 1/18/2011 1:36 PM, Mathias Gaunard wrote:
I'm a bit puzzled at one thing though: why is object_pool::free O(n)? Couldn't it be O(1) like for the other interfaces? I suppose that's because it tries to be order preserving, but I don't see any reason why that should be forced on users.
It has to be order preserving to make the destructor work. In Christ, Steven Watanabe
On 18/01/2011 22:43, Steven Watanabe wrote:
AMDG
On 1/18/2011 1:36 PM, Mathias Gaunard wrote:
I'm a bit puzzled at one thing though: why is object_pool::free O(n)? Couldn't it be O(1) like for the other interfaces? I suppose that's because it tries to be order preserving, but I don't see any reason why that should be forced on users.
It has to be order preserving to make the destructor work.
Albeit that's true in the general case, I believe it would still work fine for many cases.
On Tue, Jan 18, 2011 at 3:58 AM, Paul A. Bristow <pbristow@hetp.u-net.com> wrote:
Boost.Pool is the subject of a 'make-over' by Boost.Guild. ... 2 Things missing (or don't work well) from Pool. 3 New things that you would like (this is not a promise!).
It has been some years since I've given up using Pool, so bear with me if I've forgotten some things about it. I have often implemented my own pools, instead. 1. Arenas -- supports allocating any type of any size, and it'll allocate from larger contiguous blocks like a pool does. Still zero-overhead. Deallocation is a no-op. Good for creating large read-only data structures of many types that should all be destroyed at once. 2. Lock-free versions. The current global locks make them pretty useless when you've got allocation-heavy concurrency. Very simple to implement if you drop the ordered requirement. 3. Hierarchical pools. You could for instance have a multi-threaded arena servicing a multi-threaded pool<4096> servicing a local single-threaded pool<128>. 4. Don't force object destruction on pool destruction. Fast destruction all-at-once is an important feature for pools -- my types might have useful destructors when the pool is alive, but it might be the case that none of them matter when the pool is being destroyed. 5. Don't force ordering. I might not need it. 6. Let me specify alignment. Useful for SSE which requires 16-byte alignment for good performance, to generally ensure good locality, and for concurrency where false sharing in cache lines can be devastating. aligned_storage etc. is not enough because it doesn't work for sizes determined at run-time. 4 and 5 are "shoot your own foot off" features, but seeing how pools are used for performance, I'd argue that all is fair and it should have a flexible policy to support it. -- Cory Nelson
On 18/01/11 23:37, Cory Nelson wrote:
2. Lock-free versions. The current global locks make them pretty useless when you've got allocation-heavy concurrency. Very simple to implement if you drop the ordered requirement.
I second this
6. Let me specify alignment. Useful for SSE which requires 16-byte alignment for good performance, to generally ensure good locality, and for concurrency where false sharing in cache lines can be devastating. aligned_storage etc. is not enough because it doesn't work for sizes determined at run-time.
+1 too. I can contribute the alignment code for runtime alignment boudnaries.
2 Things missing (or don't work well) from Pool.
When using the pool_allocator (or fast_pool_allocator) in an application that has several DLLs on the Windows platform, the pool allocator "singleton" is created in each DLL, so when an object is allocated in one DLL and freed in another, you get a nasty crash. I worked around this by using a factory function that resides in the first loaded DLL.
3 New things that you would like (this is not a promise!).
I definitely support a "lock-free & thread-safe" version, and, if both are not achievable at the same time, some variants towards one and the other. Cheers, Filip
participants (7)
-
Cory Nelson -
Filip Konvička -
Jeremiah Willcock -
joel falcou -
Mathias Gaunard -
Paul A. Bristow -
Steven Watanabe