
Hi, Looks like that even if I call reserve on a multi_index, it stills allocate memory while I am still below the reserve size. Is this the intended behavior ? If so, what is the best way to avoid allocation in a fixed sized multi_index Thanks ! -- Olivier Tristan Research & Development www.uvi.net

El 19/03/2019 a las 18:14, Olivier Tristan via Boost-users escribió:
Hi,
Hi Olivier,
Looks like that even if I call reserve on a multi_index, it stills allocate memory while I am still below the reserve size.
Is this the intended behavior ?
Yes it is. multi_index_container is node based, and, like std node-based containers (std::list, std::set, std::unordered_set, etc.), will allocate memory for its nodes as needed. Hashed and random-access indices provide reserve() member functions (with non-equivalent semantics) that control the size of internal contiguous memory chunks maintained by these indices additionally to the nodes. So, reserve() (when available) does not (nor is designed to) guarantee that no additional memory will be allocated on further insertions.
If so, what is the best way to avoid allocation in a fixed sized multi_index
You can write your own allocator that allocates the memory beforehand and then serves it piece by piece on allocate(). I suspect, however, that this is not going to be faster than using the default std::allocator.
Thanks !
Best, Joaquín M López Muñoz

Hi Joaquin, My goal is not to be fast but to avoid allocation as I am in a high priority thread and I want to avoid possible allocation lock. I tried the allocator solution following a thread on stackoverflow (where you answered as well) https://stackoverflow.com/questions/37095641/how-to-use-boostobject-pool-as-... using something like that boost::pool_allocator<ActiveNote, boost::default_user_allocator_new_delete, boost::details::pool::default_mutex, 256U, 0U>>; My issue with this solution is that the pool is actually shared across all object using a boost::pool_allocator that requires a mutex as there are multiple thread in my app. One solution could be to tag the allocator but this seems to be not possible anymore unless I am mistaken https://stackoverflow.com/questions/26188586/tag-template-parameter-in-boost... Is there a way otherwise to get a pool for each instances of my multi_index ? Thanks ! Le 19/03/2019 à 21:49, Joaquin M López Muñoz via Boost-users a écrit :
El 19/03/2019 a las 18:14, Olivier Tristan via Boost-users escribió:
Hi,
Hi Olivier,
Looks like that even if I call reserve on a multi_index, it stills allocate memory while I am still below the reserve size.
Is this the intended behavior ?
Yes it is. multi_index_container is node based, and, like std node-based containers (std::list, std::set, std::unordered_set, etc.), will allocate memory for its nodes as needed.
Hashed and random-access indices provide reserve() member functions (with non-equivalent semantics) that control the size of internal contiguous memory chunks maintained by these indices additionally to the nodes. So, reserve() (when available) does not (nor is designed to) guarantee that no additional memory will be allocated on further insertions.
If so, what is the best way to avoid allocation in a fixed sized multi_index
You can write your own allocator that allocates the memory beforehand and then serves it piece by piece on allocate(). I suspect, however, that this is not going to be faster than using the default std::allocator.
Thanks !
Best,
Joaquín M López Muñoz
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Olivier Tristan Research & Development www.uvi.net

El 20/03/2019 a las 9:15, Olivier Tristan via Boost-users escribió:
Hi Joaquin,
Hi Olivier, Please don't top-post: https://www.boost.org/community/policy.html#quoting
My goal is not to be fast but to avoid allocation as I am in a high priority thread and I want to avoid possible allocation lock.
I tried the allocator [...] using something like that
boost::pool_allocator< ActiveNote, boost::default_user_allocator_new_delete, boost::details::pool::default_mutex, 256U, 0U>>;
My issue with this solution is that the pool is actually shared across all object using a boost::pool_allocator that requires a mutex as there are multiple thread in my app.
Maybe you can give this allocator a try: https://probablydance.com/2014/11/09/plalloc-a-simple-stateful-allocator-for... It works with Boost.MultiIndex and does not use any mutex at all (because it's stateful and as such instances are owned by the containers using them). Joaquín M López Muñoz

Le 20/03/2019 à 11:18, Joaquin M López Muñoz via Boost-users a écrit :
Maybe you can give this allocator a try:
https://probablydance.com/2014/11/09/plalloc-a-simple-stateful-allocator-for...
It works with Boost.MultiIndex and does not use any mutex at all (because it's stateful and as such instances are owned by the containers using them).
I tried using directly boost::pool but I was struggling having an allocator that actually compiled (last issue was related to swap) This is a great example to start from. In the meantime, I've found as well info regarding boost::container::node_allocator but the memory is shared across instances and do not allow preallocation. Didn't found any other out of the box solution in boost though so I've ended up modifying the example. Still I need to do a reserve on multi_index to force the creation of all the allocator Thanks a lot ! For those interested, here is what I end up with template <typename T> struct PoolAlloc { typedef T value_type; PoolAlloc(size_t reserveSize) : reservedSize(reserveSize) { reserve(reserveSize); } template <typename U> PoolAlloc(const PoolAlloc<U>& other) : reservedSize(other.reservedSize) { reserve(reservedSize); } PoolAlloc(const PoolAlloc&) = delete; PoolAlloc& operator=(const PoolAlloc&) = delete; PoolAlloc(PoolAlloc&&) = default; PoolAlloc& operator=(PoolAlloc&&) = default; typedef std::true_type propagate_on_container_copy_assignment; typedef std::true_type propagate_on_container_move_assignment; typedef std::true_type propagate_on_container_swap; void reserve(size_t to_allocate) { available.reserve(to_allocate); std::unique_ptr<value_holder[]> allocated(new value_holder[to_allocate]); value_holder* first_new = allocated.get(); memory.emplace_back(std::move(allocated)); for (size_t i = 0; i < to_allocate; ++i) { available.push_back(std::addressof(first_new[i].value)); } } bool operator==(const PoolAlloc& other) const { return this == &other; } bool operator!=(const PoolAlloc& other) const { return !(*this == other); } T* allocate(size_t num_to_allocate) { if (num_to_allocate != 1) { return static_cast<T*>(::operator new(sizeof(T) * num_to_allocate)); } else { if (available.empty()) { // first allocate 8, then double whenever // we run out of memory size_t to_allocate = 8 << memory.size(); reserve(to_allocate); } T* result = available.back(); available.pop_back(); return result; } } void deallocate(T* ptr, size_t num_to_free) { if (num_to_free == 1) { available.push_back(ptr); } else { ::operator delete(ptr); } } // boilerplate that shouldn't be needed, except // libstdc++ doesn't use allocator_traits yet template <typename U> struct rebind { typedef PoolAlloc<U> other; }; typedef T* pointer; typedef const T* const_pointer; typedef T& reference; typedef const T& const_reference; template <typename U, typename... Args> void construct(U* object, Args&&... args) { new (object) U(std::forward<Args>(args)...); } template <typename U, typename... Args> void construct(const U* object, Args&&... args) = delete; template <typename U> void destroy(U* object) { object->~U(); } union value_holder { value_holder() {} ~value_holder() {} T value; }; size_t reservedSize; std::vector<std::unique_ptr<value_holder[]>> memory; std::vector<T*> available; }; -- Olivier Tristan Research & Development www.uvi.net
participants (2)
-
Joaquin M López Muñoz
-
Olivier Tristan