A need for a complex pool type container
My requirements are a bit complex and involved, but I'll try to be as brief as possible. We have this "scheduler" type system where for each unit of time (think of 1 unit as 1 ms) there is a "bucket" of events that shall be processed. This is relative. Suppose the current time is 103. We will have a record of events to be processed at time 103, 104, 105, ..., 203. When all of the events in the "bucket" for time stamp 103 are processed, we "pop_front" to remove the bucket for 103. Everything is essentially shifted forward. Hopefully this does not require physically shifting everything forward, and simply is just a matter of using a lesser index to access the same element later on. I insert new events into existing buckets anywhere in the container, it does not necessarily have to be from the back. I always remove items from the front since they are processed linearly. One can kind of think of this as a std::deque< std::vector<> >, but the containing deque would need to be given a fixed size of 100 (as in the example above). The tricky part is shifting all of the "buckets" (the buckets in this case would be the inner-vector objects) forward when something from the front is removed. The key here is that I do not want to call pop_front() each time a bucket is processed because I want to avoid constant new/delete operations since insertion and removal will be happening quite frequently. I'm not sure what the best approach to solving this problem is. I'm hoping someone in the community will be able to suggest a way in which boost can easily solve this problem. Thanks to everyone for taking the time to read this.
On Wed, Feb 18, 2009 at 17:57, Robert Dailey
I insert new events into existing buckets anywhere in the container, it does not necessarily have to be from the back. I always remove items from the front since they are processed linearly.
One can kind of think of this as a std::deque< std::vector<> >, but the containing deque would need to be given a fixed size of 100 (as in the example above). The tricky part is shifting all of the "buckets" (the buckets in this case would be the inner-vector objects) forward when something from the front is removed. The key here is that I do not want to call pop_front() each time a bucket is processed because I want to avoid constant new/delete operations since insertion and removal will be happening quite frequently.
I'm not sure what the best approach to solving this problem is. I'm hoping someone in the community will be able to suggest a way in which boost can easily solve this problem. Thanks to everyone for taking the time to read this.
Perhaps this?
boost::circular_buffer
On Wed, Feb 18, 2009 at 5:16 PM, Scott McMurray
Perhaps this?
boost::circular_buffer
schedule(100); Then you could use
schedule.rotate(schedule.begin()+1);
to move to the next bucket, which should (in steady-state) avoid reallocation of the container or the buckets, since you never actually destruct or copy the vectors.
http://www.boost.org/doc/libs/1_38_0/libs/circular_buffer/doc/circular_buffe...
Wow, this is actually perfect. So before I do a rotate(), I need to do: schedule[0].clear() right? When I do future push_back operations on schedule[99], I want it to be adding to an empty vector and not adding to the old contents of it. I appreciate your help!
On Wed, Feb 18, 2009 at 21:44, Robert Dailey
Wow, this is actually perfect. So before I do a rotate(), I need to do: schedule[0].clear() right? When I do future push_back operations on schedule[99], I want it to be adding to an empty vector and not adding to the old contents of it.
Yes, you will. (I had thought you were popping tasks off the vectors to run them.) And be sure to keep the capacity and size of the circular_buffer the same, or else the rotate will start copying and destructing vectors. Of course, even on other containers you always have the option of c.push_back(vector<T>()); swap(c.front(), c.back()); c.pop_front(); which should also avoid any allocing or freeing by the vectors.
On Wed, Feb 18, 2009 at 8:54 PM, Scott McMurray
Yes, you will. (I had thought you were popping tasks off the vectors to run them.) And be sure to keep the capacity and size of the circular_buffer the same, or else the rotate will start copying and destructing vectors.
Well after I construct my circular buffer, I immediately do: schedule.resize( 100 ); This should give it a fixed size, right? And as long as I never call any of the pop or push functions to remove/add elements, nothing should be allocated or freed by the circular buffer, correct again?
Of course, even on other containers you always have the option of
c.push_back(vector<T>()); swap(c.front(), c.back()); c.pop_front();
which should also avoid any allocing or freeing by the vectors.
I'm assuming "c" here is the circular buffer previously named "schedule"? Won't doing a push_back() and pop_front() cause an allocation and deallocation, respectively? the rotate() looked good because it did not actually add or move anything from the circular buffer itself.
On Wed, Feb 18, 2009 at 21:58, Robert Dailey
On Wed, Feb 18, 2009 at 8:54 PM, Scott McMurray
wrote: Yes, you will. (I had thought you were popping tasks off the vectors to run them.) And be sure to keep the capacity and size of the circular_buffer the same, or else the rotate will start copying and destructing vectors.
Well after I construct my circular buffer, I immediately do: schedule.resize( 100 );
This should give it a fixed size, right? And as long as I never call any of the pop or push functions to remove/add elements, nothing should be allocated or freed by the circular buffer, correct again?
Correct. I was referring to the allocation performed by the vectors, since rotate will copy-construct if the circular_buffer's size is less than its capacity.
Of course, even on other containers you always have the option of
c.push_back(vector<T>()); swap(c.front(), c.back()); c.pop_front();
which should also avoid any allocing or freeing by the vectors.
I'm assuming "c" here is the circular buffer previously named "schedule"?
Won't doing a push_back() and pop_front() cause an allocation and deallocation, respectively? the rotate() looked good because it did not actually add or move anything from the circular buffer itself.
I was using it for an arbitrary container. The point is that while the container might do allocation, the swap there prevents the bucket vectors from needing to allocate when you refill it later. As you mention, rotate avoids the issue (if size == capacity). On a circular_buffer, push_back wouldn't actually need to allocate (though it would need more capacity than the rotate version), though list, deque or most others would. With a pool allocator, though, you could plausibly avoid that issue, if you needed a different container of buckets for some reason. Clearly, though, it's a theoretical exercise as circular_buffer seems to provide exactly what you need.
On Wed, Feb 18, 2009 at 11:22 PM, Scott McMurray
On a circular_buffer, push_back wouldn't actually need to allocate (though it would need more capacity than the rotate version), though list, deque or most others would.
What exactly do you mean by this? Do you mean that push_back would not need to allocate if you use an appropriate allocator, like a pool? How would it need more capacity than 100 (still using your initial example)?
With a pool allocator, though, you could plausibly avoid that issue, if you needed a different container of buckets for some reason. Clearly, though, it's a theoretical exercise as circular_buffer seems to provide exactly what you need.
participants (2)
-
Robert Dailey
-
Scott McMurray