Help in calculating the size of a boost::unordered_map

Hi, I would like to calculate the size of my filled up boost::unordered_map including the house keeping. I was thinking of sizeof(boost:unordered_map mymap). I asked this question on stackoverflow but they say that wont help. Can somebody tell me an easy way to find the size including the housekeeping that it takes? The boost::unordered_map stores a std::string as the key and a pointer to my class object as the value. Thanks, Ram

Hi, Can anybody please help with this? Thanks, -R On Mon, Nov 7, 2016 at 11:00 AM, Ram <sourceopen@gmail.com> wrote:
Hi,
I would like to calculate the size of my filled up boost::unordered_map including the house keeping. I was thinking of sizeof(boost:unordered_map mymap). I asked this question on stackoverflow but they say that wont help. Can somebody tell me an easy way to find the size including the housekeeping that it takes?
The boost::unordered_map stores a std::string as the key and a pointer to my class object as the value.
Thanks, Ram

What you are trying to obtain is not easy, I must say. The static size might be solved by using sizeof, but the size of the allocated data will be more complicated (and less performant) to compute. Suppose you have an unordered_map of string to integers. boost::unordere_map<string, int> my_map_; The size int is known as sizeof(int), so it seems logical to multiply this value by the size of the container in order to get the actual size of the containers IN TERMS OF THE VALUE. size_t size_of_val = sizeof(int) * my_map_.size(); Now, what if the container does not contain integers, what if it contains a class? And what if that class uses dynamic allocation (std::string for example). You cannot know the size of each element in the container unless you visit them all!! The reason above applies for the key elements, the key must be inside the container, but keys have dynamic data so they all have potentially different sizes! So, my advice would be to: 1- Compute the static size of the unordered_map using size_of 2- Sum up the size of each element (key + value) in the map Remarks: I don't know the internals of boost::unordered_map, if the class allocates data in the heap you are out of luck! Best regards Juan On Mon, Nov 7, 2016 at 8:58 AM, Ram <sourceopen@gmail.com> wrote:
Hi,
Can anybody please help with this?
Thanks, -R
On Mon, Nov 7, 2016 at 11:00 AM, Ram <sourceopen@gmail.com> wrote:
Hi,
I would like to calculate the size of my filled up boost::unordered_map including the house keeping. I was thinking of sizeof(boost:unordered_map mymap). I asked this question on stackoverflow but they say that wont help. Can somebody tell me an easy way to find the size including the housekeeping that it takes?
The boost::unordered_map stores a std::string as the key and a pointer to my class object as the value.
Thanks, Ram
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Juan :wq

Thanks Juan! What I am interested in here is mainly to get the size of the housekeeping. The key(string) and the value(pointers to class objects) are of reasonable size and that is fine. I want to see how much overhead the housekeeping causes when my unordered_map is huge. I am trying to decide between a std::map and boost::unordered_map. Thanks, -R

On Mon, 7 Nov 2016 at 09:32 Ram <sourceopen@gmail.com> wrote:
Thanks Juan! What I am interested in here is mainly to get the size of the housekeeping. The key(string) and the value(pointers to class objects) are of reasonable size and that is fine. I want to see how much overhead the housekeeping causes when my unordered_map is huge. I am trying to decide between a std::map and boost::unordered_map.
A strategy that might give you what you want is to overload global new and delete and then write some code to log the allocations. From there you can calculate the total amount of memory consumed and subtract out the expected amount of memory from the logged data. Something like the code below is close, but keep in mind that this is an example and there are many overloads for new and delete (nothrow versions, etc) so you will need to make sure that these are the versions being called. This test also doesn't account for allocator overhead and will probably only work on Windows because it uses the platform specific _msize function. On Linux, malloc_usable_size might do the trick instead of _msize. #include <map> #include <iostream> #include <malloc.h> std::size_t g_TotalMem = 0; void* operator new(std::size_t size) { g_TotalMem += size; return malloc(size); } void* operator new[](std::size_t size) { g_TotalMem += size; return malloc(size); } void operator delete(void* p) { g_TotalMem -= _msize(p); free(p); } void operator delete[](void* p) { g_TotalMem -= _msize(p); free(p); } int main() { std::map<int, int> map; std::size_t start_mem = g_TotalMem; for(int i = 0; i < 10; ++i) { map.insert({i, i}); } std::cout << g_TotalMem - start_mem << std::endl; return 0; } Good luck! -- chris

Ram, boost::unordered_map takes an allocator parameter. I think the only way to get the answer to this question is to an instrumented allocator to the container that you can query to determine how much memory was requested. Jon From: Boost-users <boost-users-bounces@lists.boost.org> on behalf of Ram <sourceopen@gmail.com> Reply-To: Boost Users <boost-users@lists.boost.org> Date: Monday, November 7, 2016 at 3:58 AM To: Boost Users <boost-users@lists.boost.org> Subject: Re: [Boost-users] Help in calculating the size of a boost::unordered_map Hi, Can anybody please help with this? Thanks, -R On Mon, Nov 7, 2016 at 11:00 AM, Ram <sourceopen@gmail.com> wrote: Hi, I would like to calculate the size of my filled up boost::unordered_map including the house keeping. I was thinking of sizeof(boost:unordered_map mymap). I asked this question on stackoverflow but they say that wont help. Can somebody tell me an easy way to find the size including the housekeeping that it takes? The boost::unordered_map stores a std::string as the key and a pointer to my class object as the value. Thanks, Ram _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Hi Ram, On 7 November 2016 at 05:30, Ram <sourceopen@gmail.com> wrote:
Hi,
I would like to calculate the size of my filled up boost::unordered_map including the house keeping. I was thinking of sizeof(boost:unordered_map mymap). I asked this question on stackoverflow but they say that wont help. Can somebody tell me an easy way to find the size including the housekeeping that it takes?
The boost::unordered_map stores a std::string as the key and a pointer to my class object as the value.
Not sure which platform you are doing this on but if you are using some kind of Linux then this site http://valgrind.org/ might be useful BW, Ian -- -- ACCU - Professionalism in programming - http://www.accu.org -- My writing - https://sites.google.com/site/ianbruntlett/ -- Free Software page - https://sites.google.com/site/ianbruntlett/home/free-software

Thanks for all your replies Chris, Jon and Ian! I will implement a custom allocator and pass it to the unordered_map. But for starters it looks like the Valgrind solution seems easiest. I am working on Windows using Visual C++, compiler VC12. I will look for Valgrind alternatives for Windows. Thanks, -R On Nov 8, 2016 3:13 AM, "Ian Bruntlett" <ian.bruntlett@gmail.com> wrote:
Hi Ram,
On 7 November 2016 at 05:30, Ram <sourceopen@gmail.com> wrote:
Hi,
I would like to calculate the size of my filled up boost::unordered_map including the house keeping. I was thinking of sizeof(boost:unordered_map mymap). I asked this question on stackoverflow but they say that wont help. Can somebody tell me an easy way to find the size including the housekeeping that it takes?
The boost::unordered_map stores a std::string as the key and a pointer to my class object as the value.
Not sure which platform you are doing this on but if you are using some kind of Linux then this site http://valgrind.org/ might be useful
BW,
Ian
-- -- ACCU - Professionalism in programming - http://www.accu.org -- My writing - https://sites.google.com/site/ianbruntlett/ -- Free Software page - https://sites.google.com/site/ ianbruntlett/home/free-software
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (5)
-
Chris Glover
-
Ian Bruntlett
-
Jon Kalb
-
Juan Ramírez
-
Ram