[Boost-users] Help in calculating the size of a boost::unordered_map

[Ram]

Hi,

I would like to calculate the size of my filled up boost::unordered_map
including the house keeping. I was thinking of sizeof(boost:unordered_map
mymap). I asked this question on stackoverflow but they say that wont help.
Can somebody tell me an easy way to find the size including the
housekeeping that it takes?

The boost::unordered_map stores a std::string as the key and a pointer to
my class object as the value.

Thanks,
Ram

[Ram]

Hi,

Can anybody please help with this?

Thanks,
-R

Post by Ram
Hi,
I would like to calculate the size of my filled up boost::unordered_map
including the house keeping. I was thinking of sizeof(boost:unordered_map
mymap). I asked this question on stackoverflow but they say that wont help.
Can somebody tell me an easy way to find the size including the
housekeeping that it takes?
The boost::unordered_map stores a std::string as the key and a pointer to
my class object as the value.
Thanks,
Ram

[Juan Ramírez]

What you are trying to obtain is not easy, I must say.

The static size might be solved by using sizeof, but the size of the
allocated data will be more complicated (and less performant) to compute.

Suppose you have an unordered_map of string to integers.

boost::unordere_map<string, int> my_map_;

The size int is known as sizeof(int), so it seems logical to multiply this
value by the size of the container in order to get the actual size of the
containers IN TERMS OF THE VALUE.

size_t size_of_val = sizeof(int) * my_map_.size();

Now, what if the container does not contain integers, what if it contains a
class? And what if that class uses dynamic allocation (std::string for
example). You cannot know the size of each element in the container unless
you visit them all!!

The reason above applies for the key elements, the key must be inside the
container, but keys have dynamic data so they all have potentially
different sizes!

So, my advice would be to:

1- Compute the static size of the unordered_map using size_of
2- Sum up the size of each element (key + value) in the map

Remarks: I don't know the internals of boost::unordered_map, if the class
allocates data in the heap you are out of luck!

Best regards
Juan

Post by Ram
Hi,
Can anybody please help with this?
Thanks,
-R

Post by Ram
Hi,
I would like to calculate the size of my filled up boost::unordered_map
including the house keeping. I was thinking of sizeof(boost:unordered_map
mymap). I asked this question on stackoverflow but they say that wont help.
Can somebody tell me an easy way to find the size including the
housekeeping that it takes?
The boost::unordered_map stores a std::string as the key and a pointer to
my class object as the value.
Thanks,
Ram

_______________________________________________
Boost-users mailing list
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Ram]

Thanks Juan! What I am interested in here is mainly to get the size of the
housekeeping. The key(string) and the value(pointers to class objects) are
of reasonable size and that is fine. I want to see how much overhead the
housekeeping causes when my unordered_map is huge. I am trying to decide
between a std::map and boost::unordered_map.

Thanks,
-R

[Chris Glover]

Post by Ram
Thanks Juan! What I am interested in here is mainly to get the size of the
housekeeping. The key(string) and the value(pointers to class objects) are
of reasonable size and that is fine. I want to see how much overhead the
housekeeping causes when my unordered_map is huge. I am trying to decide
between a std::map and boost::unordered_map.

A strategy that might give you what you want is to overload global new and
delete and then write some code to log the allocations. From there you can
calculate the total amount of memory consumed and subtract out the expected
amount of memory from the logged data. Something like the code below is
close, but keep in mind that this is an example and there are many
overloads for new and delete (nothrow versions, etc) so you will need to
make sure that these are the versions being called. This test also doesn't
account for allocator overhead and will probably only work on Windows
because it uses the platform specific _msize function. On Linux,
malloc_usable_size might do the trick instead of _msize.

#include <map>
#include <iostream>
#include <malloc.h>

std::size_t g_TotalMem = 0;
void* operator new(std::size_t size)
{
g_TotalMem += size;
return malloc(size);
}

void* operator new[](std::size_t size)
{
g_TotalMem += size;
return malloc(size);
}

void operator delete(void* p)
{
g_TotalMem -= _msize(p);
free(p);
}

void operator delete[](void* p)
{
g_TotalMem -= _msize(p);
free(p);
}

int main()
{
std::map<int, int> map;
std::size_t start_mem = g_TotalMem;
for(int i = 0; i < 10; ++i)
{
map.insert({i, i});
}

std::cout << g_TotalMem - start_mem << std::endl;
return 0;
}

Good luck!

-- chris

[Jon Kalb]

Ram,



boost::unordered_map takes an allocator parameter. I think the only way to get the answer to this question is to an instrumented allocator to the container that you can query to determine how much memory was requested.



Jon



From: Boost-users <boost-users-***@lists.boost.org> on behalf of Ram <***@gmail.com>
Reply-To: Boost Users <boost-***@lists.boost.org>
Date: Monday, November 7, 2016 at 3:58 AM
To: Boost Users <boost-***@lists.boost.org>
Subject: Re: [Boost-users] Help in calculating the size of a boost::unordered_map



Hi,



Can anybody please help with this?



Thanks,

-R



On Mon, Nov 7, 2016 at 11:00 AM, Ram <***@gmail.com> wrote:

Hi,



I would like to calculate the size of my filled up boost::unordered_map including the house keeping. I was thinking of sizeof(boost:unordered_map mymap). I asked this question on stackoverflow but they say that wont help. Can somebody tell me an easy way to find the size including the housekeeping that it takes?



The boost::unordered_map stores a std::string as the key and a pointer to my class object as the value.



Thanks,

Ram





_______________________________________________ Boost-users mailing list Boost-***@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Ian Bruntlett]

Hi Ram,

Post by Ram
Hi,
I would like to calculate the size of my filled up boost::unordered_map
including the house keeping. I was thinking of sizeof(boost:unordered_map
mymap). I asked this question on stackoverflow but they say that wont help.
Can somebody tell me an easy way to find the size including the
housekeeping that it takes?
The boost::unordered_map stores a std::string as the key and a pointer to
my class object as the value.

Not sure which platform you are doing this on but if you are using some
kind of Linux then this site http://valgrind.org/ might be useful

BW,


Ian

[Ram]

Thanks for all your replies Chris, Jon and Ian!

I will implement a custom allocator and pass it to the unordered_map. But
for starters it looks like the Valgrind solution seems easiest. I am
working on Windows using Visual C++, compiler VC12. I will look for
Valgrind alternatives for Windows.

Thanks,
-R

Post by Ian Bruntlett
Hi Ram,

Post by Ram
Hi,
I would like to calculate the size of my filled up boost::unordered_map
including the house keeping. I was thinking of sizeof(boost:unordered_map
mymap). I asked this question on stackoverflow but they say that wont help.
Can somebody tell me an easy way to find the size including the
housekeeping that it takes?
The boost::unordered_map stores a std::string as the key and a pointer to
my class object as the value.

Not sure which platform you are doing this on but if you are using some
kind of Linux then this site http://valgrind.org/ might be useful
BW,
Ian
--
-- ACCU - Professionalism in programming - http://www.accu.org
-- My writing - https://sites.google.com/site/ianbruntlett/
-- Free Software page - https://sites.google.com/site/
ianbruntlett/home/free-software
_______________________________________________
Boost-users mailing list
http://lists.boost.org/mailman/listinfo.cgi/boost-users