Bitset consuming more memory

Question

I have a list of Bitset having 25Million Bitset. Each Bitset I'm creating by using:

Bitset.valueOf(new long[] {1})

The memory being consumed is around 1300MB. Ie: in an average its taking 52bytes. I'm not understanding why so much of memory is being consumed.

What language is that code in? – tadman Aug 10 '17 at 16:45 — tadman, Aug 10 '17 at 16:45
This was originally tagged Memcached, so it was confusing. – tadman Aug 10 '17 at 16:48 — tadman, Aug 10 '17 at 16:48

score 2 · Answer 1 · answered Aug 10 '17 at 16:51

2

Each BitSet is itself an object instance (each reference consumes memory). You should see your memory usage go down dramatically if you do

BitSet b = new BitSet(25 * 1000 * 1000);
b.set(0, 25 * 1000 * 1000, true);

answered Aug 10 '17 at 16:51

Elliott Frisch

198,278
20
158
249

It's there any better way to store 25million bits? Basically I want to associate an object to a unique set of bits, thus I can save memory. – Shashank V C Aug 10 '17 at 16:58
1

@ShashankVC read the code snippet carefully (and check the JavaDoc on `BitSet`): `b` is a bitset of 25 million bits already. – Thomas Aug 10 '17 at 17:03

Thomas · Answer 2 · 2017-08-10T17:09:56.467

Create a heapdump and check it with some analyzer (e.g. Eclipse MAT). That provides some insight of where that memory goes.

Since BitSets are objects and they contain values as well, you'll probably see the following (I bet I forgot something but you should get what I mean):

for each BitSet you have a 8-byte reference (assuming you're using a 64-bit JVM)
each BitSet contains a reference to a long[] array with one element, that's another 20 bytes (8 for the reference, 8 for the single value in the array, 4 for the length field of the array)
each BitSet contains a boolean and an int, that's another 5 bytes.

Summing that up you get 33 bytes per BitSet (I'm sure I forgot something) and 25.000.000 * 33 bytes = 825000000 bytes (or around 786 MB).

25 million bits alone would need around 3 MB (e.g. if you'd create a BitSet that large).

As you can see, a BitSet containing only one bit is a huge waste of memory. If you can't use a single BitSet (I wouldn't see a reason for that though) you'd probably better off with a boolean array with a size of 25 million. That would still need around 95MB though.

Bitset consuming more memory

2 Answers2