How to save a large number of elements in a LinkedHashMap?

Question

I have a large number of elements that I want to save in a LinkedHashMap. The problem is that I have really a large number of elements, i.e. more than the size of the Integer class. Since the size() method of the LinkedHashMap class returns an int, I think that the number of elements in the LinkedHashMap can not exceed the possibility of the int. Is that right? If yes, what can I do?

I should also mention that I actually don't know where exactly the problem is occurring in my code. That is just my guess that the problem is with the size() method of the LinkedHashMap. I just get the same error as here.

"If yes, what can I do?". Rethink your design. Why are trying to store a Map with more than 2 billion values in the first place? — Tunaki, Jan 03 '16 at 18:30
Don't forget that each entry in a `LinkedHashMap` takes 40 bytes plus the memory overhead of the actual elements, so that's a minimum of 80 gigabytes of _heap_. You probably don't want a `LinkedHashMap`. You might want a database. You might want something else entirely. — Louis Wasserman, Jan 03 '16 at 18:32
You really need to preserve the iteration order of such an amount of data? — Dragan Bozanovic, Jan 03 '16 at 19:09
Dear @Tunaki and other friends, Thank you very much for your comments. After your comments, I really realized what a catastrophe I was gonna make! So, I changed my program in a way that it works correctly. Actually, I wanted to extract some information for each word of a novel and the number of words in the novel was absolutely large. So, I changed the code in a way that I could process the text chapter by chapter and the problem was solved. Thanks again for your helps :) — user1419243, Jan 09 '16 at 10:55

RamV13 · Accepted Answer · 2016-01-03T18:49:41.187

I don't believe it's a good idea to have a Map with such a large number of values but if you really want to go ahead with it, you can override LinkedHashMap and use the BigInteger class to increase the size.

public class MyLinkedHashMap<K,V> extends LinkedHashMap<K,V> {
    BigInteger mySize;

    @Override
    public V put(K key, V Value) {
        if (super.put(key,value) == null) // this will still eventually throw an exception because of the 'size' variable incrementing internally in the HashMap
            mySize.add(BigInteger.ONE);
    }

    @Override
    public V remove(Object key) {
        if (super.remove(key) != null)
            mySize.subtract(BigInteger.ONE);
    }

    @Override
    public int size() {
        throw new UnsupportedOperationException();
    }        

    public BigInteger getSize() {
        return mySize;
    }
}

Note that because you cannot change the return type of the size() method you must create your own size method and variable to retrieve the size of the map.

Additionally, you may actually need to override HashMap itself because its put() method will still increment the existing size variable int value and will eventually cause it to go beyond the range of a Java Integer.

Lastly, just to be clear, this is NOT a good idea at all because there are many pitfalls in trying to repurpose an existing data structure in this manner (e.g. forgetting to override other methods that modify/use the size variable, programmer errors which can harm the validity of the original data structure, or other instance variables/side effects that were never originally intended to handle such large sizes etc.).

How to save a large number of elements in a LinkedHashMap?

1 Answers1