HashMap gets wrong size

Question

I am trying to export some keys as byte[] from a HashSet to a HashMap and use the HashMap do store pairs of data. However, I am running into a problem, which is that the size om the set is larger than the size of the HashMap, for some reason. I would like to know what causes this as the HashMap is within a foreach which iterates from 0 to the size of the HashSet, namely 2^20. So, I am also expecting the size of the HashMap also to be 2^20.

So, what I am trying to store in the HashMap is two byte arrays. I am currently working with the meet-in-the-middle attack on 2DES. My encryption are properly implemented. In addition, my DES key generator is also properly implemented, so I am able to generate 2^20 keys (only 20 bits of the keys are effective). However, I when I try to put the keys in the HashMap the size is not the same as the HashSet, which doesn't make any sense.

for (int i = 0; i < Math.pow(2, 20); i++) {
    possibleKeySet.add(generateDesKey());
}

for (byte[] key : possibleKeySet) {
    intermediateCipher.put((encrypt(key, plainText)).toString(), key);
}

Output: Set size: 1048576 Map size: 1048295

PS: intermediateCipher is my HashMap.

Update: I have tried to implement hashcode and equals, but i'm not sure how to implement hashcode.

class ByteArray {

    private byte[] key;

    ByteArray(byte[] key) {
        this.key = key;
    }

    byte[] getKey() {
        return key;
    }

    public boolean equals(Object obj) {
      ByteArrayOutputStream bos = new ByteArrayOutputStream();
      ObjectOutputStream oos = new ObjectOutputStream(obj);
      oos.writeObject(obj);
      oos.flush();
      byte [] data = bos.toByteArray();
      return key.equals(data);
}

    public int hashCode() {
        // what should I write here?
    }

}

Answer 1

The probability of a hash collision is (as can be seen here):

The average number of collisions for a set of size n given k inputs is:

Given an n of 2^32 and a k of 2^20, the average collision is

(2^20) * (2^20 - 1) / (2 * 2^32)
~= 2^40/2^33
~= 2^7
~= 128

The number of collisions you saw was 1048576 - 1048295 = 281. Given this info I assume that the actual entropy of the value returned by encrypt(key, plainText) is roughly ~31 bits (rather than 32).

---

To get the desired amount of keys, you may want to just keep generating values until you reach the desired size. This may make the method take a long time to complete:

while (intermediateCipher.size() < Math.pow(2,20)) {
    byte[] key = generateDesKey();
    intermediateCipher.put((encrypt(key, plainText)).toString(), key);
}

---

Answer 2

If your Set and HashMap have a different size it will probably be a collision on keys. It looks like your function encrypt(key, plainText) return duplicates.

Try with:

for (byte[] key : possibleKeySet) {
    Object oldValue = intermediateCipher.put((encrypt(key, plainText)).toString(), key);
    if(oldValue != null) {
        System.out.println("Duplicated!");
    }
}

It will probably produce some errors.

Take a note also, that generateDesKey() can also produce the same values even if they are stored in Set as two arrays in Java are only equal if they are the same object.

What you can do? Create a custom object:

class ByteArray {

    private byte[] key;

    ByteArray(byte[] key) {
        this.key = key;
    }

    byte[] getKey() {
        return key;
    }

    public boolean equals(Object obj) {
        //implement your equals logic using array members equality
    }

    public int hashCode() {
        //implement your hashCode logic using array members equality
    }

}

Answer 3

The set itself is being implemented that way that every item inside must be unique - it does not allow duplicates. That means that when you are putting to set two equal objects in result in set there will be only one.

Probably your generateDesKey() method is returning not unique values for all of 2^20 values and in the result set the items count is less than 2^20

Basically you can check the possibleKeySet size before copying values to the HashMap like

System.out.println(possibleKeySet.size());

or just by using debugger