0

I'm in the process of integrating a hash method (farmhash) to our software base. The hashing services seem to work appropriately. Basically, it turns a string of characters into an unique-ish integer value.

I've added an infrastructure to detect collisions (in a case where two input strings would result in the same output integer). Basically, for each string that is hashed, I keep the [hash result] -> [string] in a map, and every time a new string is hashed, I compare it to what's in the map; if the hash is already there, I make sure that it is the same string that has generated it. I am aware that it's potentially slow and it's potentially memory consuming, but I'm performing theses checks only on a "per request" basis: they are not enabled in release mode.

Now I'd like to test that infrastructure (as in get a collision, from a unit test point of view).

I could generate a bunch of strings (random or sequential), spam my hash infrastructure and hope to see a positive collision but I feel I'll waste my time, CPU cycles and fill the memory with a load of data without success.

How would one go about generating collisions?

Not-so-relevant-facts:

  • I'm using c++;
  • I can generate data using python;
  • The target int is uint32_t.

Update:

I have created a small naive program to brute force the detection of collision:

void
addToQueue(std::string&& aString)
{
  //std::cout << aString << std::endl;
  hashAndCheck( aString ); // Performs the hash and check if there is a collision
  if ( mCount % 1000000 )
    std::cout << "Did " << mCount << " checks so far" << std::endl;
  mQueue.emplace( aString );
}


void 
generateNextRound( const std::string& aBase )
{
  //48 a 122 incl
  for ( int i = 48; i <= 122; i++ )
  {
    addToQueue( std::move( std::string( aBase ).append( 1, static_cast<char>( i ) ) ) );
  }
}


int main( void )
{

  // These two generate a collision
  //StringId id2 = HASH_SID( "@EF" ); // Hashes only, does not check
  //StringId id1 = HASH_SID( "7\\:" ); // Hashes only, does not check

  std::string base = "";
  addToQueue( std::move( base ) );

  while ( true )
  {
    const std::string val = mQueue.front();
    mQueue.pop();
    generateNextRound( val );
  }

  return 0;
}

I could eventually have added threading and stuff in there but I didn't need it because I found a collision in about 1 second (in debug mode).

Vaillancourt
  • 1,380
  • 1
  • 11
  • 42

3 Answers3

1

You could limit the range of the integer that is outputted by the hash function; in general you should be able to pass some number into it (n) so that results will lie between 0 & n-1. If you limit it to 10 say, Then you'll definitely end up with collisions.

Richard Boyes
  • 49
  • 4
1

If you brute force search for collisions offline, you could hard code strings that cause collisions into your test so that your test is as close to production code as possible, but doesn't suffer the performance penalty of doing the brute force work each time (or, like other people have said, you can make an intentionally junky hash algorithm that causes excessive collisions)

Alan Wolfe
  • 615
  • 4
  • 16
  • I've been thinking about this method. I'd have to find a way to find a collision by brute force first. – Vaillancourt Apr 11 '15 at 17:56
  • 1
    Well since it'd be an "offline" process you don't need to worry about making it efficient. If your hash is 32 bits that means you could have an array of strings that was 2^32 items large, and you could systematically hash strings and store them in the resulting bucket until you find a bucket that already has a string in it (length greater than zero). You will probably have to dynamically allocate the array so it doesn't bust the stack. To generate the strings you just try all the single letter strings, then all the 2 letter strings, then 3. Repeat til you find a collision. – Alan Wolfe Apr 11 '15 at 18:12
0

For key k and hash function h, return a constant c:

h(k) = c

This always collides, regardless of what key you use.

Alex Reynolds
  • 95,983
  • 54
  • 240
  • 345