Floating point equality is messy. Just trying to define what it actually means is messy.
First lets consider what happens when you round the numbers.
float x = 0.4999999;
float y = 0.5000000;
float z = 1.4999999;
Assert.Equals(false, Math.Round(x) == Math.Round(y));
Assert.Equals(true, Math.Round(y) == Math.Round(z));
If you're trying to model a real world process, I would expect that x and y would be a lot more equal than y and z. But rounding forces y and z into the same bucket, and x into a different one.
No matter what scale you choose your rounding, there will always be numbers which are arbitrarily close together which are considered different, and numbers which are on opposite ends of your scale which are considered the same. If your numbers are generated by some arbitrary process, you never know if two numbers which should be considered equal will fall on the same side of or opposite sides of a boundary. If you choose to round to the nearest 0.01, the exact same example works if you just multiply x, y, and z in the example by 0.01.
Let's say you consider equality by the distance between two numbers.
float x = 4.6;
float y = 5.0;
float z = 5.4;
Assert.Equals(true, Math.Abs(x - y) < 0.5);
Assert.Equals(true, Math.Abs(y - z) < 0.5);
Assert.Equals(false, Math.Abs(x - z) < 0.5);
Now numbers which are close together are always considered equal, but you've given up the transitive property of equality. That means that x and y are considered equal, and y and z are considered equal, but x and z are considered not equal. Obviously you can't build a hashset without transitive equality.
The next thing to consider is that if you're doing calculations, floating point numbers can have different precision depending on how they're stored. It's up to the compiler to decide where they will be stored, and it can convert them back and forth whenever it wants to. Calculations will be done in registers, and it can make a difference when those registers get copied to main memory, and when they lose that precision. This is harder to demonstrate in code, because it's really up to how it compiles, so let's use a hypothetical example to illustrate.
float x = 4.49;
float y = Math.Round(x, 1); // equals 4.5
float z1 = Math.Round(x); // 4.49 rounds to 4
float z2 = Math.Round(y); // 4.5 rounds to 5
Assert.Equals(false, z1 == z2);
Depending on whether the intermediate result got rounded or not, I get a different result on the final rounding. Obviously going registers -> memory isn't rounding to 1 decimal digit, but this illustrates the principal that when you choose to round can impact your result. If you pass 2 numbers to the equality function that are supposed to be the same, and one came from memory, and the other from a register, you could potentially get something that rounds 2 different ways.
EDIT: Another part to consider that may not make a difference this case, is that a float only has 24 bits of mantissa. That means that once you get past 2 to the 24th power, or 16,777,216, numbers that you would expect to be different will come back as equal, no matter what precision you thought you were rounding them to.
float x = 17000000;
float y = 17000001;
Assert.Equals(true, x == y);
So if you're fine with all those caveats because all you want is something that works most of the time, you can probably get away with trying to hash on floating point numbers. But no matter how you try to define floating point equality, you'll always end up with unexpected behavior.