Casting to Int returns different value than FloorToInt for positive number in c#

Question

[Edit] I rewrote the whole example as told by the comments.

    float value = 0.52631580f;

    float sum = 0;
    for (int i = 0; i < 10; i++)
    {
        float division = sum / value;
        int result = (int)division;

        Console.WriteLine(division.ToString("0.00000") + " - " + result.ToString("0.00000"));
        sum += value;
    }

The results are not the ones expected:

0,00000 - 0,00000
1,00000 - 1,00000
2,00000 - 2,00000
3,00000 - 3,00000
4,00000 - 4,00000
5,00000 - 5,00000
6,00000 - 5,00000 <---- It should be 6
7,00000 - 7,00000

If I do it manually, (int)6.00000000000f <-- Returns 6, of course.

How is that possible?

Do we have to guess what `sampleSize.x` is, or what the value being cast, truncated or rounded is? Casting doesn't mean truncating. — Panagiotis Kanavos, Dec 03 '20 at 19:21
if `posX` is a `float` why are you casting it to `float` in the first calculation? — juharr, Dec 03 '20 at 19:21
Casting a float to int doesn't truncate the fractional parts. It *rounds* the number with half-way numbers [rounding to the closest even number](https://learn.microsoft.com/en-us/dotnet/api/system.convert.toint32?view=net-5.0#System_Convert_ToInt32_System_Single_). This actually *reduces the statistical error* as each time, the error is +0.5 or -0.5 which over time averages to 0. If truncation was used, the error would be +0.5 always. This is actually the [*default rounding mode* in IEEE 754](https://en.wikipedia.org/wiki/Rounding#Round_half_to_even) — Panagiotis Kanavos, Dec 03 '20 at 19:30
So to `How is that possible?` the answer is `that's the standard`. You can specify different rounding rules if you want, eg round to even, away from zero, towards zero. In signal and image processing applications though, rounding/truncating is essentially noise. Round-to-even eliminates that noise. Floor/ceiling maximize it — Panagiotis Kanavos, Dec 03 '20 at 19:31
@PanagiotisKanavos I'm sorry. I just modified the text so that it's clear. I didn't know any of those details you give me . Maybe that should be the answer to this question if you want to add it. I'll study it a little bit. In case you add it as the answer, then I'll mark it correct! — Darkgaze, Dec 04 '20 at 08:24
@juharr Sorry, I forgot to remove that. I was testing. I cleared it. — Darkgaze, Dec 04 '20 at 08:24
@PanagiotisKanavos: No, casting float to int *does* truncate towards 0. See https://gist.github.com/jskeet/c1e33e772cab1f57aad421925887d00f for example. — Jon Skeet, Dec 04 '20 at 08:27
@PanagiotisKanavos By the way, I was reading the article and... most of the methods affect how to round the negative numbers. My floatValue1 is always positive. That's why I moved my FloorToInt to an (int) which somebody that uses that a lot told me that it worked the same as the floorToInt for positive numbers... The error... that's another thing that could be affecting, though. — Darkgaze, Dec 04 '20 at 08:27
It would be a lot easier to help you if you could provide a [mcve]. Yes, we could build it ourselves from what you've shown us, but it's a waste of effort for multiple people to do that when *you* can give us something we can copy/paste/compile/run. — Jon Skeet, Dec 04 '20 at 08:28
@JonSkeet Oh yes, I know but I thought this example was pretty basic. I was just wondering if somebody knew something about casting to int that I didn't know. I'll try to do that. Thanks — Darkgaze, Dec 04 '20 at 08:30
Actually we *can't* reproduce it at the moment as you haven't told us what `floatValue2` is. Please don't make us guess. — Jon Skeet, Dec 04 '20 at 08:30
@JonSkeet oops. I followed the documentation before answering [Single.ToInt32](https://learn.microsoft.com/en-us/dotnet/api/system.single.system-iconvertible-toint32?view=net-5.0) to [Convert.ToInt32](https://learn.microsoft.com/en-us/dotnet/api/system.convert.toint32?view=net-5.0#System_Convert_ToInt32_System_Single_) which specifies round-to-even, even using the question's numbers. But `(int)3.5` returns 3, not 4. — Panagiotis Kanavos, Dec 04 '20 at 08:40
@JonSkeet Ok. I created a mimnimal example. I could reproduce the same in a small piece of code. Hope it helps! — Darkgaze, Dec 04 '20 at 08:52
No, that isn't a complete example. We want something to copy, paste, compile and run. We shouldn't have to mess around adding a Main method and calling the method multiple times. I'd expect you to already *have* such a console app - so please provide it. You're asking for help - so make it as *easy* as possible for us to help you. — Jon Skeet, Dec 04 '20 at 09:02
Print the raw value of `division` in each iteration, using `Console.WriteLine(division.ToString("G16"))`. You should find the results enlightening as to what happens with floating-point precision on repeated operations. [See also](https://stackoverflow.com/q/588004/4137916). — Jeroen Mostert, Dec 04 '20 at 09:04
@JeroenMostert bet you thought about the .NET Core 3 change too? In this case the difference appears at 6 digits though. — Panagiotis Kanavos, Dec 04 '20 at 09:26
@JonSkeet Sorry about that. I don't know how to write a console app. I work with Unity, which is a game development engine that uses a different syntax for the rest of the code. But I get the idea from the different answsers I got. Thanks — Darkgaze, Dec 07 '20 at 08:28
@darkgaze: No, Unity doesn't use a different syntax. Sure, it's generally a different environment, but the C# syntax is the same. I would strongly recommend that you learn how to create console apps - it won't take long, and you'll save a *lot* of time in the long run. — Jon Skeet, Dec 07 '20 at 08:52
@JonSkeet Ok! Thanks for the suggestion. I'll try next time. — Darkgaze, Dec 08 '20 at 12:26

Panagiotis Kanavos · Accepted Answer · 2020-12-04T10:03:52.223

The code is right. And standard compliant. I changed the WriteLine line to this :

Console.WriteLine($"{sum:0.000} {value:0.000} {division:N18} - {result:0.00000}");

And got

2.105 0.526 4.000000000000000000 - 4.00000
2.632 0.526 5.000000000000000000 - 5.00000
3.158 0.526 5.999999523162841797 - 5.00000 
3.684 0.526 6.999999523162841797 - 6.00000 
4.211 0.526 7.999999046325683594 - 7.00000 
4.737 0.526 8.999999046325683594 - 8.00000

Rounding errors mean the division result isn't exactly 6, 7 or 8. With just 5 digits, I get:

3.158 0.526 6.00000 - 5.00000 
3.684 0.526 7.00000 - 6.00000 
4.211 0.526 8.00000 - 7.00000 
4.737 0.526 9.00000 - 8.00000

This isn't about how C# truncates or rounds, it's about how floating point numbers are formatted. The IEEE compliant way is to round to the desired precision when formatting.

Phew - for a moment, I actually forgot what I knew about floats.

PS: there's another funny change at 15 digits that was introduced in .NET Core 3. Before that, string formatting stopped producing digits at 15 digits which lead to quite a bit of confusion with values that were close but not quite equal to an integer.

Since .NET Core 3, the string formatter emits enough digits to produce the shortest roundtrippable string. Using

Console.WriteLine($"{division} - {result}");

Produces :

4 - 4
 5 - 5 
5.9999995 - 5 
6.9999995 - 6 
7.999999 - 7 
8.999999 - 8

Which is why they teach us in computing math to never compare floats for equality, but only check whether the absolute difference is below a threshold.

Something I always forget and have to rediscover after crashes or infinite loops.

PPS: Rounding errors

The fractional digits are produced because float doesn't have enough precision to store either sum or division without rounding.

Using $"{sum} {division} - {result}" and switching from float to double produces only one result with fractional digits:

3.1578948 6 - 6
 3.6842105999999997 6.999999999999999 - 6 
4.2105264 8 - 8

Notice that even sum has rounding errors.

Only decimal has enough precision to store an accurate result and avoid rounding errors:

2.63157900 5 - 5
 3.15789480 6 - 6 
3.68421060 7 - 7 
4.21052640 8 - 8 
4.73684220 9 - 9

Casting to Int returns different value than FloorToInt for positive number in c#

1 Answers1