7

When I run the following code, I get 0 printed on both lines:

Double a = 9.88131291682493E-324;
Double b = a*0.1D;
Console.WriteLine(b);
Console.WriteLine(BitConverter.DoubleToInt64Bits(b));

I would expect to get Double.NaN if an operation result gets out of range. Instead I get 0. It looks that to be able to detect when this happens I have to check:

  • Before the operation check if any of the operands is zero
  • After the operation, if neither of operands were zero, check if the result is zero. If not let it run. If it is zero, assign Double.NaN to it instead to indicate that it's not really a zero, it's just a result that can't be represented within this variable.

That's rather unwieldy. Is there a better way? What Double.NaN is designed for? I'm assuming some operations must have return it, surely designers did not put it there just in case? Is it possible that this is a bug in BCL? (I know unlikely, but, that's why I'd like to understand how that Double.NaN is supposed to work)

Update

By the way, this problem is not specific for double. decimal exposes it all the same:

Decimal a = 0.0000000000000000000000000001m;
Decimal b =  a* 0.1m;
Console.WriteLine(b);

That also gives zero.

In my case I need double, because I need the range they provide (I'm working on probabilistic calculations) and I'm not that worried about precision.

What I need though is to be able to detect when my results stop mean anything, that is when calculations drop the value so low, that it can no longer be presented by double.

Is there a practical way of detecting this?

Andrew Savinykh
  • 25,351
  • 17
  • 103
  • 158
  • But its really not NaN, just closer to zero than what you can represent. – SimpleVar Jun 09 '15 at 08:38
  • If you need to detect loss of precision because you cannot have precision lost, use a `Decimal` or equivalent integer type. – Adam Houldsworth Jun 09 '15 at 08:38
  • 1
    NaN isn't used for underflow. It's used for things like the square root of a negative number. – Matthew Watson Jun 09 '15 at 08:38
  • [NaN](http://en.wikipedia.org/wiki/NaN) is designed for indeterminate results, like `0/0`. You posted a perfectly valid multiplication which *can* result in a 0 due to rounding errors – Panagiotis Kanavos Jun 09 '15 at 08:41
  • 1
    @AdamHouldsworth You get the same problem with Decimal. In fact it's even worse for this example because Decimal can't even represent 9.88131291682493E-324m exactly (it represents it as 0) because it has a MUCH smaller range than Decimal. It's much better for numbers in its supported range, though. – Matthew Watson Jun 09 '15 at 08:43
  • @MatthewWatson I was only stating that if lost accuracy is the problem because of floating point errors you need to move away from floating point numbers and use integers. `decimal` is limited range but serves as an example. A custom integer of any size can be created to meet the required range or accuracy. – Adam Houldsworth Jun 09 '15 at 08:45
  • @AdamHouldsworth Yes, I agree that neither double nor decimal will solve this kind of thing. (I kind of think that you might be meaning that he could use `BigInteger` rather than decimal perhaps?) – Matthew Watson Jun 09 '15 at 08:49
  • 2
    Ah, probabilistic calculations. You definitely don't want to use floats for that. Just use rational numbers instead. Or, since it doesn't make any sense to *ever* get a value of `0`, consider `0` to *always* be an error value. There is no probability of `0` (nor `1`), that simply doesn't make any sense. If for whatever reason you reach `0` or `1`, you're done - you lost. – Luaan Jun 09 '15 at 08:58

2 Answers2

6

Double works exactly according to the floating point numbers specification, IEEE 754. So no, it's not an error in BCL - it's just the way IEEE 754 floating points work.

The reason, of course, is that it's not what floats are designed for at all. Instead, you might want to use decimal, which is a precise decimal number, unlike float/double.

There's a few special values in floating point numbers, with different meanings:

  • Infinity - e.g. 1f / 0f.
  • -Infinity - e.g. -1f / 0f.
  • NaN - e.g. 0f / 0f or Math.Sqrt(-1)

However, as the commenters below noted, while decimal does in fact check for overflows, coming too close to zero is not considered an overflow, just like with floating point numbers. So if you really need to check for this, you will have to make your own * and / methods. With decimal numbers, you shouldn't really care, though.

If you need this kind of precision for multiplication and division (that is, you want your divisions to be reversible by multiplication), you should probably use rational numbers instead - two integers (big integers if necessary). And use a checked context - that will produce an exception on overflow.

IEEE 754 in fact does handle underflow. There's two problems:

  • The return value is 0 (or -1 for negative undreflow). The exception flag for underflow is set, but there's no way to get that in .NET.
  • This only occurs for the loss of precision when you get too close to zero. But you lost most of your precision way long before that. Whatever "precise" number you had is long gone - the operations are not reversible, and they are not precise.

So if you really do care about reversibility etc., stick to rational numbers. Neither decimal nor double will work, C# or not. If you're not that precise, you shouldn't care about underflows anyway - just pick the lowest reasonable number, and declare anything under that as "invalid"; may sure you're far away from the actual maximum precision - double.Epsilon will not help, obviously.

Luaan
  • 62,244
  • 7
  • 97
  • 116
  • How exactly in your opinion decimal can help solve this problem? – Andrew Savinykh Jun 09 '15 at 08:39
  • @zespri `decimal` is an integer numerical type so doesn't suffer from floating point errors. – Adam Houldsworth Jun 09 '15 at 08:39
  • @Luaan It may be worth noting that there are two `NaN` values and they don't equate to each other, so you need to use `double.IsNaN` to check, one is the constant and the other is a result of your stated calculations I believe. – Adam Houldsworth Jun 09 '15 at 08:41
  • @luaan In this particular case, it isn't a problem of `decimal` vs `double`, it is a problem that both have a finite precision, and both can't handle `9.88131291682493E-325` – xanatos Jun 09 '15 at 08:41
  • @AdamHouldsworth There are so many `NaN` values by the standard that if you begin to count them today, your grand grand nephew will still continue counting them in the future :-) – xanatos Jun 09 '15 at 08:42
  • `decimal` has lower range but higher precision compared to `double`. To get infinite precision you must use two `BigInteger`s for numerator and denominator. – SimpleVar Jun 09 '15 at 08:43
  • Test for IsPositiveInfinity and IsNegativeInfinity. – Paul Zahra Jun 09 '15 at 08:43
  • @xanatos Fair enough, I didn't know that, all I know is if you need to care about it you need to use that static function :-) – Adam Houldsworth Jun 09 '15 at 08:43
  • @xanatos Two BigIntegers, a/b. Is it more clear? "nom and dom" really wasn't clear, tbh. I fixed it. – SimpleVar Jun 09 '15 at 08:44
  • @AdamHouldsworth "decimal is an integer numerical type so doesn't suffer from floating point errors" - Really? If that was true then `Console.WriteLine(3m*(1m/3m));` should print `3`... Try it and see what it actually prints. ;) (Are you mixing up `Decimal` and `BigInteger`?) – Matthew Watson Jun 09 '15 at 08:48
  • @MatthewWatson `2.99...` Indeed. Only *better* precision, but not quite perfect. – SimpleVar Jun 09 '15 at 08:50
  • 1
    @MatthewWatson My understanding is that your example isn't a floating point error problem. It is incorrect yes because one third cannot be accurately represented in that data type. – Adam Houldsworth Jun 09 '15 at 08:52
  • @PaulZahra I would assume that is because of the data width, 32 vs 64 vs 128 bits. – Adam Houldsworth Jun 09 '15 at 08:53
  • btw... The Decimal type has the slowest performance of all the elementary numeric data types. If your integer numbers do not attain such large values and are always positive or zero, consider the ULong type. A variable of the ULong Data Type (Visual Basic) can hold integers from 0 through 18,446,744,073,709,551,615 (1.8...E+19). Operations with ULong numbers are much faster than with Decimal, although not quite as efficient as with UInteger. – Paul Zahra Jun 09 '15 at 08:53
  • @MatthewWatson The point is that it's precise to a given decimal, unlike float. Which replicates the way *humans* calculate most of the time - that's the useful attribute. You'd still `Round` the results, just like humans (read some instructions on calculating taxes, for example - there's always stuff like "divide the two numbers, round to 4 decimal places, ..."; `decimal` allows you to do that *precisely*). – Luaan Jun 09 '15 at 08:54
  • @MatthewWatson Apologies, I may be mistaken. I thought `decimal` was classed as an integral type, but looking at the MSDN docs for integral and floating types tables, it doesn't appear in either... – Adam Houldsworth Jun 09 '15 at 08:56
  • @PaulZahra How would you represent `9.88131291682493E-324` with `ULong`? Or `0.5` for that matter? – SimpleVar Jun 09 '15 at 08:57
  • @Luaan Yes, Decimal is good for financial calculations, but it is generally unimportant for other mathematical calculations (particularly scientific and engineering related algorithms). As a mathematician, I'm somewhat wary of what I see as a kneejerk suggestion to use `decimal` whenever a rounding issue rears its head. Sure, it's often the right solution, but it really needs to be thought about more than it seems to be sometimes... – Matthew Watson Jun 09 '15 at 09:03
  • @MatthewWatson Yup. But in a scientific calculation, you don't care about getting too close to zero - it's simply close enough not to matter. *Real* math is a bad fit for either, of course - although there are a few simplified cases that can be handled gracefully. – Luaan Jun 09 '15 at 09:05
  • @Luaan, can you provide some evidence, that "Double works exactly according to the floating point numbers specification, IEEE 754". I have doubts about that because a believe that IEEE 754 defines underflow condition, and this is not the same as flushing the result to zero. But my uncertainty was what prompted the question. You appear to be quite certain, so I'm sure it will be easy for you to back up your claim. – Andrew Savinykh Jun 09 '15 at 09:18
  • @zespri Well, the number ranges in IEEE 754 explicitly specify negative and positive underflow - and they're *exactly* the same (bit-)value as `-0` and `+0`, respectively. The underflow *should* set the floating point exception flags, though (which are ignored everywhere by default). Maybe there is a solution for your problem after all, but it will probably involve native code (and will only work correctly on fully compliant hardware). Is that acceptable for you? It's not possible to share the spec, sadly - it's not an open spec. – Luaan Jun 09 '15 at 09:33
  • @zespri That said, I'd still recommend simply using `0` as an error value of itself - it should never ever appear in probabilistic calculations. The problem is, your probabilities are going to get truncated anyway, long before you reach `0`. If that bothers you (and it does if you care about `0` as a special case), rational numbers are the only way I'm affraid. – Luaan Jun 09 '15 at 09:36
  • @Luaan yep, that last comment I agree with. Do you have something like [this](https://msdn.microsoft.com/en-us/library/microsoft.solverfoundation.common.rational%28v=vs.93%29.aspx) in mind? – Andrew Savinykh Jun 09 '15 at 09:57
  • @zespri Yeah, that's one of those I had a look at. It seems it should be a great fit for your purposes. Of course, the whole Solver Foundation is huge, but it looks like a great implementation. And perhaps you'll even find other useful stuff there for your problem. – Luaan Jun 09 '15 at 11:02
5

All you need is epsilon.

This is a "small number" which is small enough so you're no longer interested in.

You could use:

double epsilon = 1E-50;

and whenever one of your factors gets smaller than epislon you take action (for example treat it like 0.0)

DrKoch
  • 9,556
  • 2
  • 34
  • 43
  • Using epsilon is a good idea. For probabilistic calculations it would make sense to turn any resulted 0 into `double.Epsilon` – SimpleVar Jun 09 '15 at 09:03
  • @YoryeNathan, yes, I like that. – Andrew Savinykh Jun 09 '15 at 09:06
  • @zespri DrKoch's version is smarter - if you just turn every zero to `double.Epsilon`, you can just as easily pull the number out of a hat. This will be much worse than just keeping "almost zero" zero for probabilities. Not only did you lose the symmetry, but you're actually able to produce any probability whatsoever as a result. There's a reason we tend to avoid dividing by zero :D – Luaan Jun 09 '15 at 09:08
  • I really don't know what you want to be doing with epsilon, I mainly commented to mention that there is a defined `double.Epsilon`. It is you that said *"If for whatever reason you reach 0 or 1, you're done - you lost."* – SimpleVar Jun 09 '15 at 09:12