Stack Overflow
Stack Overflow

Questions tagged [floating-point]

Floating point numbers are approximations of real numbers that can represent larger ranges than integers but use the same amount of memory, at the cost of lower precision. If your question is about small arithmetic errors (e.g. why does 0.1 + 0.2 equal 0.300000001?) or decimal conversion errors, please read the tag page before posting.

Many questions asked here about floating point math are about small inaccuracies in floating point arithmetic. To use the example from the excerpt, 0.1 + 0.2 might result in 0.300000001 instead of the expected 0.3. Errors like these are caused by the way floating point numbers are represented in computers' memory.

Integers are stored as exact values of the numbers they represent. Floating point numbers are stored as two values: a significand and an exponent. It is not possible to find a significand-exponent pair that matches every possible real number. As a result, some approximation and therefore inaccuracy is unavoidable.

Two commonly cited introductory-level resources about floating point math are What Every Computer Scientist Should Know About Floating-Point Arithmetic and the floating-point-gui.de.

FAQs:

Why 0.1 does not exist in floating point

Floating Point Math at https://0.30000000000000004.com/

Related tags:

Programming languages where all numbers are double-precision (64b) floats:

15006 questions
9
votes
2 answers

Is x/a the same as x*(1/a) for floats?

With float a = ...; and float inva = 1/a; is x / a the same as x * inva? And what is with this case: unsigned i = ...; float v1 = static_cast(i) / 4294967295.0f; float scl = 1.0f / 4294967295.0f; float v2 = static_cast(i) * scl; Is v1…
Danvil
  • 22,240
  • 19
  • 65
  • 88
9
votes
2 answers

PostgreSQL round(v numeric, s int)

Which method does Postgres round(v numeric, s int) use? Round half up Round half down Round half away from zero Round half towards zero Round half to even Round half to odd I'm looking for documentation reference.
mpapec
  • 50,217
  • 8
  • 67
  • 127
9
votes
1 answer

Converting a very small python Decimal into a non-scientific notation string

I am using the Python Decimal class for precise floating-point arithmetic. I need to convert the result number consistently into a standard notation number as a string. However, very small decimal numbers are rendered in scientific notation by…
dmi_
  • 1,187
  • 2
  • 12
  • 26
9
votes
3 answers

PHP: format any float as a decimal expansion

I'd like to create a function formatFloat() which takes any float and formats it as a decimal expansion string. For example: formatFloat(1.0E+25); // "10,000,000,000,000,000,000,000,000" formatFloat(1.0E+24); //…
TachyonVortex
  • 8,242
  • 3
  • 48
  • 63
9
votes
2 answers

Easiest way to get the machine epsilon in Go

What is the easiest way to get the machine epsilon in Go? What about other aspects of floating point numbers such as precision, min exponent, max exponent, wobble, etc? I realize that there is the math/const package with the max and min for the…
sbditto85
  • 1,485
  • 14
  • 21
9
votes
3 answers

Assigning floating value to a double value

I was referring this Oracle documentation. While trying to execute the following, public static void main(String args[]){ float f = 1.1f; double df = 1.1f; System.out.println("f=" + f); System.out.println("df=" + df); f =…
Nidheesh
  • 4,390
  • 29
  • 87
  • 150
9
votes
15 answers

Comparing IEEE floats and doubles for equality

What is the best method for comparing IEEE floats and doubles for equality? I have heard of several methods, but I wanted to see what the community thought.
Craig H
  • 7,949
  • 16
  • 49
  • 61
9
votes
3 answers

How to round a float value in c#?

I have a problem rounding a float value. I have a float value that is set 0.8 but in the c# code it is somehow 0.80000000000000004. So i want to round it so it becomes 0.8. i have tried: float roundedFloatvalue =…
Diemauerdk
  • 5,238
  • 9
  • 40
  • 56
9
votes
1 answer

Why does int/float multiplication lead to different results?

If I multiply a float and and integer like below, why does all multiplications lead to a differnt result? My expectation was a consistent result. I thought in both cases the int value gets implicitly converted to a float before multiplication. But…
Thorsten
  • 213
  • 1
  • 7
9
votes
4 answers

Free the x87 FPU Stack (ia32)

At my university we were just introduced to IA32 x87 FPU. But we weren't informed how to clear the FPU-Stack of no longer demanded elements. Imagine we're performing a simple calculation like (5.6 * 2.4) + (3.9 * 10.3). .data value1: .float…
tmuecksch
  • 6,222
  • 6
  • 40
  • 61
9
votes
3 answers

I got different results with pow(10,2) and pow(10,j), j=2;

This one prints 100: int j=2; int i= pow(10,2); printf("%d\n", i); and this one prints 99: int j=2; int i= pow(10,j); printf("%d\n", i); Why?
Rimgaudas Tumėnas
  • 131
  • 2
  • 7
9
votes
2 answers

Rounding up c# giving wrong answer

I am programming a stocks/production program for a school project (bike factory) and I have to round some numbers up such as the bikes that have to be produced (due to fallout it are doubles). If I use the following code: double test =…
Ben Non
  • 129
  • 1
  • 9
9
votes
4 answers

Floating point formatting in printf()

I have an array of floats where data are stored with varying decimal points so some are 123.40000, 123.45000, 123.45600...now if i want to print these values in the string without the 0s in the end in printf() so that they are 123.4, 123.45,…
sfactor
  • 12,592
  • 32
  • 102
  • 152
9
votes
3 answers

pow() cast to integer, unexpected result

I have some problems using an integer cast for the pow() function in the C programming language. The compiler I'm using is the Tiny C Compiler (tcc version 0.9.24) for the Windows platform. When executing the following code, it outputs the…
Jori
  • 1,122
  • 2
  • 18
  • 36
9
votes
3 answers

Extract scientific number from string

I am trying to extract scientific numbers from lines in a text file. Something like Example: str = 'Name of value 1.111E-11 Next Name 444.4' Result: [1.111E-11, 444.4] I've tried solutions in other posts but it looks like that only works for…
Josh Melson
  • 91
  • 1
  • 1
  • 2
1 2 3
…
99
100