Questions tagged [ieee-754]

IEEE 754 is the most common & widely used floating-point standard, notably the single-precision binary32 aka float and double-precision binary64 aka double formats.

IEEE 754 is the Institute of Electrical and Electronics Engineers standard for floating-point computation, and is the most common & widely used implementation thereof.

As well as formats, IEEE754 also defines the basic operations, + - * / and sqrt, as producing correctly-rounded results (error <= 0.5ulp). Other functions like pow and sin are not required to be as accurate; that's an implementation choice between precision and performance.

This is why many CPU instruction sets only include the basic operations (including sqrt).

1447 questions
-5
votes
1 answer

Different pattern using the same float constant values leads to different results

in the following snippet of go code I struggle to understand why the results are different: func main() { a := -0.2; b := -0.1; fmt.Println(a+b) //Outputs expected float value with rounding error : -0.30000000000000004 c :=…
Lou-adrien
  • 71
  • 5
-5
votes
2 answers

float to embedded c float over UART

I am trying to send python float's over UART to an Embedded c processor, the MKE14 from NXP. In python I am using the Struct library to make a 32 bit float and send this over UART. I checked both float impelementations and there both "IEEE-754". I…
R Coppens
  • 1
  • 2
-5
votes
2 answers

I knew that 0.1D+0.2D==0.30000000000000004D - why 0.1D+0.1D==0.2D?

Neither 0.1d, 0.2d or 0.3d can be represented exactly in binary. Why is 0.1D + 0.1D == 0.2D true in Java? the 64-bit binary number is how to become 0.1,0.2,0.3?????Where is the code?
pppp
  • 17
  • 1
-6
votes
2 answers

a is a double, printf("%d", a); works differently in IA32 and IA32-64

Why does The following code work totally differently on IA-32 and x86-64? #include int main() { double a = 10; printf("a = %d\n", a); return 0; } On IA-32, the result is always 0. However, on x86-64 the result…
Martin Gao
  • 61
  • 4
-6
votes
2 answers

Problematic understanding of IEEE 754

First of all i woild like to point out that i am not native speaker and i really need some terms used more commonly. And the second thing i would like to mention is that i am not a math genious. I am really trying to understand everything about…
Genis
  • 1
  • 7
-7
votes
2 answers

Blatant floating point error in C++ program

I am assigning a double literal to a double variable. The variable's value gets truncated, otherwise I cannot understand why, for example the difference diff is 0.0. Sorry for the code duplication at setprecision but I am really pissed off. #include…
Emil Mocan
  • 51
  • 7
-11
votes
1 answer

What is IEEE-754?

MDN puts the isNaN() function in a nutshell: "is this value, when coerced to a numeric value, an IEEE-754 'Not A Number' value?" What is IEEE-754 ? P.S. I have read and researched quite a bit about isNaN , and have seen THIS, thread too. I just…
Alexander Solonik
  • 9,838
  • 18
  • 76
  • 174
1 2 3
96
97