Questions tagged [ieee-754]

IEEE 754 is the most common & widely used floating-point standard, notably the single-precision binary32 aka float and double-precision binary64 aka double formats.

IEEE 754 is the Institute of Electrical and Electronics Engineers standard for floating-point computation, and is the most common & widely used implementation thereof.

As well as formats, IEEE754 also defines the basic operations, + - * / and sqrt, as producing correctly-rounded results (error <= 0.5ulp). Other functions like pow and sin are not required to be as accurate; that's an implementation choice between precision and performance.

This is why many CPU instruction sets only include the basic operations (including sqrt).

1447 questions
13
votes
1 answer

Why does my implementation of SVG arc conversion not pass QuickCheck?

I implemented W3s recommended algorithm for converting SVG-path arcs from endpoint-arcs to center-arcs and back in Haskell. type EndpointArc = ( Double, Double, Double, Double , Bool, Bool, Double, Double, Double ) type…
kasbah
  • 903
  • 5
  • 23
13
votes
3 answers

Portable serialisation of IEEE754 floating-point values

I've recently been working on a system that needs to store and load large quantities of data, including single-precision floating-point values. I decided to standardise on network byte order for integers, and also decided to store floating point…
Peter T.B. Brett
  • 1,250
  • 11
  • 20
12
votes
2 answers

How can I convert four characters into a 32-bit IEEE-754 float in Perl?

I have a project where a function receives four 8-bit characters and needs to convert the resulting 32-bit IEEE-754 float to a regular Perl number. It seems like there should be a faster way than the working code below, but I have not been able to…
Dan Littlejohn
  • 1,329
  • 4
  • 16
  • 30
12
votes
1 answer

python: unpack IBM 32-bit float point

I was reading a binary file in python like this: from struct import unpack ns = 1000 f = open("binary_file", 'rb') while True: data = f.read(ns * 4) if data == '': break unpacked = unpack(">%sf" % ns, data) print…
nemo
  • 12,241
  • 3
  • 21
  • 26
12
votes
4 answers

NAN propagation and IEEE 754 standard

I am designing a new microprocessor instruction set (www.forwardcom.info) and I want to use NAN propagation for tracing errors. However, there are a number of oddities in the IEEE 754 floating point standard that prevent this. First, the reason why…
A Fog
  • 4,360
  • 1
  • 30
  • 32
12
votes
3 answers

I do *not* want correct rounding for function exp

The GCC implementation of the C mathematical library on Debian systems has apparently an (IEEE 754-2008)-compliant implementation of the function exp, implying that rounding shall always be correct: (from Wikipedia) The IEEE floating point standard…
Rémi Peyre
  • 410
  • 3
  • 12
12
votes
1 answer

IEEE-754 double (64-bit floating point) vs. long (64-bit integer) revisited

I'm revisiting a question (How to test if numeric conversion will change value?) that as far as I was concerned was fully solved. The problem was to detect when a particular numeric value would overflow JavaScript's IEEE-754 Number type. The…
mckamey
  • 17,359
  • 16
  • 83
  • 116
12
votes
2 answers

Sorting floating-point values using their byte-representation

If have an 8-byte section of data and write a double-precision floating-point value to it, under what conditions will comparison by numerical comparison and lexicographic sorting of the bytes agree? Current theory: positive, big-endian I believe…
cloudfeet
  • 12,156
  • 1
  • 56
  • 57
12
votes
1 answer

A surprise with 1**math.nan and 0j**math.nan

I'm surprised that >>> import math >>> 1**math.nan 1.0 And while we are at it, also that >>> 0j**math.nan 0j I didn't find any other examples. Is there a reason or some logic I've missed that makes this the right choice? Or is this a slip? I was…
Aguy
  • 7,851
  • 5
  • 31
  • 58
12
votes
4 answers

Number of floats between two floats

Say I have two Python floats a and b, is there an easy way to find out how many representable real numbers are between the two in IEEE-754 representation (or whatever representation the machine used is using)?
astrofrog
  • 32,883
  • 32
  • 90
  • 131
12
votes
2 answers

With IEEE-754, 0 < ABS(const) < 1, is (x / const) * const guaranteed to return distinct results for distinct values of X?

Assume I do this operation: (X / const) * const with double-precision arguments as defined by IEEE 754-2008, division first, then multiplication. const is in the range 0 < ABS(const) < 1. Assuming that the operation succeeds (no overflows occur),…
Quassnoi
  • 413,100
  • 91
  • 616
  • 614
12
votes
1 answer

IEE 754 total order in standard C++11

According to the IEEE floating point wikipage (on IEEE 754), there is a total order on double-precision floating points (i.e. on C++11 implementations having IEEE-754 floats, like gcc 4.8 on Linux / x86-64). Of course, operator < on double is often…
Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547
12
votes
2 answers

What numerical algorithm is simplified by defining sqrt(-0.0) as -0.0?

The IEEE 754 standard defines the square root of negative zero as negative zero. This choice is easy enough to rationalize, but other choices, such as defining sqrt(-0.0) as NaN, can be rationalized too and are easier to implement in hardware. If…
Pascal Cuoq
  • 79,187
  • 7
  • 161
  • 281
12
votes
4 answers

How to get Python division by -0.0 and 0.0 to result in -Inf and Inf, respectively?

I have a situation where it is reasonable to have a division by 0.0 or by -0.0 where I would expect to see +Inf and -Inf, respectively, as results. It seems that Python enjoys throwing a ZeroDivisionError: float division by zero in either case. …
Chuck
  • 1,850
  • 2
  • 17
  • 28
12
votes
5 answers

Why are floating point numbers printed so differently?

It's kind of a common knowledge that (most) floating point numbers are not stored precisely (when IEEE-754 format is used). So one shouldn't do this: 0.3 - 0.2 === 0.1; // very wrong ... as it will result in false, unless some specific…
raina77ow
  • 103,633
  • 15
  • 192
  • 229