10

I am storing many longitudes and latitudes as doubles, I am wondering if I can get away with storing them as floats.

To answer this question, I need to know the approximate resolution of a single precision floating point number when the stored values are longitudes / latitudes (-180 to +180).

mskfisher
  • 3,291
  • 4
  • 35
  • 48
Robert
  • 37,670
  • 37
  • 171
  • 213

4 Answers4

14

Your question may have several interpretations.

If it is just for angles and for storage on a disk or on a device i would suggest you to store your values using a totally different technique: store as 32 bit integer.

int encodedAngle = (int)(value * (0x7FFFFFFF / 180.0));

To recover it, do the contrary.

double angle = (encodedAngle / (0x7FFFFFFF / 180.0));

In this way you have full 31 bit resolution for 180 degrees and 1 bit for the sign.

You can use this way also to keep your values in ram, the cost of this coversion is higher compared to work directly with doubles, but if you want to keep your memory low but resolution high this can work quite well. The cost is not so high, just a conversion to/from integer from/to double and a multiplication, modern processors will do it in a very little amount of time, and since the accessed memory is less, if the list contains a lot of values, your code will be more friendly with processor cache.

Your resolution will be 180 / ((2^31) - 1) = 8.38190318 × 10^-8 degrees, not bad :)

Uli Köhler
  • 13,012
  • 16
  • 70
  • 120
Salvatore Previti
  • 8,956
  • 31
  • 37
  • He cannot use unsigned integers because he has to represent values from -360 to 360. Nice answer. – Pascal Cuoq Oct 28 '11 at 21:08
  • 2
    For lat/long, +-360 is surely a bogus requirement... Anyway this answer is correct, and using floating point is definitely wrong, unless you want great precision near London and horrible quantization in the Pacific... – R.. GitHub STOP HELPING ICE Oct 29 '11 at 00:23
  • 2
    @SalvatorePreviti Thanks for the your answer, its helped reduce my binary size for 21mb to 17mb. It can now be downloaded over the air! – Robert Nov 08 '11 at 11:27
  • latitude actually goes from -90 to 90. So for latitude you can just substitute 90 for 180 in the above encoding/decoding equations. – Markus Oct 17 '17 at 15:52
7

The resolution you can count on with single-precision floats is about 360 / (2 ^ 23) or 4 * 10 ^ -5.

More precisely, the largest single-precision float strictly inferior to 360. (which is representable exactly) is about 359.999969. For the whole range -360. .. 360, you will be able to represent differences at least as small as the difference between these two numbers.

Pascal Cuoq
  • 79,187
  • 7
  • 161
  • 281
  • Fantastic, by my calculations thats around a maximum of 5 meters for a lon/lat, wich is not acceptable for me. Thanks for you help! – Robert Oct 28 '11 at 20:48
  • 2
    @Robert When you need uniform resolution, floating-point formats are a bit wasteful (although I would concede they are so convenient). With single-precision floats, the resolution is limited by the 23-bit mantissa when you could have 2 ^ 32 uniformly spaced values. For double-precision, the resolution is limited by the 52-bit mantissa where you could use all 64 bits for representing uniformly spaced values. – Pascal Cuoq Oct 28 '11 at 20:54
2

Depends, but rather not.

32-bit float stores 7 significant digits. That is normally too little for storing the proper resolution of longitude/latitude. For example, openstreetmap.org uses six digits after the decimal point, so minimum eight, maximum total of ten digits.

In short, use float64.

Michał Leon
  • 2,108
  • 1
  • 15
  • 15
1

Usually floats are 4 bytes (32 bits) while doubles are double that. However, the exact precision if you're doing calculations is implementation (and hardware) specific. On some systems all floats will be stored as doubles, just to add to the confusion.

FeifanZ
  • 16,250
  • 7
  • 45
  • 84