Java: Trigonometry and double inaccuracy causing NaN

Question

I have a distance formula using latitude and longitude:

distance = EARTH_MILES_RADIUS
    * Math.acos(Math.sin(lat1 / RADIAN_CONV)
    * Math.sin(lat2 / RADIAN_CONV)
    + Math.cos(lat1 / RADIAN_CONV)
    * Math.cos(lat2 / RADIAN_CONV)
    * Math.cos((lng2 - lng1) / RADIAN_CONV));

lat1,lng1,lat2,lng2 are double primitives. They come to me as double primitives and there is nothing I can do about it.

The problem is that when I have a pair of longitude or latitudes that are the same the formula sometimes returns NaN. I believe this is because I am taking the arc cosine of a number very slightly greater than 1, when in fact it should be exactly 1. I would probably have problems if the points were antipodal as well, where they might be slightly less than -1.

How can I best fix this problem?

If you want to study a better great-circle distance and bearing calculation implementation in Java, here is the place UNICAR NETCDF library http://svn.unidata.ucar.edu/repos/common/java/trunk/src/main/java/ucar/unidata/geoloc/Bearing.java `Copyright notice from UNICAR on NetCDF: These applications are copyrighted by UCAR and are intended to be freely available with very minimal restriction.` — eee, Apr 20 '11 at 03:54
@eee I see in the linked code the they have `if ((lat1 == lat2) && (lon1 == lon2))` in which case the distance is set to zero, so perhaps that is the best solution. Maybe checking for equality within a certain accuracy is unnecessary (puddingfox's solution). This was solution that I tried on my system and it worked great but I wasn't sure if it would work in every case. — Muhd, Apr 21 '11 at 04:01

score 1 · Answer 1 · edited Aug 11 '11 at 00:43

1

Check for two very close (ideally equal) values in your code. For example:

boolean doubleApproxEqual(double a, double b) {
    double PRECISION = 0.000001;
    if (Math.abs(a-b) < PRECISION) //not sure what the name of the function is
                                   //cannot be bothered to check
        return true;
    return false;
}

if you get a True, do cos(1.0) or whatever

edited Aug 11 '11 at 00:43

user85421

28,957
10
64
87

answered Apr 20 '11 at 03:16

jamesbtate

1,359
3
19
25

3

Personally, I'd use `return math.abs(a-b) < PRECISION` :-) – corsiKa Apr 20 '11 at 03:22

score 1 · Answer 2 · answered Apr 20 '11 at 03:21

1

If you are indeed calculating great circle distance as I think you are, you should use the Vincenty formula instead of what you have.

http://en.wikipedia.org/wiki/Great-circle_distance

answered Apr 20 '11 at 03:21

colechristensen

475
1
3
11

Wouldn't the Vincenty formula perform worse? I am processing a very high volume of data. – Muhd Apr 20 '11 at 03:23
Unless you are trying to write a real time application, I don't see how this could be much of an issue. – colechristensen Apr 20 '11 at 03:26
Performance is a huge issue because it is a website, and the distances are typically pretty close ( < 100 miles ). – Muhd Apr 20 '11 at 03:31
Performance of T.Vincenty formula is not a real issue if correctly implemented...see my comment below your main post on the Java implementation of it which combines Vincenty's method with other methods (Rainsford's + Helmert's elliptical terms) – eee Apr 20 '11 at 03:58

score 0 · Accepted Answer · answered Aug 11 '11 at 00:22

0

Simply checking for equality was enough to fix my problem:

if (lat1 == lat2 && lng1 == lng2) ...

answered Aug 11 '11 at 00:22

Muhd

24,305
22
61
78

Java: Trigonometry and double inaccuracy causing NaN

3 Answers3

Linked