python sklearn KDTree with haversine distance

Question

I try to create a KD tree of WGS84 coordinates and find neighbors within a certain radius

from sklearn.neighbors.dist_metrics import DistanceMetric
from sklearn.neighbors.kd_tree import KDTree    
T = KDTree([[47.8665, 8.90123]], metric=DistanceMetric.get_metric('haversine'))

But get the following error:

ValueError: metric HaversineDistance is not valid for KDTree

How can I use haversine distance in a KD-Tree?

score 4 · Accepted Answer · answered Jul 05 '16 at 19:34

4

The k-d-tree can (to the best of my knowledge) only be used with Minkowski norms.

There are other trees such as the ball tree in sklearn, or the covertree in ELKI that work with Haversine distance because it is a metric.

answered Jul 05 '16 at 19:34

Has QUIT--Anony-Mousse

76,138
12
138
194

1

You can also use the KDTree but then you have to convert your longitude, latitude pairs to carthesian/euclidean values and convert the distance value back to miles or kilometers than. As far as I know you can also convert longitude and latitude to radians which gives you the distances directly in kilometers. But haven't tested that. – Matthias Aug 16 '17 at 19:39
@Matthias: Your first proposal is probably the best solution. With many queries, it will also be much faster than using latitude/longitude and haversine distance. – Eric Duminil Apr 29 '22 at 12:05

hashcode55 · Answer 2 · 2016-07-04T16:52:29.113

0

KDTree.valid_metrics

Output -

['p',
 'l1',
 'chebyshev',
 'manhattan',
 'minkowski',
 'cityblock',
 'l2',
 'euclidean',
 'infinity']

Which tells, you can't use haversine with KDTree. The reason behind it is haversine distance gives you Orthodromic distance which is the distance measure used when your points are represented in a sphere. But in a kdTree the points are organised in a tree which makes it invalid to use.

edited Jul 04 '16 at 16:52

answered Jul 04 '16 at 16:35

hashcode55

5,622
4
27
40

Thanks for the explanation. Any idea to how to make it run faster than in O(n^2) (calculating all distance pairs) ? – user307380 Jul 05 '16 at 14:02
2

There are trees which work with haversine. But the kd-tree doesn't. In python, the ball-tree is an example. – Has QUIT--Anony-Mousse Jul 05 '16 at 19:33
1

Oh I was totally unaware of that. Thanks for the info. I'll read about it. – hashcode55 Jul 06 '16 at 14:06

python sklearn KDTree with haversine distance

2 Answers2