ElasticSearch: compute arcDistance from nested field in Painless script

Question

I need to compute an arc distance inside a Painless script, but have not found a way of accessing the geo APIs in this case, i.e:

• the first point is passed to the script as a param - which means I only get primitive values
• the second point is read from a nested document - which means I can't read the field using the doc[myGeoField].value API

In both cases, I am unable to instantiate a org.elasticsearch.index.fielddata.ScriptDocValues.GeoPoints which would give me access to the .arcDistance() method.

I am also unable to use the org.elasticsearch.common.geo.GeoUtils class which is not whitelisted in Painless.

Is there any other option for computing an arc-distance in a script ?

Note: for reasons that would be too long to explain here, I have to use a script for this use case - doing it in a query / filter is not an option.

Answer 1

What you are trying to do, seems to be impossible because of poor design choices by Elastic.

I was trying to do the same thing (only with planeDistance), and ended up implementing planeDistance in Painless.

It seems that methods such as arcDistance and planeDistance are, inexplicably, implemented in the GeoPoints class only, and not in GeoPoint, although it would make more sense. Also, these methods only work for the first element in a list of GeoPoints.

I really hope someone can correct me, although my experience with Painless has been a pain in the ass. If you don't expect to change it often, I suggest you go with something more mature, like a Java plugin. Painless is painful to debug, has a total lack of tools, and it has no documentation whatsoever. They also deprecated Groovy and Python, as if Painless is a decent replacement.

Nevertheless, here's the code for the distance in miles, for completion (version of planeDistance, use it if you don't need very accurate distance):

double lat1 = location1.lat;
double lon1 = location1.lon;
double lat2 = location2.lat;
double lon2 = location2.lon;
double to_radian = 0.01745329251; /* PI / 180 */
double earth_radius_in_miles = 3959;

double x = (lon2 - lon1) * to_radian * Math.cos((lat2 + lat1) / 2.0 * to_radian);
double y = (lat2 - lat1) * to_radian;
double distanceInMiles = Math.sqrt(x * x + y * y) * earth_radius_in_miles;

Answer 2

I solve this issue with inner_hits, see the query below:

{
    "query": {
        "nested": {
            "path": "campi",
            "query": {
                "query_string": {
                    "query": "*"
                }
            },
            "inner_hits": {
                "script_fields": {
                    "distanceInMeters": {
                        "script": {
                            "inline": "!doc['campi.location'].empty ? doc['campi.location'].arcDistance(params.lat, params.lon) : 0",
                            "lang": "painless",
                            "params": {
                                "lon": -43.9207766,
                                "lat": -19.910621
                            }
                        }
                    }
                }
            }
        }
    }
}

Below, an example of Response:

{
    "took": 42,
    "timed_out": false,
    "terminated_early": false,
    "_shards": {
        "total": 3,
        "successful": 3,
        "failed": 0
    },
    "hits": {
        "total": 42,
        "max_score": 1.0,
        "hits": [
            {
                "_index": "search",
                "_type": "baseCourse",
                "_id": "1813-67-14-2309",
                "_score": 1.0,
                "_source": {
                    "cityName": "Belo Horizonte",
                    "institutionName": "CENTRO DE EDUCAÇÃO SUPERIOR",
                    "campi": [
                        {
                            "address_zipCode": null,
                            "address_street": "Avenida Contorno, 6.475 - São Pedro",
                            "stateCode": "MG",
                            "id": 33,
                            "campus_name": "Unidade SEDE"
                        }
                    ],
                    "baseCourseName": "ADMINISTRAÇÃO"
                },
                "inner_hits": {
                    "campi": {
                        "hits": {
                            "total": 1,
                            "max_score": null,
                            "hits": [
                                {
                                    "_nested": {
                                        "field": "campi",
                                        "offset": 0
                                    },
                                    "_score": null,
                                    "fields": {
                                        "distanceInMeters": [
                                            3593.558492923913
                                        ]
                                    },
                                    "sort": [
                                        3593.558492923913
                                    ]
                                }
                            ]
                        }
                    }
                }
            }
        ]
    }
}