1

I'm trying to connect to the Nominatim GeoPy API to resolve latitude/longitude coordinates from zip codes. The implementation looks as follows (minimum reproducible):

# Create a function that returns lat/lon coordinates from zip code
def get_coord(zip_code):
    geolocator = Nominatim(user_agent="GCD Data Client")
    location = geolocator.geocode({"postalcode": zip_code, "country": "US"}, exactly_one = True)
    in_lat = location.latitude
    in_lon = location.longitude
    return in_lat, in_lon

# Inputs for data
def get_params(self, zip_code):
    # Do error checking to prevent strange, unexpected output from leaving function
    out = False
    while not out:
        try:
            in_lat, in_lon = get_coord(zip_code)
            # Make sure the lat/lon pair is valid
            if abs(in_lat) > 90 or abs(in_lon) > 180:
                raise ValueError
            out = True
        except ValueError:
            print("Make sure your lat/lon coordinates are valid (-90-90, -180-180)")
    return in_lat, in_lon

if __name__ == '__main__':
    # Get some input lats and lons
    in_lats = []
    in_lons = []
    zip_codes = [14201, 80919, 84101] # Dummy data
    for zip_code in zip_codes:
        in_lat, in_lon = get_params(zip_code)
        in_lats.append(in_lat)
        in_lons.append(in_lon)
    print(in_lats)
    print(in_lons)

However, the code is not the issue. When I run this code on my local machine (windows) I get the lat/lon coordinates as expected. I'm pretty sure the issue is the communication with the GeoPy server is getting blocked by AWS given that I get this error when I run it there:

geopy.exc.GeocoderUnavailable: HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Max retries exceeded with url: /search?postalcode=14201&country=US&format=json&limit=1 (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))

I've tried some different ec2 security configurations. One using the default SSH and HTTPS (since that is what 443 routes to). I've also tried opening it up to all inbound traffic (despite that being bad security practice) to test since it is just a dev server (and have since locked it down again :) ). In both cases I still get the same issue. Is this an SSL setting I need to somehow enable?

Zach Rieck
  • 419
  • 1
  • 4
  • 23
  • No, cert nonverify is not caused by AWS. What Python and particularly OpenSSL library are you running on AWS versus on Windows, and with what truststores aka CA bundles? OpenSSL 1.0.2 can't verify a server using a LetsEncrypt cert with the 'compatibility' chain including the DST root which expired a week ago, as many sites including nominatim.openstreetmap.org do, iff the DST root is still in the truststore. See https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/ and https://www.openssl.org/blog/blog/2021/09/13/LetsEncryptRootCertExpire/ and dozens of other recent Stack Qs. – dave_thompson_085 Oct 08 '21 at 07:26

1 Answers1

2

I tried it, and I can confirm it's not an ec2 related issue, as I tried it on ec2, it works.

In short, you may need to use the latest python version or change your system certificate file.

This is my test result in 2021: in my environment, python 3.6.8 + geopy 2.2.0 + centos 7 works; python 2.7.18 + geopy 1.23.0 + centos 8 works as well; but python 2.7.5 + geopy 1.23.0 + centos 7 doesn't work and reports the certificate error.

Finally, I found the root cause, it's due to the expiration on Sept. 30 of the older 'DST' root cert used by LetsEncrypt; see https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021 . Python is affected if it uses OpenSSL below 1.1.0; see https://openssl.org/blog/blog/2021/09/13/LetsEncryptRootCertExpire.

And I made python 2.7.5 + geopy 1.23.0 + centos 7 + openssl 1.0.2k-fips work with workaround 1 which is removing the (DST Root CA X3) root certificate from the system certificate, the detailed steps are:

  1. Check DST Root is in /etc/pki/tls/certs/ca-bundle.crt, and ISRG Root X1 is in ISRG Root X1.
  2. Copy the "# DST Root CA X3" section into /etc/pki/ca-trust/source/blacklist directory as pem file
  3. run update-ca-trust
  4. Check DST Root is no longer in /etc/pki/tls/certs/ca-bundle.crt, and ISRG Root X1 is in ISRG Root X1.

Ref:

LingYan Meng
  • 699
  • 4
  • 12
  • I need some time to go through this, but this looks correct and makes sense to me. I'll verify and mark as the accepted answer if it works as expected – Zach Rieck Oct 08 '21 at 18:43