Error 502 while trying to use geopy and ArcGIS

Question

So I keep running into an issue where my code gets a HTTP Error 502:Bad Gateway. I have essentially the same code that works fine, just with a smaller data file. Is it just the amount of data that is messing it up or have I created another issue. Thanks for your help!

    import pandas
    from geopy.geocoders import ArcGIS
    nom = ArcGIS(timeout=300)
    info = pandas.read_csv('file here')
    info['Address'] = info['city'] + ', ' + info['state'] + ', ' + 'USA'
    info['Coordinates'] = info['Address'].apply(nom.geocode)
    print(info)

        Traceback (most recent call last):
        File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\site-packages\geopy\geocoders\base.py", line 367, in _call_geocoder
    page = requester(req, timeout=timeout, **kwargs)
        File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
        File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 640, in http_response
    response = self.parent.error(
        File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
       File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 502, in _call_chain
    result = func(*args)
       File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
    urllib.error.HTTPError: HTTP Error 502: Bad Gateway

During handling of the above exception, another exception occurred:

       Traceback (most recent call last):
       File "C:/Users/roger/PycharmProjects/", line 9, in <module>
    info['Coordinates'] = info['Address'].apply(nom.geocode)
       File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\series.py", line 3848, in apply
    mapped = lib.map_infer(values, f, convert=convert_dtype)
       File "pandas\_libs\lib.pyx", line 2329, in pandas._libs.lib.map_infer
       File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\site-packages\geopy\geocoders\arcgis.py", line 197, in geocode
    response = self._call_geocoder(url, timeout=timeout)
       File "C:\Users\roger\AppData\Local\Programs\Python\Python38\lib\site-packages\geopy\geocoders\base.py", line 389, in _call_geocoder
    raise ERROR_CODE_MAP[http_code](message)
       geopy.exc.GeocoderServiceError: HTTP Error 502: Bad Gateway

RateLimiter might help, see https://geopy.readthedocs.io/en/stable/#usage-with-pandas — KostyaEsmukov, Jun 11 '20 at 06:42

score 1 · Answer 1 · answered Jun 14 '20 at 15:53

I wound up solving the problem by batching out the data in groups of 50. This was the most I could do without ArcGIS giving me a 502 error. Hope this helps

    info['Address'] = info['city'] + ', ' + info['state'] + ', ' + 'USA'
    total_length = len(info)
    def make_file(data):
         data.to_csv('file here', mode='a')
    query_start = 0
    query_end = query_start + 50
    while query_start < total_length + 1:
        brevinfo = info[query_start:query_end]
        brevinfo['Coordinates'] = brevinfo['Address'].apply(nom.geocode)
        brevinfo["Latitude"] = brevinfo["Coordinates"].apply(lambda x: 
                               x.latitude if x != None else None)
        brevinfo["Longitude"] = brevinfo["Coordinates"].apply(lambda x: 
                                x.longitude if x != None else None)
        print(brevinfo)
        make_file(brevinfo)
        query_start += 50
        query_end += 50
        time.sleep(1)

Error 502 while trying to use geopy and ArcGIS

1 Answers1