4

I have a large data (10k rows) of locations (lat, lon) and I would like to compute a 10min-walk isochrone starting from each point with OSMnx. (I tried with openrouteservice but have some limitations). I tried with this example: https://github.com/gboeing/osmnx-examples/blob/v0.13.0/notebooks/13-isolines-isochrones.ipynb

import pandas as pd
import geopandas as gpd
from shapely.geometry import Point, Polygon
import networkx as nx
import osmnx as ox
ox.config(log_console=True, use_cache=True)

def get_isochrone(lon, lat, walk_time=10, speed=4.5):
    loc = (lat, lon)
    G = ox.graph_from_point(loc, simplify=True, network_type='walk')
    gdf_nodes = ox.graph_to_gdfs(G, edges=False)
    x, y = gdf_nodes['geometry'].unary_union.centroid.xy
    center_node = ox.get_nearest_node(G, (y[0], x[0]))
    meters_per_minute = speed * 1000 / 60 #km per hour to m per minute
    for u, v, k, data in G.edges(data=True, keys=True):
        data['time'] = data['length'] / meters_per_minute
    subgraph = nx.ego_graph(G, center_node, radius=walk_time, distance='time')
    node_points = [Point(data['x'], data['y']) for node, data in subgraph.nodes(data=True)]
    polys = gpd.GeoSeries(node_points).unary_union.convex_hull
    return polys

and then apply it on my large scale pandas DataFrame:

df.apply(lambda x: get_isochrone(x.lon, x.lat), axis=1)

But took such amount of time. 100 rows is about 3 min running time. Is there any other methods, packges to achieve this goal ? With a reasonable running time ?

Last question, what are the limitations of OSMnx request, especially for large data ? Thank you

A. Traoré
  • 76
  • 7
  • One improvement you could make is to try and avoid downloading graph edges several times. If you have a lot of points in your `df` that are closer together than 10 minutes walk, then you lose a lot of time downloading the graph data again and again. One solution could be to download the whole graph for the envelope of your dataset in memory (if it fits) outside of `get_isochrone` (using `osmnx.graph.graph_from_polygon` for instance) – Luis Blanche Jul 08 '20 at 09:38
  • @LuisBlanche that's the problem. My (lat, lon) points are far apart. Taking `graph_from_polygon` would consider a single center node. – A. Traoré Jul 08 '20 at 10:13
  • I think you need to understand what takes time in `get_isochrone`. To do that you can use [cProfile](https://docs.python.org/3.7/library/profile.html#module-cProfile) `cProfile.run(df.apply(lambda x: get_isochrone(x.lon, x.lat), axis=1)) ` – Luis Blanche Jul 08 '20 at 12:04

2 Answers2

3

This will be an inherently slow process. If you have 10,000 locations and they are all far apart from each other, then you need to download and model 10,000 local street networks to calculate accessibility around each point. This means 10,000 server calls and data downloads and graph building and topology cleaning, etc, etc.

Accordingly, 100 rows in 3 minutes seems pretty fast to me, especially considering that it means you can complete all 10,000 rows in ~300 minutes (i.e., 5 hours) given your estimated timings. Just start the process before you go to bed, and when you wake up it'll be done. This assumes that this is a one-off computation that needn't be recomputed frequently.

The other alternative would be to parallelize it by dividing it up among say 10 containers or processes, each of which tackles 1,000 locations. This would finish in ~30 minutes, given your estimated timings.

Last question, what are the limitations of OSMnx request, especially for large data ?

The limitation for working with massive network models with OSMnx is the amount of RAM on your computer.

gboeing
  • 5,691
  • 2
  • 15
  • 41
  • Yes. I had run the program yesterday but met an API Overpass error. But yes, I'm going to do it again and hope I won't encounter this error during the process. – A. Traoré Jul 09 '20 at 08:29
1

You may be able to reduce time by erasing the for loop inside the function. As you created a variable time data['time'] = data['length'] / meters_per_minute, in this case walk_time=10, speed=4.5 is like walking 75 meters per min or 750 meters in total. So instead of creating a variable time just use the length variable and walk 750 meters from the center node.

Also, get_nearest_node is now deprecated so now it's necessary nearest_nodes.

The code modified:

import pandas as pd
import geopandas as gpd
from shapely.geometry import Point, Polygon
import networkx as nx
import osmnx as ox
ox.config(log_console=True, use_cache=True)

def get_isochrone(lon, lat, walk_time=10, speed=4.5):
    loc = (lat, lon)
    G = ox.graph_from_point(loc, simplify=True, network_type='walk')
    #G = ox.project_graph(G, to_crs="4483") # Use this line if the coordinates sistem returned from polys is changed from the original (check which crs you are using)
    gdf_nodes = ox.graph_to_gdfs(G, edges=False)
    x, y = gdf_nodes['geometry'].unary_union.centroid.xy
    center_node = ox.nearest_nodes(G, Y = y[0], X= x[0])
    walking_meters = walk_time * speed * 1000 / 60 #km per hour to m per minute times the minutes to walk
    subgraph = nx.ego_graph(G, center_node, radius=walking_meters, distance='length')
    node_points = [Point(data['x'], data['y']) for node, data in subgraph.nodes(data=True)]
    polys = gpd.GeoSeries(node_points).unary_union.convex_hull
    return polys
PabloB
  • 11
  • 2