0

I have an OD matrix of some 3 million paths that I would like to route on the OSM network. Using Dijkstra, I get about 20 paths per minute or a total time of 2,500 hours. That would be barely ok if the nearest_node algorithm wouldn't miss a relatively distant nearest node. With Bellman-Ford, I don't seem to have that problem but it is a factor 40 slower - and I don't have 100K hours. ;} I have some 24 GB RAM on my choice of Ubuntu or Windows machine. Btw, public web routing services run into the same problem of not being able to find a route. My OD locations are the centroids of Census blocks in the Bay Area (where some remote ones may by a few miles from the nearest road). My two questions are: (1) how can I avoid the route can be found problem, and (2) how can I speed up the process (would t help if I use a Cloud instance with say 512 GB of RAM)? The function I am trying to get improved is route = networkx.shortest_path(G, start_node, end_node, weight='travel_time') in conjunction with start_node = osmnx.get_nearest_node(G, start) and its respective end_node counterpart

Jochen
  • 1

1 Answers1

0

My two questions are: (1) how can I avoid the route can be found problem, and (2) how can I speed up the process (would t help if I use a Cloud instance with say 512 GB of RAM)?

Regarding 1, I'd have to see an minimal reproducible example to understand what exactly you're experiencing. But regarding 2, shortest path solving is CPU bound rather than memory bound. You'd be better served, if you do it in the cloud, by distributing the paths to solve among say 100 cores so the whole process finishes within a day.

gboeing
  • 5,691
  • 2
  • 15
  • 41
  • Thank you for (2), Geoff. As for (1), here is the link (on my Google Drive) to the Python code file (small) and the dataset (16 MB): https://drive.google.com/drive/folders/1KoPVliZMr3x-reM012u6ETIslmZQ1ZwL?usp=sharing – Jochen Jun 19 '20 at 17:49
  • I do not have permission to access that URL. I'd suggest pasting a complete minimal reproducible code snippet into your original question: just enough to reproduce what you're experiencing, without downloading outside files. I'll take a look. – gboeing Jun 19 '20 at 21:09
  • Is this enough? – Jochen Jun 19 '20 at 21:22
  • Here is an OD path that cannot be routed - even with the Bellman-Ford algorithm:From: "37.849593,-122.2497462"     To: "37.5579748,-121.9669258", which according to Google Maps is from 6389 Florio St, Oakland, CA 94618 to 39200 Brahms Terrace, Fremont, CA 94538.This is the code snippet:start = (37.849593,-122.2497462)end = (37.5579748,-121.9669258)start_node = ox.get_nearest_node(G, start) end_node = ox.get_nearest_node(G, end)route = nx.shortest_path(G, start_node, end_node, weight='travel_time') This throws KeyError: 5864867474 NetworkXNoPath: No path to 5864867474. – Jochen Jun 20 '20 at 03:28
  • I can't test that because it's not reproducible: if you provide a complete code snippet (edit your original question to add it) that I can completely run from the top down, I can give you an answer. It will depend on the specifics of the variables you left undefined here in your comment, most importantly `G` itself. – gboeing Jun 20 '20 at 03:35
  • Here's a guide to reproducible examples: https://stackoverflow.com/help/minimal-reproducible-example – gboeing Jun 20 '20 at 03:35
  • See my comment six hours earlier (I sent the full code and sample data to your SCU email address). But to answer the question about G, G = create_graph("37.62711370030286, -121.71885913815301", 250000, "drive") – Jochen Jun 20 '20 at 12:21