0

No matter the size of the graph and the server I use, any time I attempt to route by the dijkstra_one_to_many algorithm, I overflow my heap. Test environment is a m3.2xlarge with 30gb of RAM and 2x80gb SSD drives.

java.lang.OutOfMemoryError: Java heap space

I've tracked down the code block that is the problem inside com.graphhopper.routing.DijkstraOneToMany in the findEndNode method:

    while (true) {
        visitedNodes++;
        EdgeIterator iter = outEdgeExplorer.setBaseNode(currNode);
        while (iter.next()) {
            int adjNode = iter.getAdjNode();
            int prevEdgeId = edgeIds[adjNode];
            if (!accept(iter, prevEdgeId))
                continue;

            double tmpWeight = weighting.calcWeight(iter, false, prevEdgeId) + weights[currNode];
            if (Double.isInfinite(tmpWeight))
                continue;

            double w = weights[adjNode];
            if (w == Double.MAX_VALUE) {
                parents[adjNode] = currNode;
                weights[adjNode] = tmpWeight;
                heap.insert_(tmpWeight, adjNode);
                changedNodes.add(adjNode);
                edgeIds[adjNode] = iter.getEdge();

            } else if (w > tmpWeight) {
                parents[adjNode] = currNode;
                weights[adjNode] = tmpWeight;
                heap.update_(tmpWeight, adjNode);
                changedNodes.add(adjNode);
                edgeIds[adjNode] = iter.getEdge();
            }
        }

        if (heap.isEmpty() || isMaxVisitedNodesExceeded() || isWeightLimitExceeded())
            return NOT_FOUND;

        // calling just peek and not poll is important if the next query is cached
        currNode = heap.peek_element();
        if (finished())
            return currNode;

        heap.poll_element();
}
```

It seems to never find the end node and the internal data structure (min heap?) grows and grows and grows until I run out of heap space. Why is this happening?

I can post my config.properties as well if that is needed. Thank you Peter for putting together an awesome piece of open source software.

trincot
  • 317,000
  • 35
  • 244
  • 286
Chadderall
  • 89
  • 9
  • Well, have you tried increasing your heap space? (How large is the graph and what is the current heap size?) One assumes if your (not-shown) `isMaxVisitedNodesExceeded()` is working correctly that you are not running your `heap` field variable into infinity... – BadZen Nov 09 '16 at 23:01
  • I set the heap size to 27gb via jvm args. The graph of the north america pbf is 4gb. Maybe I can lower the max number of nodes visited, but I don't think I am using the algorithm classes correctly. – Chadderall Nov 10 '16 at 18:24

1 Answers1

0

The DijkstraOneToMany class is currently not intended to be (easily) used from outside e.g. it is not thread safe. You could switch to a simple Dijkstra without a different finish condition to lower your memory requirements for simple cases.

That said ... there can be the following issues:

  • make sure that you cache the calls to DijkstraOneToMany as it creates big initial datastructures
  • again: use it from one thread only (e.g. via ThreadLocal)
  • It seems to never find the end node -> Maybe you use the QueryGraph with it? That will not really work as we create so called virtual nodes in the QueryGraph which DijkstraOneToMany does not know, instead try to pick the next tower node e.g. via avoiding QueryGraph completely or manually via an EdgeIterator

Thank you Peter for putting together an awesome piece of open source software.

It was not just me - it is a community effort :) !

Karussell
  • 17,085
  • 16
  • 97
  • 197
  • So from my understanding, it sounds like my query point may get snapped to a virtual edge and the DijkstraOneToMany is constantly traversing the base graph looking for an ID it knows nothing about? – Chadderall Nov 10 '16 at 23:49
  • Yes. The query graph introduces virtual node and edge IDs to eg. avoid special handling in the algorithm for places somewhere between two junctions. – Karussell Nov 11 '16 at 14:18