3

I have an A* planning algorithm written purely in Python (using this excellent resource on path planning). The grid class with associated methods for movement and costs is defined as follows:

class SquareGrid:
      def __init__(self, width, height):
          self.width = width
          self.height = height
          self.walls = []
          self.weights = []

      def in_bounds(self, id):
          (x, y) = id
          return 0 <= x < self.width and 0 <= y < self.height

      def passable(self, id):
          return id not in self.walls

      def neighbors(self, id):
          (x, y) = id
          results = [(x + 1, y), (x + 1, y - 1), (x, y - 1), (x - 1, y - 1), (x - 1, y), (x - 1, y + 1), (x, y + 1), (x + 1, y + 1)]
          if (x + y) % 2 == 0: results.reverse()  # aesthetics

          results = [r for r in results if self.in_bounds(r)]
          results = [r for r in results if self.passable(r)] # this is where things slow down
          return results

      def cost(self, from_node, to_node):
          return (to_node[0] - from_node[0])**2 + (to_node[1] - from_node[1])**2

Now, I'd like to speed up execution a bit using Cython's static compiler goodness. One option would have been to rewrite the whole thing with static typing in Cython. I profiled the pure Python code using cProfiler to see where the bottleneck lies, and unsurprisingly enough, about 70% of the total execution time went into the neighbors method (which computes the valid neighboring nodes around the current node). More specifically, the list comprehension line in neighbors calls passable more than 33,000 times for a given toy example. passable checks if the node given in its argument is marked as "obstacle" or not by searching through SquareGrid's knowledge of obstacles (SquareGrid.walls, a list of location tuples) and returns a boolean accordingly. It seemed to me that just by optimizing this particular method, I'd get significant gains in speed. So I set about rewriting passable in Cython.

I'm a complete noob at Cython and C/C++ in general, so I'd appreciate if any mistake in understanding how this thing actually works is pointed out. I created a pyrex file passable_cy.pyx with the hope that compiling it using Cython/GCC++ and then binding it to a SquareGrid object in the main script would be enough. This is how I defined passable_cy.pyx:

cpdef bint passable(object self, tuple id):
    return id not in self.walls

The accompanying setup.py file:

from distutils.core import setup
from Cython.Build import cythonize

setup(name="cynthonized_passable", ext_modules=cythonize(['passable_cy.pyx']),)

And this is how I bound the new cynthonized method to SquareGrid in the main A* python script:

 g = SquareGrid(100, 100) # create grid with dimensions 100x100
 g.passable = types.MethodType(passable_cy.passable, g)

Everything compiled properly and the whole thing executed without problems. There was no speed improvement whatsoever, which I kind of expected (seemed too straightforward). How do I proceed from here? Is this method-binding thing the best way to go about it? I'm sure more can be done in passable_cy.pyx, but I'm too unfamiliar with C/C++ to know what to do.

Dalek
  • 4,168
  • 11
  • 48
  • 100
wheatley
  • 51
  • 4
  • 4
    Before bringing in Cython, you should consider making algorithmic improvements. `passable` is currently making a linear search through a list of all walls just to check whether its argument is a wall. Why not have SquareGrid store an *actual grid* of walls, so you can just look at the grid cell in O(1) and see whether there's a wall there? – user2357112 Jan 01 '18 at 07:14
  • 1
    Look-up in a build-in data structure isn’t something Cython could speed up. You could use a set instead of a list for ‘walls’ - the lookup would become O(1) instead of O(n). – ead Jan 01 '18 at 15:15
  • Using a set for `SquareGrid.walls` did speed up things massively, thanks! Although this does satisfy my performance requirements, do you think it's worth exploring the Cython option here? If so, a few pointers (heh) on how to proceed would be much appreciated. – wheatley Jan 02 '18 at 01:04
  • Cython would be my last option, if I were sure everything else is optimal. Is passable still the bottleneck? – ead Jan 02 '18 at 06:54
  • No, the passable bottleneck has been resolved. I'm fairly confident the pure Python code is as fast as it can get, barring a few minor alterations. I'm just curious if any more significant speed up can be achieved with Cython. – wheatley Jan 02 '18 at 19:23
  • I doubt you'd get much more speed-up. You'd probably need to make moderately big changes for pretty little gain. If I had to do it I'd probably make `results` an `N` by `2` typed array and try to deal with typed numbers rather than tuples. I can't see how you'd realistically beat a quick `set` lookup though. – DavidW Jan 03 '18 at 18:56
  • I get why people are telling you to avoid Cython here, but I can't get an answer to the overall question asked in this question: is it possible to cythonize just specific methods of a class? Does cython work with that? – Eric C. Feb 17 '22 at 15:31

0 Answers0