Tornado on PyPy

Question

I'm getting the following error message when trying to run my tornado server on pypy:

/pypy3-2.4.0-osx64/site-packages/pkg_resources/__init__.py:80: UserWarning: Support for Python 3.0-3.2 has been dropped. Future versions will fail here.

Does anyone know what this is talking about?

Also why does normal python interpreter allow the following function:

   @tornado.gen.coroutine
   def get(self,id):
      doc=[]
      cursor = self.c.find({"_id":id})

      while (yield cursor.fetch_next):
         doc.append(cursor.next_object())

      return doc

However pypy complains about using a return inside a generator. I did some reading and apparently the right way is to yield instead of return?

   @tornado.gen.coroutine
   def get(self,id):
      doc=[]
      cursor = self.c.find({"_id":id})

      while (yield cursor.fetch_next):
         doc.append(cursor.next_object())

      yield doc

I changed to yielding to get rid of the errors in pypy then went back to normal python and it crashed.

That's not an error, that's a warning issued by pkg_resources telling you that future versions relying on newer python features will not run on the used pypy version (pypy3 2.4 provides only compatibility with python3.2.5). That only means you won't be able to use newer versions of setuptools. — mata, Apr 16 '16 at 10:54
Before trying to move to pypy I was running: **Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 23 2015, 02:52:03) I downloaded pypy Python 3.2.5 compatible PyPy3 2.4.0** Ok so python 3.4 might have features i'm using which aren't compatible with this pypy because it only supports up to python 3.2 Is that right? — johhny B, Apr 16 '16 at 10:59
Ok seems as if PyPy hasn't yet caught up to the latest version of 3.4 which is a shame :( Time to start donating to PyPy everyone!! — johhny B, Apr 16 '16 at 11:38

score 3 · Accepted Answer · answered Apr 16 '16 at 14:47

3

The current release of pypy3 is based on cpython 3.2. This is old enough that many packages have dropped support for it. Tornado doesn't support cpython 3.2 any more, but we do support pypy3 (the difference is the support for u"" unicode literals, which is present in pypy3 but wasn't re-added to cpython until 3.3).

You cannot replace return with yield in a coroutine; this will raise a BadYieldError. Instead, you must replace return x with raise gen.Return(x). It wasn't possible to mix return and yield in the same function until python 3.3.

answered Apr 16 '16 at 14:47

Ben Darnell

21,844
3
29
50

Thanks Ben all makes sense. As an aside if you were to look to optimise a tornado webserver without going backwards in releases (or looking for nasty hacks trying to get an older version of PyPy going). What would you look to do beyond running more processes with nginx? Would you consider Cython optimisation as the next step? – johhny B Apr 16 '16 at 14:56
Always start with profiling to figure out where your bottlenecks are. Cython is great if you have a few relatively small hotspots, but in my experience it's difficult to use it to get significant gains from a Tornado app. I would generally go straight to adding more machines once the low-hanging optimization fruit has been picked. – Ben Darnell Apr 16 '16 at 15:18
Excellent! Cheers Ben. – johhny B Apr 16 '16 at 15:24

Tornado on PyPy

1 Answers1