3

In my code (https://github.com/chembl/chembl_webresource_client/blob/master/chembl_webresource_client/web_resource.py#L98)

I'm calling grequests.map:

ret = grequests.map(rs, size=50)

with rs as list of URLs. If the size of that list is larger than some value (i my case 300) I'm getting:

File "/Library/Python/2.7/site-packages/gevent/hub.py", line 331, in switch
      return greenlet.switch(self)
gevent.hub.LoopExit: This operation would block forever

regardless of size parameter (i tried different values between 10 and 50 as well as ignoring this argument completely).

This happens only on macOS. In Ubuntu any size of URLs list works.

I'll appreciate any help as this is a showstopper for my small library.

mnowotka
  • 16,430
  • 18
  • 88
  • 134
  • Which gevent and greenlet version do you use? ``python -c "import gevent; print gevent.__version__"`` , ``python -c "import greenlet; print greenlet.__version__"`` – reclosedev Apr 16 '14 at 18:37
  • @reclosedev 1.0, 0.4.2 but please note, that is what you get install by default after typing 'pip install grequests'. – mnowotka Apr 16 '14 at 18:47

1 Answers1

0

perhaps you're hitting this bug: https://github.com/gevent/gevent/issues/423

will be fixed in 1.0.1.

can you try master branch?

Denis
  • 3,760
  • 25
  • 22
  • It turned out that each request was executed in separate session. This was clearly not what I wanted so after solving this and sharing one session between all requests, the code is faster and doesn't give deadlocks anymore. On the other hand executing each requests in separate session should cause any problems (appart from being extremely inefficient) so thank you for the info. – mnowotka May 01 '14 at 11:00
  • @mnowotka do you have a code example as maybe that could help clear things up – jmoz Dec 03 '14 at 20:50