Using gevent monkey patching with threading makes thread work serially

Question

I am using gevent and I am monkey patching everything.
It seems like the monkey patching causes the threading to work serially.

My code:

import threading
from gevent import monkey; monkey.patch_all()

class ExampleThread(threading.Thread):
    def run(self):
        do_stuff()  # takes a few minutes to finish
        print 'finished working'

if __name__ == '__main__':
    worker = ExampleThread()
    worker.start()
    print 'this should be printed before the worker finished'

So the thread is not working as expected.
But if I remove the monkey.patch_all() it is working fine.
The problem is that I need the monkey.patch_all() for using gevent (now shown in the code above)

My solution:

I changed the

monkey.patch_all()

to

monkey.patch_all(thread=False)

so I am not patching the thread.

jcollado · Accepted Answer · 2012-02-08T12:07:51.577

33

When threads are monkey patched in gevent, they behave as coroutines. This means that you have to explicitly yield control to make it possible for other coroutines to execute.

The way to do this is call a blocking operation that has been patched (this will yield automatically) or gevent.sleep:

#!/usr/bin/env python
from gevent import monkey, sleep
monkey.patch_all()
import threading

class ExampleThread(threading.Thread):
    def run(self):
        for i in xrange(10):
            print 'working'
            sleep()

if __name__ == '__main__':
    worker = ExampleThread()
    worker.start()
    print 'this will be printed after the first call to sleep'

edited Feb 08 '12 at 12:07

answered Feb 08 '12 at 12:02

jcollado

39,419
8
102
133

i have edited my question. i cant use sleep because my work takes a few minutes, – yossi Feb 08 '12 at 12:09
3

@yossi If you have a task that takes a long time to complete and isn't possible to yield control at some point, then you need real threads instead of coroutines. In such a case, I'd say you're better off not patching threads. – jcollado Feb 08 '12 at 12:18
I found this interesting. What would be the use case of this vs the standard theading module. – Merlin Sep 10 '12 at 21:24
1

@Merlin The motivation section in [PEP342](http://www.python.org/dev/peps/pep-0342/) might be a good source for that. In summary, threads aren't really cheap and if we need a lot of them, then it's worth considering alternatives. – jcollado Sep 11 '12 at 08:02

score 0 · Answer 2 · answered Jun 07 '18 at 22:27

You can leave your Thread-based class in place if you substitute the Thread with Greenlet, for example like this:

from gevent import monkey
from gevent import Greenlet
from threading import Thread


class ThreadLikeGreenlet(Greenlet):
    def __init__(self, name=None, target=None, args=(), kwargs=()):
        super().__init__(target, *args, **dict(kwargs))
        self.name = name

def is_gevent_patched():
    return monkey.is_module_patched('threading')

if is_gevent_patched():
    Thread = ThreadLikeGreenlet  # substitute Thread with Greenlet

class ExampleThread(Thread):
    ...

it will work as you wish then.

Using gevent monkey patching with threading makes thread work serially

2 Answers2

Linked