0

I'm trying to create an asynchronous download of messages from my e-mail by Gevent and IMAP. I use Python 2.7.8 and Gevent 1.1.1.

self.connection = imaplib.IMAP4_SSL(self.hostname, self.port, **ssl_files)

...

rv, data = self.connection.search(None, "ALL")
numbers = [number for number in data[0].split()]

threads = [gevent.spawn(self._get_message, message) for message in numbers]
gevent.joinall(threads)

Function _get_message have this code:

def _get_message(self, message_number):
    logging.info("Message {0} called!".format(message_number))
    rv, data = self.connection.fetch(message_number, '(RFC822)')
    if not self.is_ok(rv): return
    message = email.message_from_string(data[0][1])

But every time I have this error:

Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\gevent\greenlet.py", line 534, in run
    result = self._run(*self.args, **self.kwargs)
  File "C:/Users/Eugene/PycharmProjects/Extract Email by Criteria/imap_email.py", line 108, in _get_message
    rv, data = self.connection.fetch(message_number, '(RFC822)')
  File "C:\Python27\lib\imaplib.py", line 456, in fetch
    typ, dat = self._simple_command(name, message_set, message_parts)
  File "C:\Python27\lib\imaplib.py", line 1088, in _simple_command
    return self._command_complete(name, self._command(name, *args))
  File "C:\Python27\lib\imaplib.py", line 910, in _command_complete
    typ, data = self._get_tagged_response(tag)
  File "C:\Python27\lib\imaplib.py", line 1017, in _get_tagged_response
    self._get_response()
  File "C:\Python27\lib\imaplib.py", line 929, in _get_response
    resp = self._get_line()
  File "C:\Python27\lib\imaplib.py", line 1027, in _get_line
    line = self.readline()
  File "C:\Python27\lib\imaplib.py", line 1189, in readline
    return self.file.readline()
  File "C:\Python27\lib\socket.py", line 451, in readline
    data = self._sock.recv(self._rbufsize)
  File "C:\Python27\lib\site-packages\gevent\_sslgte279.py", line 455, in recv
    return self.read(buflen)
  File "C:\Python27\lib\site-packages\gevent\_sslgte279.py", line 314, in read
    self._wait(self._read_event, timeout_exc=_SSLErrorReadTimeout)
  File "C:\Python27\lib\site-packages\gevent\_socket2.py", line 173, in _wait
    raise _socketcommon.ConcurrentObjectUseError('This socket is already used by another greenlet: %r' % (watcher.callback, ))
ConcurrentObjectUseError: This socket is already used by another greenlet: <bound method Waiter.switch of <gevent.hub.Waiter object at 0x02B1C558>>
<Greenlet at 0x2afe710: <bound method EmailImapConnection._get_message of <__main__.EmailImapConnection object at 0x02A63C50>>(4)> failed with ConcurrentObjectUseError

Code on Github Gist: https://gist.github.com/gitex/0d9385fa3ab1d98f30713f6eeeab454f

I have monkey.patch_all() in the beginning. Only works the first socket, the rest are locked. I assume that the socket is blocked inside IMAP. How can I solve this problem?

JRazor
  • 2,707
  • 18
  • 27
  • 1
    imaplib is not a multithreading safe library: there is only one socket, and imaplib works in a blocking manner. – Max Apr 25 '16 at 13:19
  • 1
    Also, the imap protocol is not capable of returning more than one message at a time. You'd need to keep multiple connections and pool them. – Max Apr 25 '16 at 13:20
  • @Max, thanks Max, you very help me. Now everything works fine! – JRazor May 02 '16 at 14:20

0 Answers0