Python FTP get file from as400

Question

I have a simple code (using Flask-SocketIO and ftplib) for downloading csv file from as400 machine, but I got an exception which I do not understand...

Do you know what the problem is?

The code I have:

try:
    localFile = open(localDir + '/' + iso + '/' + output, 'w')
except:
    emit('update-status', (iso, 'Error: Couldn\'t create file ' + localDir + '/' + iso + '/' + output + ' !'))
    return

print('RETR ' + as400Dir + '/' + output)
try:
    ftp.retrlines('RETR ' + as400Dir + '/' + output, localFile.write)
except ftplib.all_errors as e:
# except Exception as e:
    emit('update-status', (iso, 'Error: Couldn\'t download file ' + as400Dir + "/" + output + ' ! -> ' + str(e)))
    return

localFile.close()

Console output:

RETR /xreff/pgmref.csv
Exception in thread Thread-5:
Traceback (most recent call last):
  File "D:\LocalData\xxx\Install\Python35-32\lib\threading.py", line 914, in _bootstrap_inner
    self.run()
  File "D:\LocalData\xxx\Install\Python35-32\lib\threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "D:\LocalData\xxx\Install\Python35-32\lib\site-packages\socketio\server.py", line 452, in _handle_event_internal
    r = server._trigger_event(data[0], namespace, sid, *data[1:])
  File "D:\LocalData\xxx\Install\Python35-32\lib\site-packages\socketio\server.py", line 481, in _trigger_event
    return self.handlers[namespace][event](*args)
  File "D:\LocalData\xxx\Install\Python35-32\lib\site-packages\flask_socketio\__init__.py", line 236, in _handler
    *args)
  File "D:\LocalData\xxx\Install\Python35-32\lib\site-packages\flask_socketio\__init__.py", line 618, in _handle_event
    ret = handler(*args)
  File "./index.py", line 453, in updateDatabase
    downloadResourceFiles(iso, user, password, xreff, gafint, sugar)
  File "./index.py", line 545, in downloadResourceFiles
    downloadFiles()
  File "./index.py", line 523, in downloadFiles
    ftp.retrlines('RETR ' + as400Dir + '/' + output, localFile.write)
  File "D:\LocalData\xxx\Install\Python35-32\lib\ftplib.py", line 467, in retrlines
    with self.transfercmd(cmd) as conn, \
AttributeError: __exit__

And if I try except Exception as e instead of except ftplib.all_errors as e I get:

RETR /xreff/pgmref.csv
emitting event "update-status" to b563563a763e48fbba2d394ae8c871e7 [/]
b563563a763e48fbba2d394ae8c871e7: Sending packet MESSAGE data 2["update-status","NL","Error: Couldn't download file /xreff/pgmref.csv ! -> __exit__"]

How do you create the ``ftp`` object? Also, what Python version are you using? — Irmen de Jong, Aug 11 '17 at 12:42
Python 3.5.1. The ftp object is created without an issue and works with previous calls **ftp = ftplib.FTP(ip)** — KissLick, Aug 11 '17 at 12:56
I don't know what role flask-socketio plays here but can you try the ftp code separated from Flask? The error is a weird one and comes from within the ``ftplib`` itself where it cannot use the underlying socket as a context manager because it's not providing the proper ``__exit__`` method. That's quite weird — Irmen de Jong, Aug 11 '17 at 13:42
But other methods using the ftp works just fine: **ftp.cwd('/')** or **ftp.sendcmd("RCMD CRTDIR DIR('" + as400Dir + "')")** — KissLick, Aug 11 '17 at 13:48
Yes, so? The problem occurs once you call retrlines. As I explained, it attempts to use a context manager that fails. I guess the other methods don't do this so they work. The question is what in your code triggers the problem in retrlines... Can you successfully use it to get a file from a different location? (I ask because I find it hard to believe there's a bug in the ftplib module from the standard library) — Irmen de Jong, Aug 11 '17 at 13:56
Yeah, I bypassed all the Flask-SocketIO stuff and it works... So the problem is interference between Flask-SocketIO and ftplib. btw. it works with only Flask (but without Flask-SocketIO). *EDIT: I am gonna raise an issue at the GitHub page of Flask-SocketIO..* — KissLick, Aug 11 '17 at 14:22
Sounds like a good idea. I don't know how to help you further, sorry. (as I'm unfamiliar with flask-socketIO) — Irmen de Jong, Aug 11 '17 at 14:48
Are you using eventlet or gevent along Flask-SocketIO? This is probably a bug in their implementation of sockets. — Miguel Grinberg, Aug 11 '17 at 19:06

Miguel Grinberg · Accepted Answer · 2017-08-11T22:04:03.870

I was able to reproduce your problem. This seems to be a bug in eventlet. You did not say if you are using eventlet in combination with Flask-SocketIO, but given that this is what the docs recommend I assume you were.

Here is a simple eventlet script that fails in the same way as ftplib does:

import eventlet
eventlet.monkey_patch()

import socket

with socket.create_connection(('google.com', 80)) as s:
    pass

Output of running this script:

$ python test.py
Traceback (most recent call last):
  File "test.py", line 6, in <module>
    with socket.create_connection(('google.com', 80)) as s:
AttributeError: __enter__

The problem is that eventlet's socket class does not have the __exit__ method that enables it to be used as a context manager. I reported an issue to the eventlet project: https://github.com/eventlet/eventlet/issues/430

Note that this error only occurs in Python 3. The context manager support for socket.create_connection() does not exist in Python 2, so in that version the ftplib implementation of retrlines() is done in a different way.

@KissLick the bug fix I proposed was accepted into eventlet, so now you have to wait for a new release and you will be all set. — Miguel Grinberg, Aug 13 '17 at 02:52

Python FTP get file from as400

1 Answers1