2

I am trying to download a file from an FTP server in memory, transform it to a dataframe but also return it as bytes. Code as follows:

import io
import pandas as pd
from ftplib import FTP

ftp_connection.cwd(ftp_folder)
download_file = io.BytesIO()
ftp_connection.retrbinary('RETR ' + str(file_name), download_file.write)
download_file.seek(0)
file_to_process = pd.read_csv(download_file, engine='python')

After searching on Stack Overflow, the suggestion was to just read the io stream:

download_file.read()
ValueError: I/O operation on closed file.

Not sure what to try next, without writing the file somewhere and reading it again as bytes.

Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992
DBa
  • 261
  • 1
  • 3
  • 11

2 Answers2

2

read_csv probably closes the "file". So read it before you call read_csv:

download_file.seek(0)
contents = download_file.read()
download_file.seek(0)
file_to_process = pd.read_csv(download_file, engine='python')
Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992
  • 1
    Awesome, thanks for this. I was stuck for ages with a "pandas.errors.EmptyDataError: No columns to parse from file" error and the key was to seek(0) first – zylatis Apr 24 '20 at 06:05
1

I think that's easier with tentaclio package :

with tentaclio.open("ftp://user:password@host/path/name_file.csv") as reader:
    df = pd.read_csv(reader)
thomas faure
  • 11
  • 2
  • 1
    Please correct the indentation via [edit](https://stackoverflow.com/posts/64680992/edit) and consider explaining the code a little. – Ruli Nov 04 '20 at 15:29