Using FTPlib to upload a 200kb html page

Question

I'm using the following code to upload a html file to my website however when uploading it seems to be missing some data...

My content is 1930 lines with a "length" of 298872 (I guess that's how many characters)

## Login using the ftplib library and set the session as the variable ftp_session
ftp_session = ftplib.FTP('ftp.website.com','admin@website.com','password')

## Open a file to upload
html_ftp_file = open('OUTPUT/output.html','rb')
## Open a folder under in the ftp server
ftp_session.cwd("/folder")
## Send/upload the the binary code of said file to the ftp server
ftp_session.storbinary('STOR output.html', html_ftp_file )
## Close the ftp_file
html_ftp_file.close()

## Quit out of the FTP session
ftp_session.quit()

Why isn't this uploading 100% of the file? it's uploading like 98% of it...

I've been looking around and can't find out what the max character limit is or the max file size is, could this be fixed by uploading it in parts (I'm unsure how to do that)

Passive

*cmd* 'CWD /folder'
*resp* '250 OK. Current directory is /folder'
*cmd* 'TYPE A'
*resp* '200 TYPE is now ASCII'
*cmd* 'PASV'
*resp* '227 Entering Passive Mode (xxx,xxx,xxx,xxx,73,19)'
*cmd* 'STOR output.html'
*resp* '150 Accepted data connection'
*resp* '226-File successfully transferred'
*resp* '226 3.235 seconds (measured here), 48.23 Kbytes per second'
*cmd* 'QUIT'
*resp* '221-Goodbye. You uploaded 157 and downloaded 0 kbytes.'
*resp* '221 Logout.'

Active

*cmd* 'CWD /folder'
*resp* '250 OK. Current directory is /folder'
*cmd* 'TYPE A'
*resp* '200 TYPE is now ASCII'
*cmd* 'PORT xxx,xxx,xxx,xxx,203,212'
*resp* '200 PORT command successful'
*cmd* 'STOR output.html'
*resp* '150 Connecting to port 52180'
*resp* '226-File successfully transferred'
*resp* '226 4.102 seconds (measured here), 38.03 Kbytes per second'
*cmd* 'QUIT'
*resp* '221-Goodbye. You uploaded 157 and downloaded 0 kbytes.'
*resp* '221 Logout.'

MunkeyWrench · Accepted Answer · 2013-11-11T17:56:17.023

2

From your code it looks like your FTP mode is binary, but you are uploading a ASCII file (html). Try changing your FTP mode to ASCII or zip your file first (which would be a binary file), send it and then unzip at your destination.

Here is and eaxmple from http://effbot.org/librarybook/ftplib.htm

import ftp
import os

def upload(ftp, file):
    ext = os.path.splitext(file)[1]
    if ext in (".txt", ".htm", ".html"):
        ftp.storlines("STOR " + file, open(file))
    else:
        ftp.storbinary("STOR " + file, open(file, "rb"), 1024)

ftp = ftplib.FTP("ftp.fbi.gov")
ftp.login("mulder", "trustno1")

upload(ftp, "trixie.zip")
upload(ftp, "file.txt")
upload(ftp, "sightings.jpg")

edited Nov 11 '13 at 17:56

answered Nov 08 '13 at 22:12

MunkeyWrench

356
2
8

I'm unsure how to do either of those :/ – Ryflex Nov 09 '13 at 14:17
Change FTP.storbinary to FTP.storlines to switch to an ascii mode. Here are a couple of links that might help. http://docs.python.org/2/library/ftplib.html http://www.inmotionhosting.com/support/website/file-management/corrupt-file-ftp-transfer – MunkeyWrench Nov 10 '13 at 17:19
When you say cutting short do you mean file size or are you saying the actual text of the html file is truncated? – MunkeyWrench Nov 11 '13 at 17:34
both, the html source is missing the bottom like 50 lines and if I login to the websites ftp and download the out.html it's still missing the bottom section. – Ryflex Nov 11 '13 at 17:37
The local file that you are trying to upload is completely intact right? The truncation happens when you are uploading? If this is so I would suspect that there is something peculiar with your file. The storlines reads each line of the file (like readlines) and uploads and appends it to the text file. I would try changning to binary mode. It would mean an extra step of zipping your file. Here is the link to the zip commands http://docs.python.org/2/library/zipfile.html. Then change storlines to storbinary as in the example above. – MunkeyWrench Nov 11 '13 at 17:52
Changing to binarystor is to test the connection. If your script can successfully upload the binary file then we know that there is problem in the ascii mode. You could turn on passive mode. http://slacksite.com/other/ftp.html FTP.set_pasv(boolean) or you could just turn on passive mode and see if it fixes it. – MunkeyWrench Nov 11 '13 at 18:23
How would I unzip it if I upload it as a zip? I've added the debugging to OP – Ryflex Nov 12 '13 at 00:04
Yeah I sort realized that after I posted it. You would need something server side to unzip it. Could use a different Python script. After I thought about it a bit. Turning on Passive mode and staying with the ascii method will probably work best for you. Assuming passive mode solves the problem of the upload stopping prematurely. – MunkeyWrench Nov 12 '13 at 20:24
I found a possible solution. In your code you are opening the file using rb, According to http://docs.python.org/release/1.5/tut/node46.html using the b will open the file in binary mode. Try changing rb to just r in the line. html_ftp_file = open('OUTPUT/output.html','rb') – MunkeyWrench Nov 12 '13 at 20:38

score 1 · Answer 2 · answered Nov 12 '13 at 00:58

Try the following which will upload the file in binary mode. The trick is to set the type of file transfer to binary mode (TYPE I) before calling storbinary.

with open('OUTPUT/output.html','rb') as html_ftp_file:
    # ftp_session.set_pasv(1) # If you want to use passive mode for transfer.
    ftp_session.voidcmd('TYPE I')  # Set file transfer type to Image (binary)
    ftp_session.cwd("/folder")
    ftp_session.storbinary('STOR output.html', html_ftp_file)

Using FTPlib to upload a 200kb html page

Passive

Active

2 Answers2