GDrive gives 500 Error on Upload, Doesn't return object

Question

I have a google upload manager that works fine most of the time, but when uploading a lot of files, it cuts out part way and returns a 500 Internal Server error (always on the same file).

media_body = MediaFileUpload(filepath, mimetype=mimeType_in, resumable=True)

if create == True:   # CREATE 
    result = self.service.files().create(
                                            body=meta,
                                            media_body=media_body).execute()
else:   # REPLACE
    result = self.service.files().update(
                                        body=meta,
                                        media_body=media_body,
                                        fileId=fileID).execute()

The issue is that when this error occurs, it is an exception that gets thrown, not anything that gets stored in result (in fact, result won't exist). Thus, I can't get any information on it. I think it might have to do with a file being too big or something, but I can't resume because I have no info. Thoughts?

score 0 · Answer 1 · answered Aug 14 '18 at 12:41

In the Drive API documentation, it was stated that "500: Backend Error" is an unexpected error occurred while processing the request.

Suggested action: Use exponential backoff, include a check before retrying non-idempotent requests.

Same idea was implied in this SO post.

Exponential backoff is a standard error handling strategy for network applications in which the client periodically retries a failed request over an increasing amount of time. Exponential backoff may be a good strategy for handling those errors.

This SO post could also help about uploading huge file.

It was suggested to use a resumable media upload or chunked upload when uploading.

For large media files, you can use resumable media uploads to send files, which allows files to be uploaded in smaller chunks. This is especially useful if you are transferring large files, and the likelihood of a network interruption or some other transmission failure is high.

Thanks, but the issue wasn't the definition of the error, it was how to determine why since it doesn't actually return an error object. For instance, resuming media only is possible if when it breaks, an object is returned that lets you know where the break happens. Since nothing gets returned, it's impossible to continue. — Elliptica, Aug 15 '18 at 05:26

score 0 · Accepted Answer · answered Aug 15 '18 at 05:31

0

After much testing, I found out the reason for the error was a failed mimetype conversion during upload due to an encoding problem. There's no other information returned other than HTTP 500, but switching off conversion fixed the problem. So if you get a 500 Internal Server error (Windows) or SSLError: [SSL: SSLV3_ALERT_BAD_RECORD_MAC] sslv3 alert bad record mac (on a Mac) and you notice it's always occurring on the same file(s), check that your file encoding supports the kind of mimetype conversion you're trying to do.

answered Aug 15 '18 at 05:31

Elliptica

3,928
3
37
68

How to figure out about is the mimetype conversion?, I have been uploading a file for a few days succesfully, each time this file is getting bigger. Today it reached a size that i can't upload, and i constantly such 500 error without any explanation. If i chunk it whatever the smaller size it happens something interesting - every chunk request succeeds except the last one. – miguelfg Nov 20 '18 at 11:41
Unfortunately, there isn't any information returned to tell you it's a bad conversion, the only way I was able to find it was by testing. If you notice it's always the same filetype that fails (even though it's a different file) you might have a conversion problem. – Elliptica Nov 27 '18 at 02:09

GDrive gives 500 Error on Upload, Doesn't return object

2 Answers2