17

Im trying to upload my files to Google Drive via REST API (resumable upload). Everything looks good (XMLHttpRequest triggers onprogress and onload events), but after it (onload triggered) Google Drive PUT request fail with 500 Internal Server Error. File not appears in my Google Drive folder. Error 500 comes in xhr.onload, not in xhr.onerror.

Same thing if im trying to upload that file via Google Drive interface. It happens sometimes, and i haven't environment with 100% reproducing.

Filetype Adobe .DNG or Canon .CR2 and filesize ~28MB.

What im doing wrong? Is it known bug or limitations to filetypes or file?

Possible reasons: filesize limitations, filetype limitations, or maybe my token is expires while my file is uploading?

UPD: Im using this uploader as is, only with cosmetic changes.

Vlad Tsepelev
  • 2,056
  • 1
  • 21
  • 32
  • 1
    It should be fairly easy to eliminate 2 of your 3 possibilities. A filetype limitation would trigger for any file size, and I would expect a 405 status. Unless you have convert=true, Drive won't care what your file type is. If your token expires, you should get a 401. So my personal guess would be it's a bug/filesize issue. This page https://support.google.com/drive/answer/37603?hl=en says the limit for non-converted files is 10GB. – pinoyyid Nov 04 '13 at 15:40
  • This is server response params:{"error":{"errors":[{"domain":"global","reason":"internalError","message":"InternalError"}],"code":500,"message":"InternalError"}} – Vlad Tsepelev Nov 06 '13 at 15:48
  • I would assume it's a Drive bug unless somebody from Google confirms otherwise. You might want to try the older docs list API which seems more reliable. – pinoyyid Nov 06 '13 at 17:28
  • I spotted this question http://stackoverflow.com/questions/14628024/how-to-continue-an-upload-after-access-token-expire?rq=1 which confirms that if your problem was access-token expiration, that you would see a 401, not a 500. – pinoyyid Nov 06 '13 at 19:37
  • Could you provide a code sample so that we can try and reproduce the error? I've uploaded 400MB videos with .Net, it might be related to the JS Client, or are you calling the Rest API directly? – smokybob Nov 08 '13 at 01:02
  • Updated post. Im using this uploader https://github.com/googledrive/cors-upload-sample – Vlad Tsepelev Nov 09 '13 at 10:59
  • 1
    Does the problem occur from another Google Drive account? Does it work with a smaller file? – Nick Nov 14 '13 at 09:51
  • @NickKarnik, for small files its OK, and repeats with another accounts. Mb problem is connected with raw photo filetypes? – Vlad Tsepelev Nov 14 '13 at 14:58
  • I think this would be related to raw photo files, because DRIVE tries to convert them. This post says that it will allow you to preview the file up to 25MB, this could be related with your troubles. https://support.google.com/drive/answer/2423485?hl=en – jmventar Nov 14 '13 at 16:46

3 Answers3

5

OK. I got it.

Files uploads successfully with Content-Type == "application/octet-stream". Looks like bug on GoogleDrive side with mime-type'd files. In this scenario my raw files (DNG, CR2, NEF etc.) stores in GoogleDrive with incorrect mime-type (as a result there is no preview for these files).

So i can't filter files by mime-type anymore.

Query string = (mimeType = 'image/x-adobe-dng' or mimeType = 'image/x-canon-cr2' or mimeType = 'image/x-nikon-nef').

I tried to filter files by keyword title, but looks like titles doesn't contain extension, but in response item titles contain extension.

Query string = (title contains '.dng' or title contains '.cr2' or title contains '.nef').

So i have to filter my files not by mime-type or by title, but by fullText keyword.

Query string = (fullText contains '.dng' or fullText contains '.cr2' or fullText contains '.nef').

Conclusion:

  1. GoogleDrive uploader checks Content-Type, even if convert option is set to false.
  2. GoogleDrive fails from time to time with this convertation.
  3. Uploader works fine with Content-Type == 'application/octet-stream'.
  4. GoogleDrive query string keyword title doesn't contain file extension, but in response titles have extension.
  5. GoogleDrive query string keyword fullText contains filename with extension (mb its not very fast for text files).
  6. To test your request you can use this tool https://developers.google.com/drive/v2/reference/files/list in the end of page.
  7. GoogleDrive API crashes from time to time with HTTP status 500 Internal Server Error:

{"error":{"errors":[{"domain":"global","reason":"backendError","message":"Backend Error"}],"code": 500,"message": "Backend Error"}}

Vlad Tsepelev
  • 2,056
  • 1
  • 21
  • 32
  • The issue is specific to types which can be thumbnailed. Very large files can cause that service to timeout, which results in the 500 error. Not sure when this will be fixed, but on the roadmap. – Steve Bazyl Feb 28 '14 at 19:03
  • @SteveBazyl, if you have questions about thumbnailing digital photo raw formats I can provide some advices for you or your team. – Vlad Tsepelev Mar 02 '14 at 22:22
  • 1
    It's not the formats that are the problem, rather an architectural constraint. There's a fix planned. – Steve Bazyl Mar 14 '14 at 23:18
  • Do you find any solution for this issue? I am facing same issue while uploading google documents file type. If I don't set the mime type the Google document file appears as zip file and not as google document. – Mayuresh Jun 12 '17 at 06:51
2

I think this would be related to raw photo files, because DRIVE tries to convert them to JPG as is done in Google+, unless you specify convert=false. You should specify in your upload API to not convert that files.

This second post says that it will allow you to preview the file up to 25MB, this size limitation could be related with your troubles. Maybe you could try to upload another photo with less than that size to ensure it's not this the limitation.

pinoyyid
  • 21,499
  • 14
  • 64
  • 115
jmventar
  • 687
  • 1
  • 11
  • 32
  • About convertation - No. I use resumable upload and convert set to false. More than that status 500 happens from time to time. About filesize - No. Sometimes big files uploads correctly, sometimes not. Only one thing is clear - it happens on big files. Small files always uploads correctly. – Vlad Tsepelev Nov 14 '13 at 18:35
  • sounds to me like you might just have spotty internet connectivity, which would be very hard to pin down in logs and errors. try the same setup at work, starbucks, etc and see if your batting average improves. – dandavis Nov 19 '13 at 20:49
0

I have also seen this 500 error with google analytics a few times and its normaly due to the fact that the server is busy. If I wait a bit and try again it works. Try reading this https://developers.google.com/drive/handle-errors#implementing_exponential_backoff This worked form me with Google Analtyics API, normaly by the time it gets to step 6 its got access again. Haven't tried it with google Drive, but then i havent tried to upload a 25m file before. But i'm betting thats your problem.

Linda Lawton - DaImTo
  • 106,405
  • 32
  • 180
  • 449
  • While this is correct, it is only a small part of the story. The various Google backends throw many different 500s, some of which succeed on a retry, many of which don't. So doing expo-backoff is sometimes the right thing to do, but sometimes it will backoff indefinitely until you eventually give up and inform the user. – pinoyyid Nov 21 '13 at 05:25
  • In my case its normaly with users with large amounts of data in google analytics quering sevral months at a time. Adding filters and stuff it seamed to take a while for the server to process it. Adding a few retryes fixed the issue. That and seting the system to loop month by month if theres to much data. – Linda Lawton - DaImTo Nov 21 '13 at 07:31