5

The Google Drive REST API sometimes returns a 500: Internal Server Error when attempting to upload a file. Most of these errors actually correspond to a successful upload. We retry the upload as per Google's recommendations only to see duplicates later on.

What is the recommended way of handing these errors?

Rajiv
  • 2,587
  • 2
  • 22
  • 33

2 Answers2

0

Google's documentation seems to indicate that this is an internal error of theirs, and not a specific error that you can fix. They suggest using exponential backoff, which is basically re-attempting the function at increasing intervals.

For example, the function fails. Wait 2 seconds and try again. If that fails, wait 4 seconds. Then 8 seconds, 16, 32 etc. The bigger gaps mean that you're giving more and more time for the service to right itself. Though depending on your need you may want to cap the time eventually so that it waits a maximum of 10 minutes before stopping.

The retrying package has a very good set up for this. You can just from retrying import retry and then use retry as a decorator on any function that should be re-attempted. Here's an example of mine:

@retry(wait_exponential_multiplier=1000, wait_exponential_max=60*1000, stop_max_delay=10*60*1000)
def find_file(name, parent=''):
    ...

To use the decorator you just need to put @retry before the function declaration. You could just use retry() but there are optional parameters you can pass to adjust how the timing works. I use wait_exponential_multiplier to adjust the increase of waiting time between tries. wait_exponential_max is the maximum time it can spend waiting between attempts. And stop_max_delay is the time it will spend retrying before it raises the exception. All their values are in milliseconds.

SuperBiasedMan
  • 9,814
  • 10
  • 45
  • 73
-2

Standard error handling is described here: https://developers.google.com/drive/handle-errors

However, 500 errors should never happen, so please add log information, and Google can look to debug this issue for you. Thanks.

Ali Afshar
  • 40,967
  • 12
  • 95
  • 109
  • I will log the entire response when I get a 500 and reply when I find one. – Rajiv Apr 11 '13 at 21:42
  • Vote for this - same problem here http://stackoverflow.com/questions/19769000/google-drive-uploading-file-size-limit – Vlad Tsepelev Nov 04 '13 at 13:45
  • Solved for me — see my previous comment. Check your mime-type and try to upload your files with Content-Type: applicatio/octet-stream. GoogleDrive fails sometimes with file convertation. – Vlad Tsepelev Nov 21 '13 at 00:19
  • I am seeing this with uploading a CSV file and asking google to convert it. Here's the exception stack trace (am using the java sdk): http://pastebin.com/DFVvtD3J – mooreds May 23 '14 at 20:35
  • Hi @ali-afshar where can I file this additional detail? – mooreds May 23 '14 at 20:42
  • Here's my SO question about this same issue: http://stackoverflow.com/questions/23896489/google-drive-upload-returns-50x-server-errors-intermittently in case that helps anyone. – mooreds May 27 '14 at 19:25
  • 2
    `However, 500 errors should never happen` -- it certainly happens, for unexplained reasons too. The Drive docs (such as the ones you linked to) do not explain 5xx errors such as 500. But as someone who has used Java Drive SDK often, I can assure you they happen all the time. I just had it happen ~60 times back to back, while trying to either 1. rename a file or 2. insert a file – Don Cheadle Feb 03 '15 at 23:56