Google's documentation seems to indicate that this is an internal error of theirs, and not a specific error that you can fix. They suggest using exponential backoff, which is basically re-attempting the function at increasing intervals.
For example, the function fails. Wait 2 seconds and try again. If that fails, wait 4 seconds. Then 8 seconds, 16, 32 etc. The bigger gaps mean that you're giving more and more time for the service to right itself. Though depending on your need you may want to cap the time eventually so that it waits a maximum of 10 minutes before stopping.
The retrying
package has a very good set up for this. You can just from retrying import retry
and then use retry
as a decorator on any function that should be re-attempted. Here's an example of mine:
@retry(wait_exponential_multiplier=1000, wait_exponential_max=60*1000, stop_max_delay=10*60*1000)
def find_file(name, parent=''):
...
To use the decorator you just need to put @retry
before the function declaration. You could just use retry()
but there are optional parameters you can pass to adjust how the timing works. I use wait_exponential_multiplier
to adjust the increase of waiting time between tries. wait_exponential_max
is the maximum time it can spend waiting between attempts. And stop_max_delay
is the time it will spend retrying before it raises the exception. All their values are in milliseconds.