Can't set timeout longer than 30s for Google Cloud/App Engine tasks

Question

I have a task I am running with Google App Engine/Cloud Tasks (as documented here) which uses Python 3.7 and could in some cases take up to a few hours to complete.

As such, I am trying to extend the default timeout length for the task.

Consulting the documentation for the Python CloudTasksClient library, its shown that the create_task method has a timeout parameter which is a float. What's not explained is what the default value is, what unit (e.g. seconds, minutes) it is, or what the maximum it can be set to is.

I tried setting timeout to 86400 (aka 24 hours) and got the following error:

The request deadline is 2023-05-25T11:49:43.227120541-07:00. The deadline cannot be more than 30s in the future.

This doesn't make any sense to me — why should a task be limited to taking longer than 30 seconds? We use manual scaling in the Standard environment, so according to this in the docs the timeout should be extendable up to 24 hours.

Thanks for the help in advance!

@yedpodtrzitko Code is pretty much exactly the same as this example: https://cloud.google.com/tasks/docs/creating-appengine-tasks — element119, May 25 '23 at 01:35
Since it's a float, try ```86400.0```. What you currently have might be interpreted as an integer by the library — NoCommandLine, May 25 '23 at 04:16

score 0 · Answer 1 · answered May 25 '23 at 00:16

0

The deadline varies for different targets according to the docs:

for HTTP target it's 30 minutes
for AppEngine target it's either 60 mins (Flex) or 24 hours (Standard)

So now it depends what the target of your task is. If it's HTTP, maybe the value 30 is correct, but they got the units in the error message wrong (seconds instead of minutes)?

Please provide more details about what how does your task look exactly.

answered May 25 '23 at 00:16

yedpodtrzitko

9,035
2
40
42

The target is an AppEngine worker, and as I mentioned we are using the standard environment. We're creating an "App Engine HTTP request" pretty much exactly as documented here: https://cloud.google.com/tasks/docs/creating-appengine-tasks – element119 May 25 '23 at 01:00

NoCommandLine · Accepted Answer · 2023-06-01T01:54:30.513

Maybe this will help

timeout.py (google/api_core/timeout.py) has the following piece of code

 class TimeToDeadlineTimeout(object):
 """A decorator that decreases timeout set for an RPC based on how much time
 has left till its deadline. The deadline is calculated as
 ``now + initial_timeout`` when this decorator is first called for an rpc.

 In other words this decorator implements deadline semantics in terms of a
 sequence of decreasing timeouts t0 > t1 > t2 ... tn >= 0.

 Args:
     timeout (Optional[float]): the timeout (in seconds) to applied to the
         wrapped function. If `None`, the target function is expected to
         never timeout.
 """

From the above, a possible solution to your problem (providing enough time for your long running task to end) would be to supply a value of None to the timeout. I tested with a value of None and didn't receive an error i.e. the following code was successful

client.create_task(parent=parent, task=task, timeout=None)
If you must provide a value for the timeout, it looks like it can't be more than 30 seconds (this is the default value as seen in _DEFAULT_MAXIMUM_TIMEOUT = 30.0 # seconds which can be found in timeout.py

Thanks, tried the None argument and that worked, at least for tasks up to 10 minutes! Additionally I'm realizing i had to update the instance class and scaling type for the worker — element119, Jun 02 '23 at 04:50

Can't set timeout longer than 30s for Google Cloud/App Engine tasks

2 Answers2