Cloud Tasks client ignores retry configuration

Question

Basically what the title says. The API and client docs state that a retry can be passed to create_task:

        retry (Optional[google.api_core.retry.Retry]):  A retry object used
            to retry requests. If ``None`` is specified, requests will
            be retried using a default configuration.

But this simply doesn't work. Passing a Retry instance does nothing and the queue-level settings are still used. For example:

from google.api_core.retry import Retry
from google.cloud.tasks_v2 import CloudTasksClient

client = CloudTasksClient()
retry = Retry(predicate=lambda _: False)
client.create_task('/foo', retry=retry)

This should create a task that is not retry. I've tried all sorts of different configurations and every time it just uses whatever settings are set on the queue.

Same problem. Same solution attempt. Same result. – Braden Holt Dec 09 '21 at 18:54 — Braden Holt, Dec 09 '21 at 18:54

score 0 · Answer 1 · answered Sep 18 '19 at 18:23

0

You can pass a custom predicate to retry on different exceptions. There is no formal indication that this parameter prevents retrying. You may check the Retry page for details.

answered Sep 18 '19 at 18:23

George

1,488
1
10
13

Okay then maybe that was a bad example. None of the retry parameters have any effect at a per-task level. – Liam Meck Sep 18 '19 at 19:41
Available parameters are: predicate (Callable[Exception]), initial (float), maximum (float), multiplier (float), deadline (float). Which ones did you modify? How did you measure the effect of your modification? How did you determine that queue-level settings are used? What would you like to achieve by modifying one, or more of the parameters on this list? – George Sep 20 '19 at 18:49

score 0 · Accepted Answer · answered Sep 23 '19 at 13:50

0

Google Cloud Support has confirmed that task-level retries are not currently supported. The documentation for this client library is incorrect. A feature request exists here https://issuetracker.google.com/issues/141314105.

answered Sep 23 '19 at 13:50

Liam Meck

21
2

score 0 · Answer 3 · answered Nov 13 '19 at 22:26

Task-level retry parameters are available in the Google App Engine bundled service for task queuing, Task Queues. If your app is on GAE, which I'm guessing it is since your question is tagged with google-app-engine, you could switch from Cloud Tasks to GAE Task Queues.

Of course, if your app relies on something that is exclusive to Cloud Tasks like the beta HTTP endpoints, the bundled service won't work (see the list of new features, and don't worry about the "List Queues command" since you can always see that in the configuration you would use in the bundled service). Barring that, here are some things to consider before switching to Task Queues.

Considerations

Supplier preference - Google seems to be preferring Cloud Tasks. From the push queues migration guide intro: "Cloud Tasks is now the preferred way of working with App Engine push queues"
Lock in - even if your app is on GAE, moving your queue solution to the GAE bundled one increases your "lock in" to GAE hosting (i.e. it makes it even harder for you to leave GAE if you ever want to change where you run your app, because you'll lose your task queue solution and have to deal with that in addition to dealing with new hosting)
Queues by retry - the GAE Task Queues to Cloud Tasks migration guide section Retrying failed tasks suggests creating a dedicated queue for each set of retry parameters, and then enqueuing tasks accordingly. This might be a suitable way to continue using Cloud Tasks

Cloud Tasks client ignores retry configuration

3 Answers3