Keep the task in the queue even after maximum number of retry limits in google task queue

Question

I am using google task queues and I am setting task_retry_limit on the queue.

The default behavior is the task is removed from task queue in the following cases :

1) when the task is executed successfully or

2) when the task reaches the maximum number of retry attempts set.

In my use case, I have a problem with the second case. I want to keep the task in the task queue even after maximum number of retries

(I don't want to retry the task after task_retry_limit but I want to keep it in the task queue so that I can run it manually later)

Is there parameter in Queue.yaml which drives this?

I know that a workaround for this would be to set a moderate task_age_limit, but I don't want the task to keep retrying.

Answer 1

No, the task queues aren't presently designed to keep around tasks which reached their maximum number of retries.

I see 2 options you could try, from inside your task code when you detect it will fail on the final task retry:

• create some sort of FailedTask datastore entities with all the info/parameters required to re-create and enqueue copies of the original failing tasks later on, under manual triggers
• re-queue the task on a different queue, configured with an extremely long time between retries - long enough to not actually be retried until the moment you get to trigger them manually (you can do that for any task still pending, in any queue, at any time).

Somehow related: handling failure after maximum number of retries in google app engine task queues