Azure Queue - Reappearing messages when job gets too long or exception occur in execution of Job

Question

I have a web-job that gets triggered by queue messages. If the job gets too long, The message reappears on the queue and a new instance of the web-job is triggered consuming the same message. That is not meant to be.

We want that message to be deleted from the message queue once the Job is triggered with that message.

We had set MaxDequeueCount = 1, with the thought that It will try processing the message once and then message will be moved to Poison Queue. But MaxDequeueCount is not working for us. In our Job, We are using 8.7.0.0 version of Microsoft.WindowsAzure.Storage.

Please help!

JobHostConfiguration config = new JobHostConfiguration();
config.NameResolver = new QueueNameResolver();
config.Queues.BatchSize = Convert.ToInt32(ConfigurationManager.AppSettings["BatchSize"]);
config.Queues.MaxDequeueCount = Convert.ToInt32(ConfigurationManager.AppSettings["MaxDequeueCount"]);
var host = new JobHost(config);
host.RunAndBlock();

score 0 · Answer 1 · answered Dec 26 '18 at 07:07

0

I think you are hitting the message lease timeout. You should be able to extend the lease using UpdateMessage (you should be able to extend it up to 7 days according to this capabilities comparison).

Hope it helps!

answered Dec 26 '18 at 07:07

Itay Podhajcer

2,616
2
9
14

Do you mean to say we need to extend the timeout from 30 sec upto 7 days from code will solve this problem? Why MaxDequeCount set to 1 is not working ? Please Help . – Shubham Kalantri Dec 26 '18 at 08:11
Once the lease timeout is exceeded without the consumer notifying the queue that processing of the message has completed, the considers the processing a failed and then 're-queues' the message. You don't need to set it to 7, you need to fined the interval that suits the time that takes your logic to execute. – Itay Podhajcer Dec 26 '18 at 08:24

score 0 · Answer 2 · answered Dec 26 '18 at 10:01

0

It should be fixed since 2.1.0-beta1. But there is currently no released version of 2.1.0. The latest version including the fix is 2.1.0-beta4.

You could install the prerelease/beta version (if you want or can) to get rid of the issue. Installing the beta version is not an option for me in production, so you could install Microsoft.Azure.WebJobs 2.1.0 to have a try.

For more details, you could refer to this issue and this one.

answered Dec 26 '18 at 10:01

Joey Cai

18,968
1
20
30

Microsoft.Azure.WebJobs released version 2.1.0. is available.Do we need to use that version ? Is this will solved the problem of MaxDequeueCount ? – Shubham Kalantri Dec 27 '18 at 05:42
Yes, you could try to install 2.1.0 version webjob and try again. – Joey Cai Dec 27 '18 at 05:43
Okay ! Thank you. I will try with this version. – Shubham Kalantri Dec 27 '18 at 05:45
With WebJobs released version 2.1.0. , It is not working. Do we have any alternate solution for this? – Shubham Kalantri Dec 27 '18 at 07:27
What the version do you use before and could you give me an example about long message. – Joey Cai Dec 27 '18 at 07:29
We was using Microsoft.Azure.WebJobs released version 2.0.0. Example : We are saving upto 10000 records into the DB using web Job. – Shubham Kalantri Dec 27 '18 at 09:03

Azure Queue - Reappearing messages when job gets too long or exception occur in execution of Job

2 Answers2