2

I'm working on an Azure based project for some research and have been running into some issues when deleting messages from a CloudQueue instance. The code is fairly straightforward, so I'm a bit baffled as to why an exception is being thrown when I try to delete a message from the queue.

Here is the code that produces data for the queue:

foreach (var cell in scheme(cells))
{
    string id = Guid.NewGuid().ToString();
    var blob = sweepItemContainer.GetBlobReference(id);
    using (BlobStream stream = blob.OpenWrite())
    {
        BinaryFormatter bf = new BinaryFormatter();
        bf.Serialize(stream, cell);
    }
    sweepItemQueue.AddMessage(new CloudQueueMessage(id), new TimeSpan(1, 0, 0));
}

Here is the code that consumes the data from the queue:

var msgs = sweepItemsQueue.GetMessages(MsgAmt);
foreach (var msg in msgs)
{
     _handleMessage(msg, sweepItemsContainer);
     sweepItemsQueue.DeleteMessage(msg);
     mergeItemsQueue.AddMessage(new CloudQueueMessage(msg.AsString), new TimeSpan(1, 0, 0));
}

I don't see how the message cannot exist in the queue. Nothing else is mutating the queue besides other consumers. But I am assured that they cannot get the same message (so long as the timespan doesn't run out), so how is this happening?

Nicholas Mancuso
  • 11,599
  • 6
  • 45
  • 47
  • 1
    Is there a time difference between when you call the first block of code (adding messages to the queue) and the 2nd block (where you delete messages from the queue). Also what is the value for "MsgAmt" in your GetMessages() call? – Gaurav Mantri Apr 06 '11 at 03:57
  • There should be a time difference. The code is in separate roles, with the producer pushing all the messages, while 'worker roles' are consumers. MsgAmt is defined in a configuration file, but currently is 5. – Nicholas Mancuso Apr 06 '11 at 04:00
  • 1
    The reason I asked for the time difference is because when you're creating the message, you're setting message TTL to be 1 hour so the message would automatically be garbage collected after 1 hour. In the event the time difference between 1st and 2nd block of code is more than an hour, then you would not find the messages. – Gaurav Mantri Apr 06 '11 at 04:17
  • Its only a few seconds to a few minutes right now, tops. Nowhere near an hour. – Nicholas Mancuso Apr 06 '11 at 04:27
  • 1
    I forgot to ask, what is the exception thrown by your code? And you're getting exception when you call DeleteMessage(), Correct? – Gaurav Mantri Apr 06 '11 at 04:38
  • MessageDoesNotExist is being thrown by the line 'sweepItemsQueue.DeleteMessage(msg);' – Nicholas Mancuso Apr 06 '11 at 04:57

2 Answers2

2

There are two timeouts that you need to worry about, how long the message lives in the queue (which you've specified in the your .AddMessage() call and the visibility timeout that is set when you call .GetMessages() (by default this is 30 seconds, there is an overload that allows you to specify the timeout). When you call .GetMessages() all of the messages returned are invisible to other consumers for the period 'visibilityTimeout'. Once this period finishes all of the messages you haven't already deleted become visible to all other consumers.

To check if this is the problem I would try using the overload of .GetMessages() with it's maximum visibility timeout of 2 hours. If this is the problem you can fine tune this value down to a more sensible number. Another option would be to just retrieve one message at a time.

knightpfhor
  • 9,299
  • 3
  • 29
  • 42
1

Another answer from Steve Marx, basically look at the storage exception and move on. I have seen this in other frameworks too.: Steve Marx blog post

 try
 {
    q.DeleteMessage(msg);
 }
 catch (StorageClientException ex)
 {
 if (ex.ExtendedErrorInformation.ErrorCode == "MessageNotFound")
 {
     // pop receipt must be invalid
     // ignore or log (so we can tune the visibility timeout)
 }
 else
 {
    // not the error we were expecting
     throw;
 }
}
ElvisLives
  • 2,275
  • 2
  • 18
  • 24