AWS SQS Delete Batch Message, Error: TooManyEntriesInBatchRequest

Question

I am trying to delete AWS SQS Batch messages at a time, it deletes first 10 messages of array chunk but not deleting 2nd chunk, throw error

AWS.SimpleQueueService.TooManyEntriesInBatchRequest (client): Maximum number of entries per request are 10. You have sent 12.

I am following these steps.

I have array of 12 messages, and converted them into chunks like this

$chunks = array_chunk($entries, 10, true); // make chunks of 10 items per array

Chunk 1

Chunk 2

Array
(
    [10] => Array
        (
            [Id] => 1f2f5165-56a1-4df4-a70a-d22869b408ec
            [ReceiptHandle] => AQEByG1UiIz/+Uqlb3R5gkfVqOzNrWPYq8YEc3XluoqYJpVQD13TcpeFAPNU4C0NybzsKtRLLQfsM58dUehrlGP5JmU9H+ROgAGDF5R6qVNyot+dJtRKmUJdmt2A2r6NkiYMg7hgrU4+upRMhTlOAY/Ni5HGOvJbhvR/upHKSVe5gWoabQTG6Puvvg8L/pamET+PcImDxQPgSNzLc9L/8SwhTwYyqdYG4CwhymelqiAtI7siHpyBBmC24JyX6IWL/3pJenI0yBZhREdNsd2hHLcZAWoCrhq4EFsneAgcnm9UWOCuorceCOQHU94trl38z9tbXXbpEjI5ZVWlrhc6Abx6vknegjIdAoR2gLwkvUdSTHkV4QxoFZOKjLoYJBeV2ugaHgJi/52/DdL9HNQoQziwZg==
        )

    [11] => Array
        (
            [Id] => 726b8b47-be1f-4aea-aa30-f24d08361d52
            [ReceiptHandle] => AQEBcWCMm29Cic5pJH+1I0ppsiQxjQw9Q0znj26PZ2P1YQqu5tMvTsCX6t63JgoYvxzH34Ss6oXxzuXe8vXiIVoj7kSFiZdBnPVq+tcEoEFnCfBOtg2WQVONWlN5JYulAIewxXbQS2JZJjIu4HWHHzhS0oRw62OMz325/pbcuaW2a1mfU+Zs7bF9eK6AOCl4RJ0VBACSodpNJtlyqOxnS7C6zkuuVERGuQIxhFFTx44pAHzvQ3+WklGkY05QNNdxUaGzvQGhqzlwFiPwhBi6Cg22PgrSwnSt8Aef1azykibngUP035LOi1ig6Tc5jrWFX39MN+UYt6AVBTELxv4uTIi1VFJue8f8mRxJSL4H+YQH0BBy/zxPAJZANK4E066jbxtgffk8ktJcZOGBCUuFeFSrVA==
        )

)

Then Loop through the Entries and making delete action

//loop through and delete one by one
foreach($chunks as $k=>$entry_chunk){

    $result = $this->Sqs->deleteMessageBatch([
        'Entries' => $entry_chunk,
        'QueueUrl' => $queueUrl
    ]);
}

This deletes the 1st Chunks (of 10 items) while 2nd comes into, it give me error

AWS.SimpleQueueService.TooManyEntriesInBatchRequest (client): Maximum number of entries per request are 10. You have sent 12.

How I can delete multiple messages at a time?

How is this connected to [tag:batch-file]?? – aschipfl Mar 31 '16 at 11:54 — aschipfl, Mar 31 '16 at 11:54
sorry, Removed the batch file tag, not related to this – Qazi Mar 31 '16 at 11:58 — Qazi, Mar 31 '16 at 11:58

score 1 · Accepted Answer · answered Mar 31 '16 at 18:08

1

$chunks = array_chunk($entries, 10, true);

That third argument seems wrong, because of this, in the second chunk:

 [10] => Array

The 0th element of the second chunk shouldn't have index 10, should it? Should be 0. That seems to imply there are 10 undefined elements preceding it, so you're still apparently trying to delete 12 messages in one batch... 10 invalid, undefined ones and the 2 remaining ones. Because of this error, SQS may not be getting around to telling you that the first 10 messages don't even make sense, because you've exceeded the maximum messages per request -- which is 10 -- and the request is therefore already invalid.

The third argument $preserve_keys is set to true, but should be false.

preserve_keys

When set to TRUE keys will be preserved. Default is FALSE which will reindex the chunk numerically

http://php.net/manual/en/function.array-chunk.php

answered Mar 31 '16 at 18:08

Michael - sqlbot

169,571
25
353
427

I already tried this, means `Key preserving = false` still same error – Qazi Mar 31 '16 at 18:15
does calling `deleteMessageBatch()` method in loop, consider new object in each iteration ? means My code is correct or not ? – Qazi Mar 31 '16 at 18:18
your suggestion somehow worked, but now the problem is, eg: I have `36` messages and it manages in `4 chunks`, So, when I loop each chunk, sometime it deletes `15` messages sometime `19` means it not deleting all chunks. – Qazi Apr 01 '16 at 08:19
Honestly, I don't really use php -- I just googled to see how `array_chunk()` worked, and it seemed as if you were doing it wrong. Dumping and examining the values of $k, $entry_chunk, and $result *should* help you identify why it is still not working the way you expect. – Michael - sqlbot Apr 02 '16 at 14:45
can you refer this question to any Known person ? I am also not able to find any help regarding this issue. the confusing thing is, some time it deletes some of items from 2nd chunk while looping. – Qazi Apr 02 '16 at 14:53
As I mentioned in the previous comment, if you will properly capture and examine the contents of those variables, you should be able to identify the reason for what you are seeing. I am not a representative of Amazon Web Services, so referring the question is not something I can assist with. Additionally, Stack Overflow is a Q&A site, not a code debugging or consulting service. – Michael - sqlbot Apr 02 '16 at 21:30

AWS SQS Delete Batch Message, Error: TooManyEntriesInBatchRequest

1 Answers1