-1

I am pushing 5 messages to the SQS and expecting that my lambda should get those 5 messages and just log, when i trigger the function I see that the publisher lambda is pushing 5 messages to the sqs but the consumer lambda is not getting those 5 messages instead it is getting only one. Any idea why?

# publisher lambda configuration
fetchUserDetails:
    handler: FetchUserDetails/index.fetchUserDetails
    timeout: 900
    package:
      individually: true
      artifact: "./dist/FetchUserDetails.zip"
    reservedConcurrency: 175
    environment:
      SEND_EMAIL_SQS_URL: ${self:custom.EMAILING_SQS_URL}

# consumer lambda configuration
sendEmails:
    handler: SendEmails/index.sendEmails
    timeout: 30
    package:
      individually: true
      artifact: "./dist/SendEmails.zip"
    events:
      - sqs:
          arn:
            Fn::GetAtt:
              - SendEmailSQS
              - Arn
          batchSize: 1

# SQS configuration
SendEmailSQS:
      Type: "AWS::SQS::Queue"
      Properties:
        QueueName: ${self:custom.EMAILING_SQS_NAME}
        FifoQueue: true
        VisibilityTimeout: 45
        ContentBasedDeduplication: true
        RedrivePolicy:
          deadLetterTargetArn:
            Fn::GetAtt:
              - SendEmailDlq
              - Arn
          maxReceiveCount: 15

# publisher lambda code
const fetchUserDetails = async (event, context, callback) => {
  console.log("Input to the function-", event);

  /* TODO: 1. fetch data applying all the where clauses coming in the input
   *   2. push each row to the SQS */

  const dummyData = [
    {
      user_id: "1001",
      name: "Jon Doe",
      email_id: "test1@test.com",
      booking_id: "1"
    },
    {
      user_id: "1002",
      name: "Jon Doe",
      email_id: "test2@test.com",
      booking_id: "2"
    },
    {
      user_id: "1003",
      name: "Jon Doe",
      email_id: "test3@test.com",
      booking_id: "3"
    },
    {
      user_id: "1004",
      name: "Jon Doe",
      email_id: "test4@test.com",
      booking_id: "4"
    },
    {
      user_id: "1005",
      name: "Jon Doe",
      email_id: "test5@test.com",
      booking_id: "5"
    }
  ];

  try {
    for (const user of dummyData) {
      const params = {
        MessageGroupId: uuid.v4(),
        MessageAttributes: {
          data: {
            DataType: "String",
            StringValue: JSON.stringify(user)
          }
        },
        MessageBody: "Publish messages to send mailer lambda",
        QueueUrl:
          "https://sqs.ap-southeast-1.amazonaws.com/344269040775/emailing-sqs-dev.fifo"
      };
      console.log("params-", params);
      const response = await sqs.sendMessage(params).promise();
      console.log("resp-", response);
    }
    return "Triggered the SQS queue to publish messages to send mailer lambda";
  } catch (e) {
    console.error("Error while pushing messages to the queue");
    callback(e);
  }
};

# consumer lambda code, just some logs
const sendEmails = async event => {
  console.log("Input to the function-", event);

  const allRecords = event.Records;
  const userData = event.Records[0];
  const userDataBody = JSON.parse(userData.messageAttributes.data.stringValue);

  console.log("records-", allRecords);
  console.log("userData-", userData);
  console.log("userDataBody-", userDataBody);
  console.log("stringified log-", JSON.stringify(event));
};

# permissions lambda has
- Effect: "Allow"
  Action:
    - "sqs:SendMessage"
    - "sqs:GetQueueUrl"
  Resource:
    - !GetAtt SendEmailSQS.Arn
    - !GetAtt SendEmailDlq.Arn
John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Abhishek wagh
  • 1
  • 2
  • "I'm trying to text my friend but it looks like he's not getting my messages - any ideas why?" - well are you connected? is your SIM active? are they using their SIM card? My point is: we need more details including Lambda code, SQS setup, IAM permissions etc. – Ermiya Eskandary Oct 20 '21 at 11:54
  • Hi Abhishek, you may provide SQS configuration information like rate limit and batching to get a resolution – Alpesh Patil Oct 20 '21 at 12:00
  • @ErmiyaEskandary Sorry for the less information, I have edited the question. – Abhishek wagh Oct 20 '21 at 13:07
  • @Abhishekwagh I dont see IAM permissions part here. In aws the services are bound with access permissions (IAM). I think it must be permission issue. – Sangam Belose Oct 20 '21 at 13:10

1 Answers1

0

Your consumer is only looking at one record:

const userData = event.Records[0];

It should loop through all Records and process their messages, rather than only looking at Records[0].

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470