Listening to remote AWS SQS from local using serverless

Question

I want to execute the lambda function locally , on SQS event which is on my AWS account. I have defined the required event but this not getting triggered.

How can this be achieved?

I am able to send the messages to the same queue using cron event from my local.

Here are few things I tried... but didnt work for me .


functions:
  account-data-delta-test:
    handler: functions/test/data/dataDeltatestGenerator.handler
    name: ${self:provider.stage}-account-data-delta-test
    description: account delta update  - ${self:provider.stage}-account-data-delta-test
    tags:
      Name: ${self:provider.stage}-account-data-delta-test
    # keeping 5 minute function timeout just in case large volume of data.
    timeout: 300
    events:
      - sqs:
          arn:
            Fn::GetAtt: [ testGenerationQueue, Arn ]
          batchSize: 10

----------

      Policies:
        - PolicyName: ${self:provider.stage}-test-sqs-policy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
            - Effect: Allow
              Action:
              - sqs:ReceiveMessage
              - sqs:DeleteMessage
              - sqs:GetQueueAttributes
              - sqs:ChangeMessageVisibility
              - sqs:SendMessage
              - sqs:GetQueueUrl
              - sqs:ListQueues
              Resource: "*"     

---------------

---
Resources:
  testGenerationQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: ${self:provider.stage}-account-test-queue
      VisibilityTimeout: 60
      Tags:
        -
          Key: Name
          Value: ${self:provider.stage}-account-test-queue  

-------------

const sqs = new AWS.SQS({
    region: process.env.REGION,
});


exports.handler = async (event) => {
    console.error('------------ >>>>CRON:START: Test delta Job run.', event);
    log.error('------------ >>>>CRON:START: Test delta Job run.', event);
});

What do you mean by "execute the lambda function locally"? Are you referring to `sam local`? [Invoking Functions Locally - AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-invoke.html) — John Rotenstein, Dec 21 '19 at 03:31
Try configuring an IAM user and access key with access to the SQS queue for your local machine. — Md khirul ashik, Dec 22 '19 at 05:45
@JohnRotenstein I am trying serverless local or npx sls local invoke... — Aryan, Dec 23 '19 at 15:28
@Mdkhirulashik, I have the access keys and profile configured. And I am able to send the message to the queue through the cron written in same project. But somehow sqs event defined in serverless.yml is not calling the piece of code. — Aryan, Dec 23 '19 at 22:12
Did you try to do the same operation using CLI? Lambda local utilize docker container to simulate lambda environment. It could be a permission issue for the container. — Md khirul ashik, Dec 24 '19 at 05:52

score 9 · Accepted Answer · edited May 06 '20 at 13:54

You can't trigger your local Lambda function from your remote context because they haven't nothin in common.

I suppose your goal is to test the logic of Lambda function, if so you have two options.

Option 1

A faster way could be invoke function locally using sam local invoke. In this way, you could provide this command some argument, one of those arguments is the event source (i.e. the event information that SQS will send to the Lambda as soon this is triggered).

sam local invoke -e sqs.input.json account-data-delta-test

and your sqs.input.json would look like this (generate using sam local generate-event sqs receive-message)

so you will actually test your Lambda locally.

Pros: is fast

Cons: You still have to test the trigger when you will deploy on AWS

Option 2

In a second scenario you will sacrifice the bind between a queue and Lambda. You have to trigger your function at fix interval and explicitly use the ReceiveMessage in your code.

Listening to remote AWS SQS from local using serverless

1 Answers1

Option 1

Pros: is fast

Cons: You still have to test the trigger when you will deploy on AWS

Option 2

Pro: you can read a real message from a real queue.

Con: you have to invoke function at regular interval and this is not handy.