Failed to put record in kinesis by lambda

Question

Since I am new to lambda and kinesis so apologies in advance.

I have an SQS subscribed to SNS. A lambda polls the message from SQS and publish it to kinesis after XYZ processing.

I want to know if putting record to kinesis by lambda fails what is retry mechanism for it ?

Suppose it retries multiply by code retry strategy and again fails. Since lambda has not been able to put it to kinesis after multiple retries do we have any mechanism to get notified about failure of putting records in kinesis? Will it be considered as lambda not able to process the record, put it to SQS DLQ?

How often do it fails (e.g pushing a record to an SQS subscribed to an SNS have almost negligible failures)?

Can you elaborate more on why a lambda fails to put record in a kinesis stream? A very basic way would be wrapping it inside `try:...except:...` and either retrying or putting that into SQS DLQ. — Amey Dahale, Jun 17 '18 at 12:27
It could be any internal error that lambda wasn't able to put record in kinesis. The concern is we want to be notified that some critical record has not been put to kinesis for further processing. Just take a look to the record.What is the way to get notified? — SAA, Jun 17 '18 at 12:38
Say for example I want to be notified via email. Then how can it be achieved? — SAA, Jun 17 '18 at 12:46

score 0 · Accepted Answer · answered Jun 17 '18 at 12:57

Assumptions: An aws lambda function is trying to add a record in aws kinesis stream.

import boto3


kinesis_client = boto3.resource
attempt = 0

while attempt < 5:
  attempt += 1
  try:
    # trying to add a record in a kniesis stream
    response = kinesis_client.put_record(
        StreamName='some_stream_1',
        Data=data,
        PartitionKey='sone_key_123',
    )
  except Exception as err:
    if attempt < 5:
      pass
    else:
      # TODO send an email notification using aws SES
      ses_client.send_email()

Failed to put record in kinesis by lambda

1 Answers1