1

I have log messages (with log timestamps) coming through logstash which get a timestamp as they enter logstash. After the message leaves logstash it is headed for SNS. I need to add a third timestamp as the message passes through SNS so that when I recieve the message I can tell where delays are, if any. There are more nodes along the way for the message but I'm only concerned with SNS for now. My question, "Specifically how do I add a timestamp to a message in SNS?".

Aaron Watkins
  • 33
  • 1
  • 8

2 Answers2

3

SNS automatically timestamps messages with the time the message was received by SNS ("published" -- not when it was delivered by SNS to the next service in the chain).

Timestamp

The time (GMT) when the notification was published.

https://docs.aws.amazon.com/sns/latest/dg/json-formats.html

If you're sending the messages to an HTTP or SQS endpoint, you also need to enable raw message delivery to access the structure containing the timestamp. Since this codes your original message into a JSON wrapper, you'll also need to JSON.parse() (or equivalent) the Message attribute if your original message is also in JSON.

Michael - sqlbot
  • 169,571
  • 25
  • 353
  • 427
0

SNS itself doesn't have the ability to modify data sent through it. But you could make a Lambda function that your SNS topic sends to.

Here is how to invoke Lambda from SNS

In Lambda you could then add the appropriate timestamp and then push forward to another SNS topic. Sure, that might add some additional latency. But depending on your scenario, that might be the best way forward.

colde
  • 3,192
  • 1
  • 15
  • 26
  • I am seeing with testing that there is a timestamp field in the metadata after leaving SNS => SQS which surrounds my original message. I assume it was SNS when the metadata was added. – Aaron Watkins May 01 '18 at 21:29