3

I have a service (deployed on ECS) that needs to listen and react to inserts on DynamoDB table.

Currently I am reading the table every second, but I would like to lower the latency. Is there a way I can subscribe to changes in DynamoDB and be notified (via websockets, long polling or similar mechanism) when some record is inserted?

I have checked out DynamoDB Streams, but I couldn't find a way to subscribe to the changes there either. There is a JS project that does something similar, but it has a "interval to poll" setting...

Is it possible to listen for changes without polling? If yes, how?

johndodo
  • 17,247
  • 15
  • 96
  • 113

2 Answers2

4

It seems it is not possible. This video explains the concepts very nicely, and it turns out that even Lambda doesn't really get "triggered" - it polls the DynamoDB streams every 250 ms.

If a delay of 1/4 s is not a problem, one could setup events to traverse like this: 

DynamoDB => DynamoDB Streams => Lambda => IOT => your app (via MQTT client or websockets)

Seems a bit convoluted though. Still, I would love to be proven wrong on this, so please comment if I'm missing something.

johndodo
  • 17,247
  • 15
  • 96
  • 113
1

This is exactly what DynamoDB Streams is for. Have you tried out this tutorial to see if it gets you where you want to be as an example? Yes, it is in Lambda, but it could just as easily be deployed in a script somewhere else like you are trying to do.

NoSQLKnowHow
  • 4,449
  • 23
  • 35
  • 1
    Thank you for the answer, this is what I would have thought - however afaict Lambda is the only way to get notifications from DynamoDB Streams. I have added an answer with what I think is the current state of affairs. – johndodo Oct 24 '19 at 12:17