1

I am reasonably familiar with AWS, I use a stack of ASP.NET MVC and MySQL, as well as Redis for caching/messaging.

Normally to keep a count of something, I would use a MySQL table to keep a count of something, for an action on my website. I can easily execute the following from within my code:

UPDATE mycounts SET mycount = mycount + 1 WHERE id = @countId

I can guarantee that this is executed ATOMIC-ally, so I can pretty much guarantee that if the table is up that the count will go up by exactly one for each call.

My problem with this implementation is that:

  • It requires a database instance, this is expensive
  • It requires the database to be up at all times, any down time and I'm going to lose counts

What is the cheapest way to keep a count of something on the AWS platform, without having to have an RDS database? That is both fault tolerant and accurate...

Luke
  • 22,826
  • 31
  • 110
  • 193

1 Answers1

4

You could provision an absolute minimum throughput dynamoDB table and use atomic counters there. It'll be dirt cheap.

Henry
  • 1,646
  • 12
  • 28
  • I've been looking into atomic counters. My only worry with this is that it doesn't scale very well. I have a feeling that I'll have to write some rows to dynamodb and then have a lambda function calling that processes the rows and increments the count. – Luke Jul 04 '17 at 09:30
  • What part exactly are you worried about scale? DynamoDB is primarily designed for scale - you can scale to millions of reads or writes per second, TBs of data, etc without any massive impact (provided you've designed your table properly) – Henry Jul 04 '17 at 10:01
  • I think it's to do with incrementing a single value that resides in a single location, because a single value resides on a single shard. Some have suggested implementing a distributed count in Dynamodb and then flushing the data to a central count. Sounds like it's not an easy thing to do when it's scaling to a lot of reads. Any thoughts are appreciated. Maybe it'll be alright for my use case though, it'll be thousands of updates a day, not a second – Luke Jul 04 '17 at 14:50
  • DynamoDB uses atomic operators which mean that the operation will do the update - you cant end up in a situation (As far as I know) where you, for example, have a count of 3 and try to do two atomic operations which somehow end you up at 4. It'll always go to 5. (To the best of my knowledge). Dynamo is designed to scale this sort of traffic over tens of thousands of operations *per second*. One famous related example is using dynamoDB as a vote counting tool - do be aware that you can only do 3000 writes per second on a partition in Dynamo. But it sounds like that isnt an issue for you – Henry Jul 04 '17 at 14:55
  • You've probably seen it, but this is worth looking at: https://stackoverflow.com/questions/9368293/atomic-counters-in-dynamodb As I said, the only limitation i'm aware of is either the 10 GB per primary key (unlikely to be an issue for you), or 3000 WCUs (that's kind of 3000 writes per second) - if you're using multiple Puts in one call then, for an atomic counter, you can probably deliver massively higher throughput than 3000 writes a second/ – Henry Jul 04 '17 at 14:58
  • Thanks Henry, this is all very useful information for me. I'm a bit of a noob when it comes to DynamoDb. Time for me to get stuck in. – Luke Jul 04 '17 at 15:04
  • Dont worry, me too...! I know a lot about the theory of it. Adrian Cantril gives a great (allegedly) DynamoDB course on A Cloud Guru, fyi. – Henry Jul 04 '17 at 15:22