3

I need a NoSql database to write continuous log data. Approx. 100 write per second. And a single data is contains 3 column and less than 1kb. Read is necessarily only once a day, then I can delete all daily data. But I can't decide that which is the cheapest solution; Google App Engine and Datastore or Heroku and Mongolab?

ecabuk
  • 1,641
  • 1
  • 18
  • 20
  • 1
    Writing logs in real time is a requirement? Since you want to read them only once a day, why you don't store everything in a file and read it when you need? – a.b.d Sep 22 '12 at 21:23
  • Yes, you are right but it's a web application and I want to write each api requests, so when writing to a file (file locking when writing or something else) isn't be a problem? – ecabuk Sep 22 '12 at 21:32
  • 1
    I don't know wich framework/language you are using but this is a very common use case. I am almost sure that you can find a logging library that match your requirements. – a.b.d Sep 22 '12 at 21:41
  • If you want MongoDB-like API you may want to try: https://github.com/mungo-appengine/mungo – quarks Feb 25 '14 at 01:46

2 Answers2

8

I can give you costs for GAE:

Taking billing docs and assuming you'll have about 258M operations per (86400 second per day * 100 requests/s) this would cost you 

Writing: 258M record * ($0.2 / 100k) = $516 for writing unindexed data

Reading: 258M records * ($0.07 / 100k ops) = $180 for reading once a month

Deleting 258M rec * ($0.2 / 100k) = $516 for deleting unindexed data

Storage: 8.6M entities at 1kb per day = 8.6GB per day = 240 GB / month = averaged 120 GB

Storage cost: 120 GB * 0.12$/GB = $15 / month

So your total operation per month on GAE would be about $1300 per month. Note that using a structured database for writing unstructured data is not optimal and it reflects on the price.

Peter Knego
  • 79,991
  • 11
  • 123
  • 154
1

With App Engine, it is recommended that you use memcache for operations like this, and memcache does not incur database charges. Using python 2.7 and ndb, memcache is automatically used and you will get at most 1 database write per second.

At current billing:

6 cents per day for reads/writes.

Less than $1 per day storage

Richard
  • 217
  • 1
  • 6