0

Sorry if there's already an answer for this, I searched for it and I didn't find exactly my scenario.

Once again is a question like "What is the fastest/best performance DB?". But since the answer depends on the scenario, my scenario is this: I want to write many logs to DB, thousands per second. But I will not read them often. Indeed 99,99% of them will never be read again, but once in a while I will need to read. Schema is not complex, just key/value. Once in a while I will read by value and I will not care at all if this read takes minutes. The correctness of the read will be critical, but not the performance.

So far it seems the best solutions are things like MongoDB, Cassandra... and perhaps the best DynamoDB?

talonmies
  • 70,661
  • 34
  • 192
  • 269
antonio.fornie
  • 1,740
  • 2
  • 13
  • 23
  • There really is no "best"; if there were, you wouldn't need to ask it would just be common knowledge. Your candidates are all worthy and are more a matter of taste than any fact. As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or specific expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. – msw Jul 16 '12 at 14:34
  • Are your going to write logs structured? How big you expect them to be in a long delay? – Viktor Stolbin Jul 16 '12 at 15:12
  • Well, I thought it would be a good fit question if I said what's the best for a very specific scenario, like the described. Even if the answer is a set of products, instead of one above all. – antonio.fornie Jul 18 '12 at 15:20
  • Not structured. It's the simplest data structure you could imagine: key:value. A table with 2 columns. And very big. Indeed it will be as big as possible given limitations we may find. You can imagine: thousands of inserts per second. We never do updates. Very rarely we will do reads. And for us it's important not delete anything unless is really old. We just need to know that in the future, if we need to read back one of the values by key, it's possible to do it even if it takes a long time to do the read. We may do this read once a month but it would save much money for us, so it's worth it. – antonio.fornie Jul 18 '12 at 15:26

2 Answers2

1

Any DBMS, i would say, switch to the lowest isolation level and no index. If you put that together with a good storage system, maybe a RAID 0 with SSDs. Fastes writes ever.

Its hard to say wich DBMS is best, usually you want the best dbms that is good in doing something in particular but you need a dbms that basically just write something with the least of restriction, ive heard mysql can be great in this.

memo
  • 441
  • 2
  • 6
  • Good to know, but I thought something like MongoDB or DynamicDB would be faster and specially much more scalable than anything I could ever do with mySql.By the way, don't you think using a cloud service would scale much better than using something like a RAID? Just my opinion, I'm not expert on that. – antonio.fornie Jul 18 '12 at 15:31
0

If fast writes are what you're after, you have a few options. Assuming that you will be the one to maintain the DB you can write the inserts to memory, and flush them after they get to a certain size. that way you aren't hitting the disk so many times.

If I'm not mistaken MongoDB does that already, plus if you disable the journaling it can drastically increase write performance which is exactly what you're going for.

Either way, caching and bulk inserting is the way to go with any database.

Jared Wadsworth
  • 839
  • 6
  • 15