2

I am building an application where i am tracking user activity changes and showing the activity logs to the users. Here are a few points :

  1. Insert 100 million records per day.
  2. These records to be indexed and available in search results immediately(within a few seconds).
  3. Users can filter records on any of the 10 fields that are exposed.
Arun Manivannan
  • 4,213
  • 3
  • 29
  • 38
Ashish Arora
  • 21
  • 1

3 Answers3

1

I think both Mongo and Oracle will not accomplish what you need. I would recommend offloading the search component from your primary data store, maybe something like ElasticSearch:

http://www.elasticsearch.org/

Cody Caughlan
  • 32,456
  • 5
  • 63
  • 68
1

My recommendation is ElasticSearch as your primary use-case is "filter" (Facets in ElasticSearch) and search. Is it written to scale-up (otherwise Lucene is also good) and keeping big data in mind.

Aravind Yarram
  • 78,777
  • 46
  • 231
  • 327
0

100 million records a day sounds like you would need a rapidly growing server farm to store the data. I am not familiar with how Oracle would distribute these data, but with MongoDB, you would need to shard your data based on the fields that your search queries are using (including the 10 fields for filtering). If you search only by shard key, MongoDB is intelligent enough to only hit the machines that contain the correct shard, so it would be like querying a small database on one machine to get what you need back. In addition, if the shard keys can fit into the memory of each machine in your cluster, and are indexed with MongoDB's btree indexing, then your queries would be pretty instant.

snez
  • 2,400
  • 23
  • 20