Which nosql database for heterogeneous records?

Question

I'm looking at different options for storing log entries for easier querying/reporting.
Currently I write scripts that parse and find the data, but the data is becoming more and more in demand, so it's becoming worth it to put the log data in a database.

Log entries are composed of key-value pairs, such as{"timestamp":"2012-04-24 12:34:56.789", "Msg":"OK" (simplified example).
I'm sure that eventually the log format will be extended to, say {"timestamp":"2012-04-24 12:34:56.789", "Msg":"OK", "Hostname":"Bubba", which means that the "schema" or "document definition" will need to change. Also, we're a Windows + .NET shop.

Hence, I was primarily looking for some NoSQL engine and found RavenDB attractive to use from .NET.
However, I have a hard time finding information about how it, and other NoSQL databases, work with heterogeneous records.
What would be a good fit in your opinion?

score 3 · Accepted Answer · answered Apr 24 '12 at 20:19

With RavenDB you can just store the different types of docs and it will be able to handle the "changes" in schema. Because it is in fact "schema-free", you can write indexes that will only index the fields that are there. See this blog post for some extra info. It's talking about migrations, but the same applies here.

Also the dynamic fields option will help you here. So given doc with arbitrary properties:

public class LogEntry
{
    public string Id { get; set; }
    public List<Attribute> Attributes { get; set; }
}

public class Attribute
{
    public string Name { get; set; }
    public string Value { get; set; }
}

You can write queries like this:

var logs = session.Advanced.LuceneQuery<LogEntry>("LogEntry/ByAttribute")
    .WhereEquals("Msg", "OK")
    .ToList();

Which nosql database for heterogeneous records?

1 Answers1