How to properly create a Map/Reduce Index for RavenDB in C#

Question

I'm working on an app that uses RavenDB on the back end. It's my first time using Raven, and I'm struggling with Map/Reduce.

I have been reading the doc's, but unfortunately I'm not getting anywhere in the process.

Basically I have thousands of documents like this.

{
  .....
  "Severity": {
    "Code": 6,
    "Data": "Info"
  },
  "Facility": {
    "Code": 16,
    "Data": "Local Use 0 (local0)"
  },
  .....
}

And out of it, I need to make a single query with output that looks like this.

{"Severity": [
    {"Emergency":0},
    {"Alert":0},
    {"Critical":0},
    {"Error":0},
    {"Warning":0},
    {"Notice":0},
    {"Info":2711},
    {"Debug":410}
],
"Facility": [
    {"Kernel Messages":0},
    {"User-Level Messages":0},
    {"Mail System":0},
    {"System Daemons":0},
    {"Security/Authorization Messages":0},
    {"Internal Syslogd Messages":0},
    {"Line Printer Subsystem":2711},
    {"Network News Subsystem":410},
    ....
    {"Local Use 0 (local0)": 2574},
    ...
]}

Whereby the "Key" in the Severity/Facility Array is the Data portion of the above json data, and the "value" in the Severity/Facility Array is the document Countfor each Code type.

Example:
Using the above data as a guideline,

There are 2711 documents in my database with an Info severity.
There are 410 documents in my database with a Debug severity.
There are 2574 documents in my database with a local0 facility.
etc...

---

What I'd like to do is generate the appropriate indexes when the app starts up (or check if they already exist), but I don't even know where to begin.

note: the app needs to generate the index, it's not enough to just manually write it into the RavenDB Web UI.

Answer 1

You will need to combine several techniques to achieve this, but it is quite doable.

Here is an index that should work well for you.

public class MyIndex : AbstractMultiMapIndexCreationTask<MyIndex.ReduceResult>
{
    public class ReduceResult
    {
        public string Source { get; set; }
        public string Code { get; set; }
        public string Data { get; set; }
        public int Count { get; set; }
    }

    public MyIndex()
    {
        AddMap<MyDoc>(docs => from doc in docs
                              select new
                                     {
                                         Source = "Severity",
                                         doc.Severity.Code,
                                         doc.Severity.Data,
                                         Count = 1
                                     });

        AddMap<MyDoc>(docs => from doc in docs
                              select new
                                     {
                                         Source = "Facility",
                                         doc.Facility.Code,
                                         doc.Facility.Data,
                                         Count = 1
                                     });

        Reduce = results => from result in results
                            group result by new { result.Source, result.Code }
                            into g
                            select new
                            {
                                g.Key.Source,
                                g.Key.Code,
                                g.First().Data,
                                Count = g.Sum(x => x.Count)
                            };

        TransformResults = (database, results) =>
                           from result in results
                           group result by 0
                           into g
                           select new
                           {
                               Severity = g.Where(x => x.Source == "Severity")
                                           .ToDictionary(x => x.Data, x => x.Count),
                               Facility = g.Where(x => x.Source == "Facility")
                                           .ToDictionary(x => x.Data, x => x.Count)
                           };
    }
}

You also need a container class for the transformed result:

public class MyDocCounts
{
    public IDictionary<string, int> Severity { get; set; }
    public IDictionary<string, int> Facility { get; set; }
}

You would query it like this:

var result = session.Query<MyIndex.ReduceResult, MyIndex>()
                    .As<MyDocCounts>()
                    .ToList().First();

The .ToList() may seem redundant, but it's necessary because we are grouping in the transform.

A complete unit test is here. The output of which looks like this:

{
  "Severity": {
    "AAA": 20,
    "BBB": 20,
    "CCC": 20,
    "DDD": 20,
    "EEE": 20
  },
  "Facility": {
    "FFF": 20,
    "GGG": 20,
    "HHH": 20,
    "III": 20,
    "JJJ": 20
  }
}