Parsing Log File - C# or any other

Question

I'm trying to parse a log file containing info like this:

2015-03-08 10:30:01     /user849/connect
2015-03-08 10:30:01     /user262/open-level2-price
2015-03-08 10:30:01     /user839/open-detailed-quotes
2015-03-08 10:30:02     /user145/add-technical-drawing
2015-03-08 10:30:02     /user108/connect
2015-03-08 10:30:03     /user850/filter-changed
2015-03-08 10:30:03     /user818/open-level2-price
2015-03-08 10:30:03     /user841/column-width
2015-03-08 10:30:03     /user850/filter-changed
2015-03-08 10:30:04     /user850/connect
2015-03-08 10:30:04     /user420/duration
2015-03-08 10:30:04     /user851/filter-changed
2015-03-08 10:30:04     /user217/duration
2015-03-08 10:30:05     /user82/update-column-properties
2015-03-08 10:30:05     /user809/open-level2-price
2015-03-08 10:30:05     /user382/add-technical-drawing
2015-03-08 10:30:06     /user851/connect
2015-03-08 10:30:07     /user350/add-technical-drawing
2015-03-08 10:30:09     /user849/filter-changed
2015-03-08 10:30:09     /user842/sort
2015-03-08 10:30:09     /user849/open-market-watch
2015-03-08 10:30:10     /user429/interval
2015-03-08 10:30:10     /user218/change-columns
2015-03-08 10:30:11     /user749/connect
2015-03-08 10:30:13     /user759/open-detailed-quotes
2015-03-08 10:30:14     /user753/connect
2015-03-08 10:30:14     /user377/connect

I'm trying to find the 3 most used operations, and their percentages, what I have in mind is reading the file, parsing the lines with some regular expressions, or filling them to a data table and then processing that data table, but i was not able to do it.

Can you advice me what to do, where to start, or maybe some code sample (Preferably c#) ?

Thanks in advance!

---

Edit:

Well, (I've successfully done it now) As for what I've tried, here's is my code

string filePath = @"6458.log";

try
{
    DataTable logLines = new DataTable("LogLines");
    //logLines.Columns.Add(new DataColumn("DateTime", System.Type.GetType("System.DateTime")));
    logLines.Columns.Add(new DataColumn("User", typeof(string)));
    logLines.Columns.Add(new DataColumn("Operation", typeof(string)));

    string[] lines = System.IO.File.ReadAllLines(filePath);

    foreach (string line in lines)
    {
        var cols = line.Split(new char[] { ' ', '/' }, StringSplitOptions.RemoveEmptyEntries);

        DataRow dr = logLines.NewRow();

        //dr["DateTime"] = cols[0] + " " + cols[1];
        dr["User"] = cols[2];
        dr["Operation"] = cols[3];

        logLines.Rows.Add(dr);
    }

    var query = from row in logLines.AsEnumerable()
                group row by row.Field<string>("Operation") into operations
                orderby operations.Count() descending
                select new
                {
                    Name = operations.Key,
                    CountOfClients = operations.Count()
                };
}
catch (Exception ex)
{
    throw(ex) ;
}

Please back your hint with code for further explanation!

Thanks Again

Answer 1

You can add the lines of the file to List then use linq to get the data you want

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using System.Text.RegularExpressions;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            List<data> logs = new List<data>();
            var path=Path.Combine(Environment.CurrentDirectory+@"\file.txt");
            using (StreamReader sr = new StreamReader(path))
            {
                string line;
               while((line = sr.ReadLine()) != null)
               {

                   var log = Regex.Split(line, "     ");
                   logs.Add(new data { LogDate=DateTime.Parse(log[0]),Operation=log[1]});
               }
            }

            // here you can use linq to get the data you want from logs list



            // end of main 
        }


        public class data
        {
            public DateTime LogDate { get; set; }
            public string Operation { get; set; }
        }
        // end of class
    }


}

file.txt is the log file you want to read

Answer 2

If you just want to get the actions and the number of calls you can use this piece of code.

Dictionary<string, int> items = new Dictionary<string, int>();

foreach(string line in lines)
{
    var cols = line.Split(new char[] { '/' }, StringSplitOptions.RemoveEmptyEntries);
    var operation = cols[2].Trim();

    if(items.Keys.Any(x => x.Equals(operation)))
    {
        items[operation]++;
    }
    else
    {
        items[operation] = 1;
    }
}

After this you have a dictionary where the actions are the keys and the count of calls is the value.

If you want to make the parsing a bit more error resistant you can change this part without chaning the logic at all.

If you want to get the count of all actions use this.

var actionCount = items.Sum(x => x.Value);

And if you want to get the percentages of for example "connect" actions you can use this.

var percentage = 100.0 / actionCount * items["connect"];

But you have to check if there are any "connect" entries in the dictionary of this line will fail. You can check with if a key exists you can use

items.ContainsKey("connect");