Improve performance of reading event log

Question

I am queries event log of different domain controllers, I have to keep querying that after some time interval.

Below is the code I am using to query it.

public static void FindAllLog(string machineName)
        {
            //EventLog log = new EventLog("", "");
            //log.
            EventLog[] remoteEventLogs;
            // Gets logs on the local computer, gives remote computer name to get the logs on the remote computer.
            remoteEventLogs = EventLog.GetEventLogs(machineName);
            Console.WriteLine("Number of logs on computer: " + remoteEventLogs.Length);

            for (int i = 0; i < remoteEventLogs.Length; i++)
            {
                Console.WriteLine("Log : " + remoteEventLogs[i].Log);
                ReadEventLog(machineName, remoteEventLogs[i].Log, DateTime.Now.AddDays(-30));
                //ReadAppEventLog(machineName, remoteEventLogs[i].Log);                
            }
        }

public static void ReadEventLog(string machine, string logType,DateTime fromDate)
        {
            EventLog ev = new EventLog(logType, machine);
            var entry = (from EventLogEntry e in ev.Entries
                         where e.TimeGenerated >= fromDate
                         orderby e.TimeGenerated
                         select e);//.LastOrDefault();
            foreach (EventLogEntry CurrentEntry in entry)
            {
                Console.WriteLine("Event ID : " + CurrentEntry.EventID);
                Console.WriteLine("Event Source : " + CurrentEntry.Source);
                Console.WriteLine("Event TimeGenerated : " + CurrentEntry.TimeGenerated);
                Console.WriteLine("Event TimeWritten : " + CurrentEntry.TimeWritten);
                Console.WriteLine("Event MachineName : " + CurrentEntry.MachineName);
                Console.WriteLine("Entry Type : " + CurrentEntry.EntryType.ToString());
                Console.WriteLine("Message :  " + CurrentEntry.Message + "\n");
                Console.WriteLine("-----------------------------------------");
            }
        }

When first time I am querying a domain controller, I have to read log of last 30 days. Else just read latest log from the last time we left. Its taking hell lot of time to query it? I tried with WMI, same issue hell lot of time and it is sometime giving “Invalid Query Error”? How to improve this? Any model you suggest to do this task, I am doing multithreading here for each Domain Controller?

Thanks

I assume that you want them in order, or do you not care about the order in which they are written? — Joshua Drake, May 16 '12 at 12:50
To start you could take a look at the [Parallel.ForEach Method](http://msdn.microsoft.com/en-us/library/system.threading.tasks.parallel.foreach.aspx) and see if the entry object will support it. — Joshua Drake, May 16 '12 at 13:04
I can knock off orderby e.TimeGenerated. But my main issue is like first time fetching is so slow when i will be querying 30 days events from all event logs. — sunder, May 16 '12 at 13:05
You could also potentially use the [Parallel.For Method](http://msdn.microsoft.com/en-us/library/dd783539) in your `FindAllLog` method. — Joshua Drake, May 16 '12 at 13:07
Found this SO question [What is the Fastest way to read event log on remote machine?](http://stackoverflow.com/questions/914446/what-is-the-fastest-way-to-read-event-log-on-remote-machine) — Joshua Drake, May 16 '12 at 13:14
Provide the exact code that is causing you the performance issues. — Peter Ritchie, May 16 '12 at 13:14

score 2 · Accepted Answer · answered Jul 18 '12 at 07:14

To answer this question. I tried all style of reading event logs.

Using .NET2.0 approach using EventLog class, then reading using .NET3.0 approach using EventLogQuery and EventLogReader class, finally I tried WMI approach.

I have to read event logs based on time or in time slice for every 5 mins or so.

You guys will surprised to know that WMI will retrieve data way more faster then other .NETx approach and we will get more fields and no OS dependencies or firewall issues.

But other two approaches have cons.

just wanted to share this.

Thanks

score 1 · Answer 2 · answered Sep 25 '13 at 09:03

Avoid LINQ when reading from EventLog. Try this:

// Store indices of last accessed EventLogEntries in Dictionary {logType, lastIndex}
private static readonly Dictionary<string, int> _lastIndices = new Dictionary<string, int>();

public static void FindAllLog(string machineName)
{
    //EventLog log = new EventLog("", "");
    //log.
    EventLog[] remoteEventLogs;
    // Gets logs on the local computer, gives remote computer name to get the logs on the remote computer.
    remoteEventLogs = EventLog.GetEventLogs(machineName);
    Console.WriteLine("Number of logs on computer: " + remoteEventLogs.Length);

    for (int i = 0; i < remoteEventLogs.Length; i++)
    {
        Console.WriteLine("Log : " + remoteEventLogs[i].Log);
        ReadEventLog(machineName, remoteEventLogs[i].Log, DateTime.Now.AddDays(-30));
        //ReadAppEventLog(machineName, remoteEventLogs[i].Log);                
    }
}

public static void ReadEventLog(string machine, string logType, DateTime fromDate)
{
    int lastIndex;
    EventLog ev = new EventLog(logType, machine);
    IList<EventLogEntry> entries = new List<EventLogEntry>();

    if (_lastIndices.ContainsKey(logType))
        lastIndex = _lastIndices[logType];
    else {
        lastIndex = 0;
        _lastIndices.Add(logType, 0);
    }

    // Try to avoid LINQ because it uses Enumerator and Loops EVERYTIME trough all items.
    // Start Looping from top of the list and break if Entry has Index less than lastIndex or
    // if Entry has TimeWritten less than fromDate
    for (var i = ev.Entries.Count - 1; ev.Entries[i].Index > lastIndex && ev.Entries[i].TimeWritten > fromDate; i--)
        entries.Add(ev.Entries[i]);

    if (entries.Count > 0) // Set lastIndex for corresponding logType
        _lastIndices[logType] = entries.Max(e => e.Index);

    foreach (EventLogEntry CurrentEntry in entry.OrderBy(e => e.TimeWritten))
    {
        Console.WriteLine("Event ID : " + CurrentEntry.EventID);
        Console.WriteLine("Event Source : " + CurrentEntry.Source);
        Console.WriteLine("Event TimeGenerated : " + CurrentEntry.TimeGenerated);
        Console.WriteLine("Event TimeWritten : " + CurrentEntry.TimeWritten);
        Console.WriteLine("Event MachineName : " + CurrentEntry.MachineName);
        Console.WriteLine("Entry Type : " + CurrentEntry.EntryType.ToString());
        Console.WriteLine("Message :  " + CurrentEntry.Message + "\n");
        Console.WriteLine("-----------------------------------------");
    }
}

I used here the TimeWritten property because it is more reliable than TimeGenerated. The TimeGenerated can be out of order but the TimeWritten is allways ascending as well as the Index. I hope this helps.

Improve performance of reading event log

2 Answers2

Linked

Related