How should I filter my VMWare Logs? What should I search for? What should I exclude?

Question

My VMWare logs are full of SSL errors and some other things that appear less than important. (e.g. SSL Errors from the admin console)

What is the best way to extract valuable information from these logs, in hopes of providing better/more reliable service?

I'm looking for specific key words or phrases that will assist in this. I'm not needing regex expressions, just english words to focus on or ignore. My intent is that I'll later tie this into my monitoring solution.

score 1 · Accepted Answer · answered Sep 18 '11 at 08:59

1

There are two basic methods.

Run the logs through a filter that will extract those things you do want to see.
Run the logs through a filter that will ignore those things you don't want to see.

There are an almost infinite number of way to do the filtering. What will work best for you will depend on just what you want to see and what your level of skill is in scripting and/or programming. My personal preference is to parse the logs through a Perl script that will ignore all those things I don't want to see. The output can be anything from a simple text file to an Excel spreadsheet, depending on what logs I'm processing. Of course the details of the implementations are beyond the scope of SF and are a better fit for StackOverflow.

answered Sep 18 '11 at 08:59

John Gardeniers

27,458
12
55
109

Thank you... do you have any key words I should search for ignore based on your VMWare Experience? Do all "errors" have "error" in the string? There doesn't appear to be a logical structure to the class of each line... it's more like a kernel braindump when I look at it. – makerofthings7 Sep 18 '11 at 19:12
@makerofthings7, this is where it gets hard. You need to study the log format to determine how to create the filters. Not being familiar with those particular logs I can offer no advice in this regard. – John Gardeniers Sep 18 '11 at 21:53
1

Perhaps I'll revise the question so I can get the SO community to recommend something – makerofthings7 Sep 18 '11 at 22:27

score 0 · Answer 2 · answered Sep 20 '11 at 18:53

By far the best method I've seen for managing VMware logs is Splunk - I dump all my logs there and create saved searches looking for any unusual errors. I started with Perl scripts / grepping results on a syslog collection server and found my log analysis to still be... lacking.

Simply set your Splunk host as a syslog forwarding destination and add the appropriate listener. Splunk makes the field extraction process easy so cross correlation becomes a much simpler task as well. The 500 megabyte / day free license and cross platform support made it a great choice for my environment.

How should I filter my VMWare Logs? What should I search for? What should I exclude?

2 Answers2