1

I'm using IIS 8.5 Advanced Logging and also logging some custom HTTP headers. How can I import the raw logs into SQL on a regular basis (i.e. every minute)? I want to avoid the overhead of ODBC logging, which Microsoft also recommends against. Is there a command line utility that I can script via Powershell, for example? Perhaps bcp? Something else?

Mark Richman
  • 286
  • 1
  • 5
  • 15
  • Couldn't you feed SQL the log file? I'm guessing that would be more efficient than powershelling it for instance.. https://support.microsoft.com/en-us/kb/296085 – ErikE Jun 21 '15 at 21:28
  • I need the data in the table in as near realtime as possible, which is why a job every 60s would be best. Ultimately, this will be front-ended by an ASP.NET app. – Mark Richman Jun 21 '15 at 23:44

1 Answers1

4

You can use Microsoft's log parser utility.

It has the capability of dumping into a SQL database as well as having the ability to create a checkpoint file so that it only imports records it hasn't processed. It can be used either as a stand alone executable or programmatically invoked via an ActiveX control

Abhijeet Kasurde
  • 983
  • 9
  • 20
vermyx
  • 56
  • 1
  • This is exactly what I'm after, especially the checkpoint feature. I'm going to experiment with this a bit and then revert with any more questions I might have. Marking as answered. – Mark Richman Jun 22 '15 at 12:38
  • I realize this is an old post but was this ever confirmed? The OP mentions custom fields which means that -i:IISW3C cannot be used and it seems that Checkpoints are only supported with that import format. How can you use Checkpoints with custom fields? – Jason Jan 18 '17 at 03:38
  • Checkpoint files can be used with any file that log parser understands and is not limited to the IIS log format. If it does not conform to the IIS log file structure, you can just use the text delimited file structure instead. Log parser will also allow you to have just a data file and have a separate log header file defined if your log file does not have a header line, as well as allow you to "skip" lines at the beginning of the file if it has verbose descriptions. – vermyx Jan 30 '17 at 07:35