Questions tagged [logparser]

A log parser extracts (usually text) data from a log file into a more specific data structure, making the log information easier to process. A log parser can be implemented as a standalone application, or as a library for a specific programming language. LogParser is also the name of a free Microsoft tool which provides a SQL-like interface to log files of almost any format, from generic CSV, TSV, and XML files to more specific formats.

Many server-side applications create log files, usually in a more-or-less structured text format.

While this allows for maximum flexibility in accessing these logs with a wide variety of tools, it makes programmatic processing a bit of a pain, especially when no concise format specification is available for the logs.

This is where a log parser comes in: it is either a standalone application or a library designed for one or sometimes multiple programming languages, and converts the log file into more specific data structures, such rows in an Excel sheet or class instances.

The newly created data structures contain field names and are type-safe as much as possible, making further processing much easier.

296 questions
-1
votes
2 answers

regex for python based logparser for printing aws elb logs?

I am trying to write python code to extract certain fields from elb logs but i am not able to find proper regex for all elb log fields like "user_agent" , request etc like how to print pattern "POST…
-1
votes
1 answer

IIS logentries yesterday - 24 hours

Is there anyone who can help with af logparser query? I would like all yesterday's entries from the IIS log from 00:00:00 hours to 23:59:59 hours. Someone who knows that query?
OJ1
  • 3
  • 3
-1
votes
1 answer

compile regex from log file into a dict, error when log file has a new attribute

So I need to parse an nginx log file. Half way through the log file a new variable was added at the very end to each line. I used https://github.com/bbb1991/nginx-log-parser/blob/master/main.py as an inspiration (i.e. used most of the code). import…
user3674993
  • 129
  • 1
  • 9
-1
votes
2 answers

searching multiple keyword and print out matched keyword with several field in logs

example log (sample.log) 11:00:00 INFO KOREA AAABB DDDRR 11:00:01 INFO SWEDEN TYTY PLO 11:00:02 INFO US OIILL OTLTOL 11:00:03 INFO NORTHKOREA OTL KEYWORDs (multiple) "KOREA", "OTL" What I want as a result 11:00:00 KOREA 11:00:02 OTL 11:00:03…
-1
votes
1 answer

Log parser 2.2 Query to count Unique user

I've been trying to build a query over a custom log of mine where I sort the users based on certain criteria to have some overview of them. My log contains a entry for each time a user tries to download a file, that entry contains date, ip, a custom…
Epodax
  • 1,828
  • 4
  • 27
  • 32
-1
votes
1 answer

Compact data structure for storing parsed log lines in Go (i.e. compact data structure for multiple enums in Go)

I'm working on a script that parses and graph information from a database logfile. Some examples loglines might be: Tue Dec 2 03:21:09.543 [rsHealthPoll] DBClientCursor::init call() failed Tue Dec 2 03:21:09.543 [rsHealthPoll] replset info…
victorhooi
  • 16,775
  • 22
  • 90
  • 113
-1
votes
1 answer

Log Parser New Visits

I'm trying to use log parser to only show the source that an ip address used during its first visit to the site. I want to include all sources and the number of ip addresses that accessed the site from this source on its first visit. Does anyone…
-2
votes
1 answer

IIS Logs Parsing

I have requirement for IIS Logs parsing using python. PFB sample logs for your reference. Requirement: ->>Unique IPs List with 443 port and ->>Unique IPs List with 8080 port #Software: Microsoft Internet Information Services 7.5 #Version: 1.0 #Date:…
-2
votes
2 answers

regex for logback log parsing

Why isn't the following ([a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}) (\d{4}-\d{2}-\d{2}) (\d{2}:\d{2}:\d{2},\d{3}) \[(.*?)\] ([^ ]*) +([^ ]*) - (.*)$ not matching any thing from the…
Aravind Yarram
  • 78,777
  • 46
  • 231
  • 327
-3
votes
1 answer

Looking for a COM Input Plugin allowing SQL SERVER as an Input Format

Is anyone aware of any links/urls that document how to write a Windows Script Component COM that will allow SQL Server as a custom Input Format for LogParser 2.2? Thanks for sharing this and any other insight into this.
AAsk
  • 1,431
  • 4
  • 17
  • 25
-3
votes
3 answers

How to parse Dante log file

I have a dante socks log file and trying to a way out to parse getting two parts and seperate them with a semicolon. Log File contains: Sep 11 03:02:00 (1568505720.145620) danted[10402]: info: pass(2): tcp/connect -:…
linuxman
  • 61
  • 5
1 2 3
19
20