Stack Overflow
Stack Overflow

Questions tagged [log-analysis]

Log-analysis is the process of parsing log file, created from any logging service or application.

126 questions
0
votes
0 answers

Error Log Analysis - Apache, Nginx

I want a Analyse from the Error Logs from Nginx & Apache. Only a count what is the most Error. For Counting Access.logs i have found this script: awk -F\" '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -n But i can't regex and have…
Huskynarr
  • 83
  • 2
  • 9
0
votes
1 answer

Do you know batch log processing tools for hadoop (zohmg alternatives)?

Since the zohmg project seems to be dead (no new commits since nov 2009), I would like to know if any of you used/uses it (with successful results). Or if you know anything about future of this project. And if not, is there any alternative for…
wlk
  • 5,695
  • 6
  • 54
  • 72
0
votes
3 answers

Python regex is not extracting a substring from my log file

I'm using date = re.findall(r"^(?:\w{3} ){2}\d{2} (?:[\d]{2}:){2}\d{2} \d{4}$", message) in Python 2.7 to extract the substrings: Wed Feb 04 13:29:49 2015 Thu Feb 05 13:45:08 2015 from a log file like this: 1424,Wed Feb 04 13:29:49…
mdandr
  • 1,384
  • 3
  • 9
  • 19
0
votes
1 answer

Behavioral analysis to find anomalies in Splunk

I want to perform Behavioral analysis / anomalies detection in Splunk by comparing Historical (say last months data) with todays data to find anomalies. I am analyzing FTP logs, so e.g I want to have a historical baseline/report of all users with…
Sourabh Potnis
  • 1,431
  • 1
  • 17
  • 26
0
votes
1 answer

What is the best log analyzer tool for linux server?

I am currently working on setting up server which generates reports and upload them to external SFTP. I need a log analyzer which is possible free and runs on Linux server. Which one is the best solution? I read a lot about SumoLogic, but not sure…
user3308388
  • 23
  • 1
  • 2
0
votes
2 answers

Logs guaranteed delivery using Logstash/Redis in shipper node

What do you think folks, about placing Redis in each Logstash shipper node to ensure a guaranteed logs delivery?
Ben
  • 69
  • 1
  • 5
0
votes
0 answers

Assisted manual inspection of log files

Rookie question here. I've been inspecting a lot of log files to try to pinpoint errors in an application. Specifically I'm trying to compare success scenarios with failure scenarios... but due to the volume of logs it's difficult to identify which…
Dan
  • 1,677
  • 5
  • 19
  • 34
0
votes
0 answers

On parsing logfiles using token

This is a general question, forgive me if this is common knowledge, I don't even know what exactly to search for. I deal with enormous logfiles (aren't they all) that are generally in the same format. Date - Time [calling binary] (error…
Danielle M.
  • 3,607
  • 1
  • 14
  • 31
0
votes
1 answer

How to get the users logged in from two differnt IPs using squid log

I have a squid log, and I have to find out the users who are logged in from two different IPs ( their password may be compromised ) I have extracted three info (user, time, ip) from the log and stored in another file 1110104 1397367240.280…
Harry
  • 1,572
  • 2
  • 17
  • 31
0
votes
1 answer

Real time log analyzer - file access every few seconds

I'm writing simple script in bash, to analyze some logs in real time and wondering about how to approach the fact, that every few seconds I have to find place in the file that I finished reading previous time. Right now I am doing something like…
Liberat0r
  • 1,852
  • 2
  • 16
  • 21
0
votes
1 answer

Event breaking in flume

By default flume considers one line as one event, But I want to do breaking on some other criteria how it can be achieved in flume? **10 Sep 2013 19:43:33,561 [WebContainer : 9] ERROR - An Error has occured for…
Chhaya Vishwakarma
  • 1,407
  • 9
  • 44
  • 72
0
votes
1 answer

Log analysis with Apache Pig

I have logs with this rows: in24.inetnebr.com - - [01/Aug/1995:00:00:01 -0400] "GET /shuttle/missions/sts-68/news/sts-68-mcc-05.txt HTTP/1.0" 200 1839 where the first column (in24.inetnebr.com) is the host, the second (01/Aug/1995:00:00:01 -0400)…
alfayadd
  • 23
  • 9
0
votes
1 answer

CSV file date processing in shell script

I have a csv log file with two columns, each having time stamp for request(1 st column) and response(2nd column). Here is some sample data: 2013-07-11 08:39:08.748,2013-07-11 08:39:08.748 2013-07-11 08:39:08.826,2013-07-11 08:39:08.826 2013-07-11…
Learn More
  • 1,535
  • 4
  • 29
  • 51
0
votes
1 answer

How to analyse log with > 30m measurements

Consider a Java application that receives financial trading transactions to determine their vality by applying several checks, such as if the transaction is allowed under contractual and legal constraints. The application implements a JMS message…
miraculixx
  • 10,034
  • 2
  • 41
  • 60
0
votes
1 answer

What are the differences between Taobao's open source projects: Metamorphosis and Timetunnel?

I'd like to build a logs aggregation system and I found these tools developed by Taobao. Both of them can be used to collect logs for further processing and analysis. What's the different betweent them?
cofthew7
  • 35
  • 5
1 2 3
…
8
9