Questions tagged [logging]

Computer data logging is the process of recording events in a computer program or computer system, usually with a certain scope, in order to provide an audit trail that can be used to understand the activity of the system and to diagnose problems. Be sure to include appropriate software or hardware tags in addition to this tag.

Logging

Computer data logging is the process of recording events in a computer program, usually with a certain scope, in order to provide an audit trail that can be used to understand the activity of the system and to diagnose problems. Logs are essential to understand the activities of complex systems, particularly in the case of applications with little user interaction (such as server applications).

Examples

Examples of physical systems which have logging subsystems include process control systems, and black box recorders installed in aircraft.

Many operating systems and complex computer programs include some form of logging subsystem. In the simplest case, log messages are written to a log file. Most operating systems and software frameworks also provide more sophisticated services for logging. One example is the syslog service (described in RFC 3164), which allows the filtering and recording of log messages to be performed by a separate dedicated subsystem, rather than placing the onus on each application to provide its own ad hoc logging system.

A server log is a log file (or several files) automatically created and maintained by a server of activity performed by it. A typical example is a web server log which maintains a history of page requests.

An audit log is a security-related log that provides documentary evidence of the sequence of activities that have affected at any time a specific operation, procedure, or event.

Standards

SysLog

Syslog is an informal standard for computer data logging that was developed in the 1980s by Eric Allman. It was created solely for Sendmail but proved so valuable that other applications began using it as well. It has since become the standard logging solution on Unix and Unix-like systems; there have also been a variety of implementations on other operating systems and it is commonly found in network devices such as routers.

The Internet Engineering Task Force has documented (but not formalized) the standard in RFC 5424.

Common Log Format

The Common Log Format (also known as the NCSA Common log format) and Extended Log Format are standardized text file formats used by web servers when generating log files. Because the formats are standardized, the files generated may be analyzed by a variety of web analysis programs.

Common Log Format entries take the form:

host ident authuser date request status bytes

Eg: 127.0.0.1 user-identifier frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326


References

41059 questions
362
votes
10 answers

Most useful NLog configurations

What are the best or most useful configurations for logging with NLog? (These can be simple or complex, as long as they're useful.) I'm thinking of examples like automatically rolling over log files at a certain size, changing the layout (log…
Pat
  • 16,515
  • 15
  • 95
  • 114
355
votes
8 answers

Log all queries in mysql

Is it possible for me to turn on audit logging on my mysql database? I basically want to monitor all queries for an hour, and dump the log to a file.
public static
  • 12,702
  • 26
  • 66
  • 86
323
votes
10 answers

Logging best practices

I'd like to get stories on how people are handling tracing and logging in real applications. Here are some questions that might help to explain your answer. Frameworks What frameworks do you…
Paul Stovell
  • 32,377
  • 16
  • 80
  • 108
319
votes
6 answers

How to Customize the time format for Python logging?

I am new to Python's logging package and plan to use it for my project. I would like to customize the time format to my taste. Here is a short code I copied from a tutorial: import logging # create logger logger =…
Hai Vu
  • 37,849
  • 11
  • 66
  • 93
317
votes
3 answers

How to log formatted message, object array, exception?

What is the correct approach to log both a populated message and a stack trace of the exception? logger.error( "\ncontext info one two three: {} {} {}\n", new Object[] {"1", "2", "3"}, new Exception("something went wrong")); I'd like to…
rowe
  • 3,171
  • 2
  • 15
  • 4
316
votes
23 answers

How should I log while using multiprocessing in Python?

Right now I have a central module in a framework that spawns multiple processes using the Python 2.6 multiprocessing module. Because it uses multiprocessing, there is module-level multiprocessing-aware log, LOG = multiprocessing.get_logger(). Per…
cdleary
  • 69,512
  • 53
  • 163
  • 191
310
votes
16 answers

How to enable MySQL Query Log?

How do I enable the MySQL function that logs each SQL query statement received from clients and the time that query statement has submitted? Can I do that in phpmyadmin or NaviCat? How do I analyse the log?
Feng-Chun Ting
  • 3,371
  • 3
  • 17
  • 11
303
votes
10 answers

How to log cron jobs?

I want to know how I can see exactly what the cron jobs are doing on each execution. Where are the log files located? Or can I send the output to my email? I have set the email address to send the log when the cron job runs but I haven't received…
Adrian M.
  • 7,183
  • 15
  • 46
  • 54
303
votes
5 answers

Can I change the name of `nohup.out`?

When I run nohup some_command &, the output goes to nohup.out; man nohup says to look at info nohup which in turn says: If standard output is a terminal, the command's standard output is appended to the file 'nohup.out'; if that cannot be…
David LeBauer
  • 31,011
  • 31
  • 115
  • 189
303
votes
32 answers

How to do a JUnit assert on a message in a logger

I have some code-under-test that calls on a Java logger to report its status. In the JUnit test code, I would like to verify that the correct log entry was made in this logger. Something along the following lines: methodUnderTest(bool x){ if(x) …
Jon
  • 3,509
  • 2
  • 17
  • 8
301
votes
7 answers

Retrieve last 100 lines logs

I need to retrieve last 100 lines of logs from the log file. I tried the sed command sed -n -e '100,$p' logfilename Please let me know how can I change this command to specifically retrieve the last 100 lines.
Surabhi
  • 3,508
  • 2
  • 17
  • 12
297
votes
12 answers

How to write to a file, using the logging Python module?

How can I use the logging module in Python to write to a file? Every time I try to use it, it just prints out the message.
Takkun
  • 8,119
  • 13
  • 38
  • 41
296
votes
3 answers

Good examples using java.util.logging

I want to use logs in my program. I heard about java.util.logging, but I don't know how to begin. Are there any examples of what can I do with logging? How would I use logging in my own program?
Renato Dinhani
  • 35,057
  • 55
  • 139
  • 199
286
votes
23 answers

live output from subprocess command

I'm using a python script as a driver for a hydrodynamics code. When it comes time to run the simulation, I use subprocess.Popen to run the code, collect the output from stdout and stderr into a subprocess.PIPE --- then I can print (and save to a…
DilithiumMatrix
  • 17,795
  • 22
  • 77
  • 119
285
votes
20 answers

How do I get logs from all pods of a Kubernetes replication controller?

Running kubectl logs shows me the stderr/stdout of one Kubernetes container. How can I get the aggregated stderr/stdout of a set of pods, preferably those created by a certain replication controller?
Torsten Bronger
  • 9,899
  • 7
  • 34
  • 41