Remote Apache Logging

Question

I have been researching remote logging of apache logs, and everything I have found involves piping to logger.

Two questions:

Is there any other way to remotely log apache logs?
Is piping to logger stable?

Thank you!

Jeff

In your research, what you want to be looking for is something called a 'log shipper'. When considering which log-shipper to use, you should look at what other logs you might like to ship (eg. do you want windows event logs? do you care about having to have Java installed for some log-shippers? do you want to ship logs as json? ... — Cameron Kerr, Apr 12 '15 at 14:05

score 2 · Answer 1 · answered Apr 06 '13 at 00:05

We use logstash, an open source log aggregating tool. It can pull in Apache logs from multiple machines and also logs from many other applications. The best approach is to format the output of apache httpd logs in json format with something like this:

LogFormat "{ \"@timestamp\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \"@fields\": { \"client\": \"%{X-Forwarded-For}i\", \"virtualhost\": \"%{Host}i\", \"duration_usec\": %D, \"status\": %s, \"request\": \"%U%q\", \"method\": \"%m\", \"referrer\": \"%{Referer}i\" } }" logstash_json

Then install logstash and have it collect all your logs into a central server. It is reliable, survives network outages (logstash remembers where it got up to) and has a very pretty UI with searching capability.

score 1 · Accepted Answer · answered Apr 05 '13 at 23:15

1

1) As indicated by the tag, rsyslog can do this. Write the logs to a file and use http://www.rsyslog.com/doc/imfile.html .

2) Yes. logger is a stable program apache's piping mechanism is stable (e.g., piping to rotatelogs has been in use for quite a while).

answered Apr 05 '13 at 23:15

Mark Wagner

18,019
2
32
47

I think you need at least version 7 of rsyslog to tail a file. It is unfortunate that some distribution version in current lifecycle have version 5. – Cameron Kerr Apr 12 '15 at 14:02

score 1 · Answer 3 · edited Oct 07 '21 at 06:58

Is piping to logger stable?

Many many programs pipe their output to logger. It is as stable as any other piece of software.

logger is simple and just works. logger makes use of your existing syslog configuration. It doesn't need any tweaks to syslog or rsyslog, and there is no need to grant superuser to users who need logs. Logger sends logs to syslog on the local system, and syslog handles the rest.

Note that syslog historically uses UDP, and UDP is intended to be a low-overhead, 'unreliable' protocol. See RFC5426 - "Transmission of Syslog Messages over UDP", section "4. Reliability Considerations". If you want reliable syslog, use TLS, see RFC5425, Transport Layer Security (TLS) Transport Mapping for Syslog.

Remote Apache Logging

3 Answers3