0

We have an internal app and I want to have the apache httpd server create new log files based on each of the client ips. In the documentation I see you can have multiple logs.

http://httpd.apache.org/docs/2.2/logs.html

LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog logs/access_log common
CustomLog logs/referer_log "%{Referer}i -> %U"
CustomLog logs/agent_log "%{User-agent}i"

but what I really want is something like this.

CustomLog logs/%h-access_log common

where the %h is extrapolated into the ip address i.e. 1.1.1.1-access_log

I imagine there's a mod_perl way to do this but just wondering if anyone knows a way to do something like this through standard configuration before I write a module.

jbrahy
  • 168
  • 10
  • Are you sure you can not achieve things another way, like postprocessing Apache log files ? Because having one logfile per client IP (are you thinking about IPv6 too ?) may mean a lot of (possibly tiny) files in the same directory, and various filesystems will have trouble managing tens or hundreds of thousands of files in the same directory. You may also hit the limit on the number of open files per (Apache) process. – Patrick Mevzek Mar 11 '17 at 19:43
  • Since it's an internal application we might have 1000 different servers posting to it at the most. Some of the servers post millions of records a day and some post 2 or 3. We need them to be separated so we can rotate the larger ones as necessary so we have at least the last 1000 transactions in the ip specific log file. – jbrahy Mar 12 '17 at 21:22

1 Answers1

5

Use the Apache log pipe feature: logs can be sent to an outside process. That process will have full latitude to filter things as it wants and create files as needed. See http://httpd.apache.org/docs/2.2/logs.html#piped

You will just need to write the small script that parse the log input to extract the IP address and forward the line to the appropriate file.

Otherwise if you sort of control the amount of clients, you could define all needed files beforehand, and then using conditional logging, through specific variables, see this example at the same URL:

# Mark requests from the loop-back interface
SetEnvIf Remote_Addr "127\.0\.0\.1" dontlog
# Mark requests for the robots.txt file
SetEnvIf Request_URI "^/robots\.txt$" dontlog
# Log what remains
CustomLog logs/access_log common env=!dontlog

But even with your latest comment, I still think postprocessing is a better route. YMMV.

Patrick Mevzek
  • 9,921
  • 7
  • 32
  • 43
  • I'd rather not have to spawn a process for each request. (I'm assuming that is what log piping does.) But if I don't get another answer before the bounty expires then I'll award it to you. – jbrahy Mar 13 '17 at 21:06
  • 2
    No the process is started once and gets data on STDIN continously. As written in the webpage : " Apache will start the piped-log process when the server starts, and will restart it if it crashes while the server is running. (This last feature is why we can refer to this technique as "reliable piped logging".)" – Patrick Mevzek Mar 13 '17 at 21:07
  • Also note: "As with conditional logging, piped logs are a very powerful tool, but they should not be used where a simpler solution like off-line post-processing is available." :-) – Patrick Mevzek Mar 13 '17 at 21:08
  • Didn't realized I didn't award the bounty when I marked this as the correct answer. – jbrahy Mar 18 '17 at 21:35