How to copy nginx logs of requests that took more than n seconds

Question

Consider that the access logs are of this format:

log_format detailed '$remote_addr $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for" ' '$request_length $request_time $upstream_connect_time $upstream_header_time $upstream_response_time';

How can I have a new log file that contains only requests that took more than 5 seconds?

score 1 · Answer 1 · answered Aug 08 '18 at 08:03

You can do this with built in if statemen on access_log directive like so:

http{
...
map $upstream_response_time $under {
   ~^[01234]\.[0-9]+ 1; 
   default 0; 
   } 

server {
   set $over 1;
   if ( $under = 1 ) { set $over 0; }
   access_log /some/folder/under.log combined if=$under;
   access_log /some/folder/over.log combined if=$over;
   ...
   }
...
}

score 0 · Answer 2 · answered Aug 08 '18 at 05:52

You can send logs to http-log and process them to figure out if anything is more than the time you are expecting it to.

Stopping logs based on some condition looks little complex to implement. But once logs are generated, extracting the required info from them seems correct approach.

How to copy nginx logs of requests that took more than n seconds

2 Answers2