NGINX log filter $upstream_response_time JSON ELK "-" parsefailure

Question

I have my NGINX logs formated as JSON:

log_format le_json '{ "@timestamp": "$time_iso8601", '
                   '"remote_addr": "$remote_addr", '
                   '"remote_user": "$remote_user", '
                   '"body_bytes_sent": "$body_bytes_sent", '
                   '"status": $status, '
                   '"request": "$request", '
                   '"request_method": "$request_method", '
                   '"response_time": $upstream_response_time, '
                   '"http_referrer": "$http_referer", '
                   '"http_user_agent": "$http_user_agent" }';

My log gets picked up by filebeat and sent to Logstash that have the following config:

input {
  beats {
    port => 5044
    codec => "json"
  }
}
filter {
    geoip {
      database => "C:/GeoLiteCity.dat" 
      source => "[remote_addr]"
          }
}
output {
  elasticsearch {
    template => "C:/ELK/logstash-2.2.2/templates/elasticsearch-template.json"
    template_overwrite => true
    hosts => ["127.0.0.1"]
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

The problem i'm having is $upstream_response_time. When there is no response time NGINX puts an '-' on this post. As you can see i don't put "" around $upstream_response_time because i want it as a number so i can perform calculations with this in Kibana and display. When '-' is sent i get a jsonparsefailure in Logstash because it is not a number.

I would like to set all the '-' to 0. What would be the best way to do this? I've had no success with trying to filter it in nginx-config. I think it needs to be done prior to getting shipped to Logstash because that's where the parsefailure occurs.

Any ideas?

As a workaround hack you could put a zero after it. So it either become `0.1230` or `-0`, both are perfectly valid numbers :) — Alexey Ten, Apr 22 '16 at 08:45
You don't need slashes in regexp, and you even don't need regexp at all. — Alexey Ten, Apr 22 '16 at 11:59

Alexey Ten · Accepted Answer · 2016-04-22T13:10:43.677

5

Try this:

map $upstream_response_time $temprt {
  default $upstream_response_time;
  ""      0;
}

$upstream_response_time either a number or unset. Nginx logs unset variables as dash (-), but map treats them as empty strings.

edited Apr 22 '16 at 13:10

answered Apr 22 '16 at 12:00

Alexey Ten

13,794
6
44
54

Thanks! This worked: `map $upstream_response_time $temprt { default $upstream_response_time; "" 0; }` Seems like it was empty and somewhere behind the scenes it's set to "-" if thats the case. – Jesper Persson Apr 22 '16 at 12:45
Thanks, fixed answer – Alexey Ten Apr 22 '16 at 13:10
@JesperPersson nginx replaces unset variables with dashes in log module https://github.com/nginx/nginx/blob/master/src/http/modules/ngx_http_log_module.c#L957-L960 – Alexey Ten Apr 22 '16 at 13:16

score 0 · Answer 2 · answered Jul 17 '18 at 20:06

A fleshed out example from @AlexeyTen using null instead of 0 to differentiate from real values since I wasn't 100% sure how to use mapped variables:

map $upstream_response_time $temprt {
  default $upstream_response_time;
  ""      null;
}

log_format le_json '{ "@timestamp": "$time_iso8601", '
               '"remote_addr": "$remote_addr", '
               '"remote_user": "$remote_user", '
               '"body_bytes_sent": "$body_bytes_sent", '
               '"status": $status, '
               '"request": "$request", '
               '"request_method": "$request_method", '
               '"response_time": $temprt, '
               '"http_referrer": "$http_referer", '
               '"http_user_agent": "$http_user_agent" }';

NGINX log filter $upstream_response_time JSON ELK "-" parsefailure

2 Answers2