9

Below 3 metric from ingress controller is available in our prometheus , i am trying to understand the difference between this metrics.

My understanding,

request_duration - time between the request reached in nginx and the response send back to client.

ingress_upstream_latency - time waited in nginx before the upstream server/ pod accepted the request .

response_duration - time to get response from the backend container /pod

# HELP nginx_ingress_controller_request_duration_seconds The request processing time in milliseconds
# TYPE nginx_ingress_controller_request_duration_seconds histogram

# HELP nginx_ingress_controller_ingress_upstream_latency_seconds Upstream service latency per Ingress
# TYPE nginx_ingress_controller_ingress_upstream_latency_seconds summary

# HELP nginx_ingress_controller_response_duration_seconds The time spent on receiving the response from the upstream server
# TYPE nginx_ingress_controller_response_duration_seconds histogram

So as per above metric request_duration = ingress_upstream_latency+ response_duration

  1. when i did some test, i was able to see response_duration is more than request_duration, how that is possible , or by above understanding is wrong . I am trying to understand the difference between this two metrics nginx_ingress_controller_request_duration_seconds and nginx_ingress_controller_response_duration_seconds
Sanjay Gopinath
  • 101
  • 2
  • 3
  • I am also eager to get an answer on this! Did you conclude any results? Please share. – chaosguru Jan 20 '21 at 09:53
  • `nginx_ingress_controller_request_duration_seconds_bucket` should be "Total time for NGINX and upstream servers to process a request and send a response" and `nginx_ingress_controller_response_duration_seconds_bucket` is "The time spent on receiving the response from the upstream server". – Rohlik Jun 16 '22 at 10:14
  • @Rohlik - do you know of some documentation link for what you said? – Phil Jul 23 '22 at 00:50
  • 1
    @Phil I didn't find anything official, but I got inspiration from this https://github.com/SLOCloud/SLOC/blob/e7abbdb8572fcde456ff70423aac0c967555f0a9/testbeds/kubernetes/microk8s/base/ingress-nginx/grafana-dashboards/request-handling-performance.json#L59 – Rohlik Aug 09 '22 at 11:41

2 Answers2

0

FYI: nginx_ingress_controller_request_duration_seconds is request_time nginx var and it's docs say:

Time elapsed between the first bytes were read from the client and the log write after the last bytes were sent to the client.

So this one is about overall duration for the end client accessing nginx - i.e. affected by the client and his network. The other two are about communication of the nginx with the upstream/backend server.

You can't just add the 2 metrics you mentined to get the request_time, because there are other things like waiting for the client involved.

upstream_response_time can be up to several millis bigger than the request_time because of the different measuring method.

chipiik
  • 1,970
  • 15
  • 15
0

You can find the metrics documented in the nginx ingress controller docs in the Exposed Metrics section.

Quoting directly from there:

nginx_ingress_controller_request_duration_seconds Histogram\ The request processing (time elapsed between the first bytes were read from the client and the log write after the last bytes were sent to the client) time in seconds (affected by client speed).\ nginx var: request_time

nginx_ingress_controller_response_duration_seconds Histogram\ The time spent on receiving the response from the upstream server in seconds (affected by client speed when the response is bigger than proxy buffers).\ Note: can be up to several millis bigger than the nginx_ingress_controller_request_duration_seconds because of the different measuring method. nginx var: upstream_response_time

nginx_ingress_controller_ingress_upstream_latency_seconds Summary\ Upstream service latency per Ingress. Deprecated, use nginx_ingress_controller_connect_duration_seconds\ nginx var: upstream_connect_time