How can prometheus start reporting on path params as well?

Question

Background:

We have an application which acts as a gateway. Every URL served by the platform goes through this. URI is /v2/{endPoint}. With endpoint, there are mappings to hit the services URL accordingly.

All our applications are spring-boot applications. We have added micrometer and actuator dependencies in the pom.xml.

Problem Statement:

We want to add monitoring in this layer using prometheus to see how different services are performing. However, every service URL would only be reported using /v2/{endPoint}. However, we only have finite set of values for endpoint and would want it to be used for reporting.

Currently, URL used for reporting in prometheus/grafana is - /v2/{endPoint}

We want this to be individual urls such as -

/v2/account
/v2/customer

/account leads to account-service. /customer woould lead to customer service.

Is there any way in prometheus to do so ?

Answer 1

I think you can solve this inside you Spring-Boot application by manipulating the tags for the http_server_requests metric.

The http_server_requests metric is created by a Timer.Sample in the WebMvcMetricsFilter and the tags are added by the DefaultWebMvcTagsProvider.

If you override the DefaultWebMvcTagsProvider you are able to add any info out from the HttpServletRequest as a new tag OR override the existing uri tag.

For instance, a new tag with the getRequestURI value which I think should solve your problem. Then you can apply any filtering of the tag value to get it exactly as you want it.

I did a test with where I have both custom-uri="/rest/customer/1" and uri="/rest/customer/{id} as a tag.

http_server_requests_seconds_count{exception="None",custom-uri="/rest/customer/1",host="customer-service",method="GET",outcome="SUCCESS",status="200",uri="/rest/customer/{id}",} 2.0

Example code:

The WebMvcTagsProvider:

public class CustomWebMvcTagsProvider extends DefaultWebMvcTagsProvider 
{

  public CustomWebMvcTagsProvider(boolean ignoreTrailingSlash, List<WebMvcTagsContributor> contributors) {
    super(ignoreTrailingSlash, contributors);
  }

  @Override
  public Iterable<Tag> getTags(HttpServletRequest request, HttpServletResponse response, Object handler,
  Throwable exception) {
    var tags = super.getTags(request, response, handler, exception);
    // Override the existing uri tag by using "uri" as key name
    return Tags.of(tags).and("custom-uri", request.getRequestURI());
  }
}

and the Configuration file

@Configuration()
@EnableConfigurationProperties(MetricsProperties.class)
public class CustomWebMvcMetricsAutoConfiguration {

    private final MetricsProperties properties;

    public CustomWebMvcMetricsAutoConfiguration(MetricsProperties properties) {
        this.properties = properties;
    }

    @Bean
    public CustomWebMvcTagsProvider webMvcTagsProvider(
        ObjectProvider<WebMvcTagsContributor> contributors) {
        return new CustomWebMvcTagsProvider(this.properties.getWeb().getServer().getRequest().isIgnoreTrailingSlash(),
            contributors.orderedStream().collect(Collectors.toList()));
    }

}