Different Prometheus scrape URL for every target

Question

Every instance of my application has a different URL. How can I configure prometheus.yml so that it takes path of a target along with the host name?

scrape_configs:
- job_name:       'example-random'

# Override the global default and scrape targets from this job every 5 seconds.
scrape_interval: 5s

static_configs:
  - targets: ['localhost:8090','localhost:8080']
    labels:
      group: 'dummy'

score 60 · Accepted Answer · answered Jul 06 '17 at 12:30

You currently can't configure the metrics_path per target within a job but you can create separate jobs for each of your targets so you can define metrics_path per target.

Your config file would look something like this:

scrape_configs:
- job_name:       'example-target-1'
  scrape_interval: 5s
  metrics_path: /target-1-path-to-metrics
  static_configs:
    - targets: ['localhost:8090']
      labels:
        group: 'dummy'

- job_name:       'example-target-2'
  scrape_interval: 5s
  metrics_path: /totally-different-path-for-target-2
  static_configs:
    - targets: ['localhost:8080']
      labels:
        group: 'dummy-2'

Is there any valid reason why this feature is not implemented ? it's a common use case — Olivier Boissé, Mar 15 '23 at 09:07

score 24 · Answer 2 · answered Apr 03 '18 at 18:16

I achieved this by using file_sd_config option. All targets are described in separate file(s), which can be either in YML or JSON format.

prometheus.yml:

scrape_configs:
  - job_name: 'dummy'  # This will be overridden in targets.yml
    file_sd_configs:
      - files:
        - targets.yml

targets.yml:

- targets: ['host1:9999']
  labels:
    job: my_job
    __metrics_path__: /path1

- targets: ['host2:9999']
  labels:
    job: my_job  # can belong to the same job
    __metrics_path__: /path2

You can use **__metrics_path__** without **file_sd_configs**, just in the **static_configs** — Demiurg, Nov 23 '18 at 06:15

Venkat · Answer 3 · 2019-04-09T03:21:08.017

7

This is the configuration I used to get prometheus up and running.

Prometheus Endpoint : http://localhost:8080/appcontext/v1/actuator/prometheus

Configuration: Add the below config under /etc/prometheus/prometheus.yml

- job_name: 'appdev'
    scrape_interval: 5s
    metrics_path: /appcontext/v1/actuator/prometheus
    static_configs:
      - targets: ['localhost:8082'] 
        labels:
          group: 'appdev'

edited Apr 09 '19 at 03:21

answered Mar 12 '19 at 03:53

Venkat

314
3
10

Why this Prometheus Endpoint is running on port 8080, but target is on 8082. It don't have to be the same? – Robert Głowacki Jun 10 '21 at 12:19

score 6 · Answer 4 · answered Jul 05 '17 at 14:21

6

I believe you need to do some relabelling of the __metrics_path__ label set to include the varying paths of your applications.

The Prometheus configuration docs will prove useful to you here and this article should help you understand relabelling a little better.

answered Jul 05 '17 at 14:21

Conor

3,279
1
21
35

1

Thanks to your comment, i solved my problem of updating scrape URL on fly. Too bad it is hidden under complex explanations in the docs while i think this should be clearly stated as a feature. – ishan Oct 05 '17 at 10:14
2

For people who want an example, here is one: https://stackoverflow.com/questions/59866342/prometheus-dynamic-metrics-path – btwiuse Jul 11 '20 at 12:28

Different Prometheus scrape URL for every target

4 Answers4