Im trying to send the following logs to loki using promtail, here is an example of the log line 2023-04-13 09:09:49,762 | 2a81c3e799f446c5800c954c38a67c8c | INFO | test | {'foo':'bar'}
My promtail config is as follows:
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: 'http://loki:3100/loki/api/v1/push'
scrape_configs:
- job_name: scrape_django
static_configs:
- targets:
- localhost
labels:
job: django_logs
__path__: /var/log/app.log
pipeline_stages:
- regex:
expression: '(?P<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3}) \| (?P<request_id>[a-fA-F0-9]{32}) \| (?P<level>\w+) \| (?P<value>\w+) \| (?P<data>\{.*\})'
- timestamp:
source: time
location: Asia/Kolkata
format: '2006-01-02 15:04:05,000'
- labels:
level:
when im trying to access the label using job='django_logs', im getting the following error:
Empty results, no matching label for {job="django_logs"}
The pipeline stages using --inspect gives me the following output:
{stages.Entry}.Extracted["level"]:
+: INFO
{stages.Entry}.Extracted["request_id"]:
+: 2a81c3e799f446c5800c954c38a67c8c
{stages.Entry}.Extracted["time"]:
+: 2023-04-13 09:09:49,76
{stages.Entry}.Extracted["value"]:
+: test
[inspect: labels stage]:
{stages.Entry}.Entry.Labels:
-: {filename="/var/log/app.log", job="django_logs"}
+: {filename="/var/log/app.log", job="django_logs", level="INFO"}
{stages.Entry}.Extracted["data"]:
+: {'foo':'bar's}
If I remove the timestamp stage it is working
