I want to monitor a local log file with promtail. However the software which writes to the logfile always ends the log file with "\r\n**********\r\n" (new line, 10 asterisks and new line). The problem is that next time when the software write to the logfile the software basically replaces the asterisks with the new log row and when finished writing the log, adds the row of asterisks to the end.
The problem is of course that promtail's offset is 11 characters too far ahead when the logging is resumed and so I lose the first 11 characters every time. Combine this with multiline stage (the log can contain stack traces) and you can see how it's not working as one would hope.
Is there a way around this, like configure promtail to always move its offset 11 steps backwards each time?
Exert from promtail config:
scrape_configs:
- job_name: system
pipeline_stages:
- multiline:
firstline: '^\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}\.\d{3}\|'
max_wait_time: 1s
- regex:
expression: '(?P<timestamp>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}\.\d{3})\|[\w\.]+ (?P<version>[\d\.]+)\|\d+\|\d+\|[\w:]+\|(?P<level>\w+)\|-?\d+\|(?P<log_message>(?s:.+))'
- labels:
timestamp:
version:
log_message:
- timestamp:
source: timestamp
format: 2006/01/02 15:04:05.999
