2

Ok. We have Promtail to gather text log files and send them to loki.
Also I can create MySQL data source to query data from database for visialization purposes.

I am curious. Is it possible to configure Loki/Promtail or there is some logging driver which allows Loki to access them.

Why I am asking about this. Because our legacy system stores logs of our application into database into (ID, time, text) fields. So I am trying to find a way to import these logs into Loki or to configure Promtail to fetch logs not from text files but from this database table.

Is that possible?

Eugen Konkov
  • 22,193
  • 17
  • 108
  • 158

2 Answers2

1

I'm afraid that's not possible but you can do some workarounds for eg. dump the log messages into a text file and have Promtail read `em or use the Loki Push API to push logs to Loki.

Dan Dinu
  • 32,492
  • 24
  • 78
  • 114
  • Why you say that is not possible? that is. – Eugen Konkov Jun 22 '22 at 10:10
  • 1
    Sorry, I meant that you cannot configure promtail to scrape a database table for logs, I thought that's what you were asking. However, you can do that using Loki's Push API as I mentioned above. – Dan Dinu Jun 22 '22 at 12:09
1

This is possible.

Found how to post log to Loki directly:

/loki/api/v1/push is the endpoint used to send log entries to Loki. The default behavior is for the POST body to be a snappy-compressed protobuf message. Alternatively, if the Content-Type header is set to application/json, a JSON post body can be sent in the following format:

{
  "streams": [
    {
      "stream": {
        "label": "value"
      },
      "values": [
          [ "<unix epoch in nanoseconds>", "<log line>" ],
          [ "<unix epoch in nanoseconds>", "<log line>" ]
      ]
    }
  ]
}

Here is blog post how to do that with python: https://medium.com/geekculture/pushing-logs-to-loki-without-using-promtail-fc31dfdde3c6#8290

Eugen Konkov
  • 22,193
  • 17
  • 108
  • 158