1

Scenario: The logs are in following format:

<SequenceID> <Level> <Message>

I have a requirement to sort the logs based on the SequenceID at Grafana.

Background: I am using promtail to ship logs where I create labels for SequenceID and Level. However, I am unable to find any valid visualization/transformation option (panel/widget) at Grafana Dashboard that would help me in creating a table where I can sort the log entries based on the sequenceID.

Also note: The logs are generated at microsecond level, hence can't rely on timestamp.

PS: When adding transformation "Labels to fields" for the label "SequenceID", I get below error: Pls note:

  • There are multiple values in the dropdown below.
  • There's only one value in the table where it doesn't make sense to sort. Sorting based on Label in Grafana
H S Raju
  • 380
  • 3
  • 12
  • just a note, having a sequence_id label is probably not a good design if you have many values for that. You're going to have a log stream explosion which will impact performance: https://grafana.com/docs/loki/latest/best-practices/ – Dan Dinu Dec 06 '21 at 21:07
  • I agree. Too many label and value combination leads to too many streams - This results in large index and small chunks in log store. This in turn reduces the performance. Any best practice/suggestions to overcome above? – H S Raju Dec 07 '21 at 10:44
  • I noted there's a major difference between specifying labels at log processing time in Log shipper (e.g. Promtail) vs extracting label values through a pattern parser. The latter is at Log query time and doesn't result in high cardinality. This solution may not be best fit for above case but still is a noticeable point. Reference: https://grafana.com/blog/2021/08/09/new-in-loki-2.3-logql-pattern-parser-makes-it-easier-to-extract-data-from-unstructured-logs/ – H S Raju Dec 17 '21 at 08:01

3 Answers3

1

Do the following steps:

  • Select "Table" visualization

  • Select "Label to fields" transformation

  • Click in the "sequenceID" column name to change the order.

Marcelo Ávila de Oliveira
  • 19,950
  • 3
  • 39
  • 50
  • Hi @Marcelo, Please find the screenshot updated in the original question. The issue is : There are multiple values in the dropdown below. There's only one value in the table where it doesn't make sense to sort. Sorry, couldn't attach image in comment. – H S Raju Dec 06 '21 at 07:50
0

Solution: Selected "Table" visualization. The transformation steps are as below:

  • Reduce

    Mode: Series to rows

    Calculations: "Step"

    Enable - Label to fields

Further, I used Filter data values and Filter by name transformation stages to filter the empty rows and remove a few columns.

Transformation for ordering of logs based on label

H S Raju
  • 380
  • 3
  • 12
0

Sort Example

Step 1: Choose a new dashboard
enter image description here
Step 2: Add a new panel
enter image description here
Step 3: query command (example: {job="localhost_logs"} |= "GGGG") and choose the switch to table
enter image description here
Step 4: Transform (Extract fields)
enter image description here
enter image description here
Step 5: Add transformation(Organize fields)
enter image description here

Willie Cheng
  • 7,679
  • 13
  • 55
  • 68