3

I'm trying to create a metric extracted from a string from some logs.

I'm creating a new Distribution Log Metric. I'm applying the filter to get the right logs. The filter is working as I can preview it.

Then I'm entering the Field name for my metric. No issue here as I can find it in the proposed values.

Now the field is returning me a string with some query parameters. I want to convert the value of one of these query parameters as a metric. I'm adding a Regular expression to do so.

My string looks like this: https://blabla.com/Name=Foo&Value=123&Version=14 and this is my regex Version=([^&]*)

As suggested I tried the following regex Version=([0-9]+). Output is the same.

but the metric isn't working. When opening the Metrics explorer I have the following message:

Only numeric metric data can be drawn as a line chart. the data provided cannot be drawn

Anything I'm missing here?

#edit1

I tried something simpler. I have a field which is returning only value. For example "348" if I'm looking at the log. I tried to build a Distribution Metric based on this field using the following regex ([0-9.]+) and I have the same issue. The output doesn't seems to be read as numeric data.

#edit2

Adding some illustrations

the log with the requestSize field

enter image description here

The setup of the log-based metric with the regex

enter image description here

and the output

enter image description here

Simon Breton
  • 2,638
  • 7
  • 50
  • 105
  • when you say "metric isnt working" - what actually happens ? what error or other output are you seeing, if any ? also, what is expected output when you are creating a simpler version of the metric above ? in which step the workflow breaks ? – c69 Jul 15 '21 at 14:49
  • @c69 yes sorry about that I've edited my question. There is no direct error (I wish there was!). It is just that no metric is displayed. What do you mean by `simpler version of the metric above`? – Simon Breton Jul 15 '21 at 15:08
  • sure - basically, "can you make a custom metric at all ?" (so it shows up in metrics explorer, etc), "can you make log based metric at all ?" , and then finally "when you are trying to make a regexp extraction in log based custom metric - what part is broken ?" – c69 Jul 15 '21 at 15:40
  • `Only numeric metric data` sounds like regexp extracts a piece of string and saves it as a string. I think logging had the function to covert strings to integers/number, but need to check... – c69 Jul 15 '21 at 15:42
  • https://cloud.google.com/logging/docs/logs-based-metrics/distribution-metrics#create-distribution it mentions `Extraction expression: (Optional) If Field name always contains a numeric value convertible to type double, then you can leave this field empty. Otherwise, specify a regular expression that extracts the numeric distribution value from the field value` and then example of the regexp is `([0-9.]+)` (notice the dot in their example that you probably DONT need. – c69 Jul 15 '21 at 15:46
  • so try `Version=([0-9]+)` as your regexp – c69 Jul 15 '21 at 15:47
  • 1
    I tried with a simpler metric and even with a field that return only numeric value It is not working (with or without regex) – Simon Breton Jul 15 '21 at 16:06
  • try switching the chart type to "Heatmap" (from default "Line chart"). `To display metrics with a distribution value, use heatmap charts. Heatmaps use color to represent the values in the distribution. With heatmaps, you can overline percentile lines and you can configure these charts to only display outliers.` https://cloud.google.com/monitoring/dashboards#example-heat – c69 Jul 15 '21 at 19:47

1 Answers1

2

As mentioned by @c69, you can plot metrics with distribution values by using a heat map. Heatmaps use color to represent the values in the distribution. With heatmaps, you can overline percentile lines and you can configure these charts to only display outliers.

If you want to use line charts, you must convert the histogram into a numerical value. One way to perform this conversion is to plot a specific percentile of the distribution.

The percentile values for distribution metrics are computed, and the algorithm depends on the bucket counts, the bucket widths, and the shape of the histogram:

  • The 50th, 95th, and 99th percentile values are always different. However, they might be showing different percentiles within the same bucket.

  • The percentiles aren't generated from measurements because these values aren't available.

  • The width of the bucket determines the maximum error between the computed percentile and the measurements.

  • The number of samples in a histogram is important. For example, if this number is fewer than 20, then the 95th and 99th percentiles are always in the same bucket.

  • For any distribution metric, you can use the Cloud Monitoring API to determine the bucket model used for that metric. Because this model is timestamped, a service can change the bucket model.

Refer to the link for more information about charting distribution metrics.

Srividya
  • 1,678
  • 3
  • 10