2

We're using Google's security scanning and it makes queries with unique parameters on each scan. This causes new jsonPayload sub-fields to show up in our logs:

jsonPayload.params.utf8javascript_xssdetected_22312223341220000094915n_____javascript_javascript_______________noscript___title___textarea___style_

We sink these logs into BigQuery and because these are high-cardinality it causes the BigQuery table to run out of columns. I'd like to just drop all of these entries but I can't figure out the Google Cloud Logging query syntax to match fields on a pattern.

For example, I'd love to be able to exclude any lines where fields matching the following patterns exist (regex in the square brackets):

-jsonPayload.params.[.*xssdetected.*]:*
-jsonPayload.params.[.*metadata_google_internal.*]:*
-jsonPayload.params.[.*randmprmz.*]:*

Any idea if this is possible?

jwadsack
  • 5,708
  • 2
  • 40
  • 50
  • Have you tried using `!~` then double quote the patterns? See [examples](https://cloud.google.com/logging/docs/view/advanced-queries#example-regular-expressions). – Alex G Jun 30 '21 at 19:36
  • @AlexG from what I can tell that operator only works on the right-hand-side, that is, the value that we're filtering. I'm looking for something to filter on the name of the parameters. – jwadsack Jul 08 '21 at 19:50

0 Answers0