3

I am trying to understand the differences between the inclusion and exclusion filters in Google Cloud Logging, but there is very little documentation on how inclusion filters affects the Log Router/Sink. To illustrate, these are snippets that describe the information I can find on inclusion filters.

From Overview of Export Logs:

Inclusion filter: Selects which log entries to route through this sink. For inclusion filter examples, see Sample queries.

Exclusion filter: Selects which log entries to explicitly exclude from routing, even if the log entries match the sink's inclusion filter.

An excerpt from Exporting Logs from the Cloud Console

  1. Choose logs to include in the sink: Build an inclusion filter: Enter a filter to select the logs you want routed to the sink's destination.
  2. Choose logs to exclude from the sink (optional): Build an exclusion filter: Select Add exclusion and enter a filter to select logs you don't want routed to the sink's destination.

Thus, my questions are as listed below:

  1. It appears to me that these filters are overlapping in terms of functionality, so what exactly is the difference? (E.g. Are logs not ingested due to an inclusion filter identical to logs excluded via an exclusion filter? Are log entries not included by an inclusion filter also discarded in the same way as log entries excluded by an exclusion filter?)
  2. Why is there also the ability to determine exclusion filtering rates for exclusion filters? What purpose does it serve?

Edit: To further illustrate my point, there appears to be information indicating that some tangible differences between inclusion and exclusion filters used to exist, as seen on this article in Medium:

Each log entry that is received by Cloud Logging is compared against the exclusion filters and the inclusion filters. These comparisons are independent.

To determine if a log entry is exported to a destination, the log entry is compared to the log sink query of an inclusion filter. When a match occurs, the log entry is exported to the sink destination. A log entry might match multiple inclusion filters.

To determine if a log entry is discarded or saved in Cloud Logging storage, the log entry is compared to the exclusion filters. If it matches any exclusion filter, the log entry is discarded. Otherwise, the log entry is saved in Cloud Logging storage.

However, but there doesn't appear to be any similar information or even distinction between the filters on the latest version of the official documentation

Seng Cheong
  • 442
  • 3
  • 14
  • Think of it this way. You could say include `*.txt` but exclude `file.txt` (this is a logic example and not a real logging expression). Exclusion filters can limit what inclusion filters include. – John Hanley Jul 19 '21 at 21:47
  • Hi @JohnHanley, thanks for your reply. I had a look at the logging query language and saw that it supports Boolean operators in the inclusion and exclusion filters. Thus, am I right to say that I can create identical inclusion and exclusion filters e.g. `INCLUSION=*.txt AND NOT file.txt` as well as `EXCLUSION=NOT *.txt AND file.txt` ? – Seng Cheong Jul 20 '21 at 05:30
  • In addition, the language in the documentation and cloud console does seem to imply some distinction between these filters e.g. they note that for inclusion filter, if they are not provided when creating a sink, then there may be higher than expected resource usage. But I can't quite nail down if there is truly a difference between these filters. If they are identical, then won't providing either type of filters reduce resource usage anyway? – Seng Cheong Jul 20 '21 at 05:31

0 Answers0