2

IN Data fusion pipeline of GCP, the GCSFile properties having a field named "Regex path filter". How does it work?. I don't get proper documentation on this.

madhusudan
  • 21
  • 1

1 Answers1

2

You can find the regex documentation here.

How does it work? It is applied to the filenames and not to the whole path.

For example, lets say you have the following path: gs://<my-bucket>/<my/complete/path>/ and you have some CSV and JSON files inside this path.

To filter only the CSV files you would use the regex .*\.csv

Please note that this regex will only filter what starts after your path.

Black_Bacardi
  • 324
  • 4
  • 10
rmesteves
  • 3,870
  • 7
  • 23