1

I've managed to retrieve the execution parameter in my Airflow 1.10.15 dag but only under templated parameters (like PythonOperator and BashOperator) as described here

I need to use the execution parameter in an operator field that is not templated, namely GoogleCloudStorageToBigQueryOperator - field_delimiter and a few others.

Is it possible at all? Any Alternatives?

Thanks

Julian Larralde
  • 9
  • 2

1 Answers1

1

You can create your own operator with the same behavior that adds namespace to template_fields:

class MyGoogleCloudStorageToBigQueryOperator(GoogleCloudStorageToBigQueryOperator):
         template_fields = GoogleCloudStorageToBigQueryOperator.template_fields +('field_delimiter',)

Then simply use MyGoogleCloudStorageToBigQueryOperator as you are used to. It has all the abilities of GoogleCloudStorageToBigQueryOperator with additional functionality of field_delimiter as templated field.

Note: GoogleCloudStorageToBigQueryOperator is deprecated. You should import GCSToBigQueryOperator from google backport provider.

Elad Kalif
  • 14,110
  • 2
  • 17
  • 49