0

I am designing a custom sensor, MySensor, in Airflow 2 which i am creating by overriding the BaseSensorOperator. I have a parameter called file which is passed to my constructor as a template field

template_fields = ("file",)

I have an Airflow variable named file which holds the value for the parameter file.Then i instantiate a task in my DAG as follows.

my_task = MySensor(task_id="my_task", file = "{{var.value.file}}")

This works correctly as it is a template field and the value stored in the Airflow variable will be used. But the problem arises when I pass this argument as a string, that is without a Jinja template.

my_task = MySensor(task_id="my_task", file = "/my_test_file.jar")

This gives me an error AttributeError: 'MySensor' object has no attribute 'file'.

Is there something I am missing here? How can I pass non templated arguments to template field in Airflow 2

Olaf Kock
  • 46,930
  • 8
  • 59
  • 90
sanchit08
  • 119
  • 1
  • 7
  • Is `file` declared within `__init__()` of `MySensor`? – NicoE Nov 17 '21 at 11:28
  • @NicoE You mean if i have initialised it using `self.file = file` – sanchit08 Nov 17 '21 at 11:34
  • Yes, like this: https://airflow.apache.org/docs/apache-airflow/stable/howto/custom-operator.html#templating – NicoE Nov 17 '21 at 11:54
  • 1
    @NicoE yes. i solved the issue. the problem was i passed `file` to another dictionary, `params`, which was not templated. Apparently I had to provide `params` as templated as well, only then would it work. So if my understanding is that if you have to pass templated fields to a data structure, then the data structure itself has to be given under `template_fields`. I dont know if this was mentioned in the Airflow docs, or maybe I missed it – sanchit08 Nov 18 '21 at 08:25

0 Answers0