0

Because I can't use the airflow CLI, I'm actually parsing scheduler logs with grep on airflow1 in order to retrieve some infos such as :

  • check if the dag is triggered or not / if it's successful or not / start timestamp with the pattern "INFO Marking run" :

[2021-12-01 11:06:50,340] {logging_mixin.py:112} INFO - [2021-12-01 11:06:50,339] {dagrun.py:307} INFO - Marking run <DagRun prd_*** @ 2021-12-01 10:02:00+00:00: scheduled__2021-12-01T10:02:00+00:00, externally triggered: False>successful

  • when the dag is not triggered, I use the pattern 'INFO - Created' to retrieve the dag' start timestamp :

[2021-12-01 11:04:49,213] {scheduler_job.py:1298} INFO - Created <DagRun prd_*** @ 2021-12-01T10:02:00+00:00: scheduled__2021-12-01T10:02:00+00:00, externally triggered: False>

It works well on airflow1 but I can't find those data in the airflow2 scheduler logs after migration.

Does the configuration need to be changed ?

Regards, Troubadour

Troubadour
  • 1

1 Answers1

0

You should use Airflow 2 REST API instead.

It was precisely done so that you do not have to parse logs. https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html

Jarek Potiuk
  • 19,317
  • 2
  • 60
  • 61