0

I am trying to upload a DAG defined with taskflow api on google cloud composer but no matter what I do, it doesn't show up in the DAGs list, neither in airflow UI or DAGs list provided by composer.

Another strange thing is that if I upload the file with errors, these show up in composer.

e.g. `

@dag(
    schedule=None,
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
    catchup=False,
    tags=["example"],
)

`raises an error about 'schedule' not existing. After changing it to schedule_interval the error is not raised anymore.

I haven't found anything mentioning taskflow api wouldn't work on composer so far but this is my suspicion.

Also, running this on local docker compose setup works as expected.

So, could someone confirm if cloud composer doesn't work with taskflow api, or am I doing something wrong?

razvanv
  • 53
  • 6
  • Which version of airflow are you [using](https://airflow.apache.org/docs/apache-airflow/2.0.0/concepts.html#taskflow-api)? Cloud composer will take a few minutes to list your dags and can you refresh the page. Are you following this [document](https://cloud.google.com/composer/docs/run-apache-airflow-dag)?. – kiran mathew Dec 21 '22 at 11:59
  • it's airflow 2 and I don't think it's because of the refresh timer. I had a taskflow dag for 30 hours in and still no sign of it. [This](https://airflow.apache.org/docs/apache-airflow/stable/tutorial/taskflow.html) is the dag that I used – razvanv Dec 21 '22 at 13:22
  • Can you modify the last line by changing `tutorial_taskflow_api()` to `dag=tutorial_taskflow_api()`. After modifying it did appear in my ui dag’s list. – kiran mathew Dec 22 '22 at 07:24

1 Answers1

0

I had this issue when the name of the file containing the DAG was the same as the method I was using to define and call the DAG, I renamed the method so that it didn't match/collide and then the DAG appeared in my list

tim busfield
  • 41
  • 3