Questions tagged [airflow-taskflow]
67 questions
0
votes
1 answer
Airflow 2 taskflow logging
I'm creating a fairly complex task using the task-flow API (@task annotation) and I need to write some logs which will help me troubleshooting the runs during development and in the future.
I've researched the subject and found nothing in the…
cosenmarco
- 125
- 9
0
votes
1 answer
Naming Airflow dags other then the python callable when using taskflow api
I trying to create multiple dags using the taskflow API that have a variable passed into them which can be used by tasks within the dag
For example I am trying to have this code
from airflow.decorators import dag, task
from datetime import…
Nicholas Stevens
- 505
- 4
- 8
0
votes
1 answer
How to specify function for on_failure with taskflow api
How can I specify a function to run when a DAG fails using the taskflow api? Using the old style I am able to specify a function to run on_failure but I cannot figure out or find documentation to do it using the taskflow api with the DAG and task…
Chromey
- 97
- 1
- 11
0
votes
1 answer
Airflow dynamically genarated task not run in order
I have created dynamic tasks generation dag. Tasks are generated accurately, But those tasks are not trigger in order,not work in consistently.
i have noticed it triggered on alphanumeric order.
Let's check run_modification_ tasks. i have generated…
Nalinda Perera
- 67
- 8
-1
votes
2 answers
Can we upload file through airflow web-server UI and trigger a DAG?
I need to upload a log file through the airflow web server UI and parse that log file in a DAG.
My requirement is a button on UI, which upon clicking will open a file selector to upload a file to airflow. then use this file in the DAG.
Similarly, we…
Prinz Piuz
- 69
- 9
-2
votes
2 answers
Airflow Task Parallel and waiting for another to complete
Hi I have a dag that has some tasks that I need to run parallels then other tasks that have to wait until the parallel tasks all finish before it can start.
Here I've created a diagram of what I want. A1 and B1 to start at the same time when the…
unnest_me
- 119
- 1
- 2
- 9