Stack Overflow
Stack Overflow

Questions tagged [airflow-webserver]

161 questions
1
vote
0 answers

Cannot find DAG on airflow dags list

enter image description here It's the first time I tried to create airflow dag but when I checked the dag lists in terminal, there is no dag_id that I created Is there anything wrong with my file or the directory of the dag the I created
Rattapong Pojpatinya
  • 11
  • 1
1
vote
1 answer

How to get the initial start-time and end-time for Airflow DAG, including all tasks

How can I get the start and end time of the DAG in overall, which includes all the tasks (which is the initial task start time and end time of the last task)?
Austin Jackson
  • 153
  • 6
1
vote
0 answers

Apache Airflow Docker Swarm Operator tasks stuck in running

I'm using Airflow for running tasks in docker swarm using DockerSwarmOperator. After DockerSwarmOperator task gets completed, the task remains running in Airflow while the corresponding container has been exited. Here is my docker-compose…
Ata Attarian
  • 11
  • 1
1
vote
0 answers

How to identify airflow dependent DAGS and Tasks through airflow rest api

i have some dependent dags (parent and child dags). How to i identify those dags are dependent , How to classify whether dag is dependent or not. im using airflow version 2.4.0 How to get dependent dags details through airflow rest api. Ref : How to…
vbala vbala
  • 15
  • 3
1
vote
0 answers

Airflow ignoring redirect_uri with custom provider

I', trying to setup Airflow Authenticate with my custom provider but for some reason, it is ignoring the redirect URI which I have set and always use https://my-airflow/oauth-authorized/my provider which I can see in the payload. It should…
h00jraq
  • 21
  • 2
1
vote
0 answers

Why airflow marks kubernetes pod operator tasks in airflow as failed on the UI while the containers are still running on k8s?

I am having a for loop in my DAG which creates some tasks using kubernetes pod operator. These tasks run in parallel and there are enough resources available on the k8s nodes to let them run in parallel. These containers share an EFS volume as a PVC…
Naxi
  • 1,504
  • 5
  • 33
  • 72
1
vote
0 answers

Using Airflow, How to handle asynchronous API calls with Airflow Tasks?

I'm quite new to Airflow, I need to make asynchronous POST API calls to start the execution of the external service. And Need to make GET API calls to check the status of the execution and have to make that call until the execution gets completed. I…
saravana kumar
  • 255
  • 1
  • 3
  • 10
1
vote
0 answers

Airflow logs are empty in UI

I have a DAG that ran for 2 hours but I can not see any log in the UI. In addition, I can't see how many attempts there were (or if there were any at all). In the airflow webserver pod (where the UI runs), I can see some logs, but they are not…
simple_developer
  • 43
  • 5
1
vote
0 answers

Docker Ports and Airflow Tasks Test

I'd like to ask what's the meaning of this PORT setup? enter image description here Sometimes I get the "5555/tcp, 8793/tcp" part and sometimes do not. So what is the function of "5555/tcp, 8793/tcp" and how does it appear when I build my Docker…
ypangalela
  • 11
  • 2
1
vote
0 answers

Airflow localhost:8080 does not load (using docker-compose.yaml & celery executor)

I am trying to run airflow webserver at localhost:8080, but the URL never finishes loading and the airflow UI cannot be seen. My docker-compose.yaml looks like below. Please note I am using a remote database server as my Airflow & Celery backend…
thesteadyprogrammer
  • 21
  • 4
1
vote
1 answer

Issue while using Tableau Operator in Airflow

I am using AWS MWAA ver 2.2.2 and trying to use the Tableau operator to refresh a workbook. I created a Tableau connection using a username and password. In the Tableau operator I passed the following parameters …
hemant A
  • 185
  • 14
1
vote
1 answer

Postgres database refusing connection from Airflow: Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections port 5432?

I have an existing database (Postgres) that i want to connect to apache-Airflow on my host machine(Windows 10), I installed the apache-airflow on the WSL running ubuntu. The installation was smooth and working fine since i was able to get the…
ABUBAKAR MUHAMMED MUKTAR
  • 21
  • 4
1
vote
0 answers

Get files from sharepoint

My goal is to get files from sharepoint for every x minute based on their names and if possible also on creation/modified datetime. I have a link to sharepoint where files are stored. I was unsucsefully trying to find some example on internet but…
Henry
  • 537
  • 1
  • 9
  • 22
1
vote
2 answers

Why Do I get an error 403 from the GCP Composer Airflow?

Yesterday, we set up an airflow environment in the GCP composer. However, when I click on the Airflow web UI link, I get an error 403, with the note: '403. That's an error. That's all we know' I cannot seem to solve the issue. I have tried…
MC7836
  • 84
  • 8
1
vote
0 answers

pass task instance variables as op_args

I want to use my function_two multiple times with different values for the my_list vairable. I know I can use returned variables from one function into another using the xcom pull function. However, if I pull, I have to do it from within the python…
x89
  • 2,798
  • 5
  • 46
  • 110
1
2
3
…
10 11