Stack Overflow
Stack Overflow

Questions tagged [airflow-webserver]

161 questions
1
vote
0 answers

CeleryExecutor is not able to execute multi branch pipeline

The celery executor is not able to execute a multi-branch pipeline. The same which we tried from LocalExecutor it is able to execute multi branches. And Celery executor is creating one message for one DAG in Redis. is this expected…
Sanketha C R
  • 11
  • 3
1
vote
1 answer

Airflow: Task Not Found

I have a basic DAG setup with a python callable. The code inside the callable works fine and prints the log statement as well. It returns a list as expected. with DAG( ... catchup=False ) as dag: def get_delta_lean_articles(**kwargs): …
x89
  • 2,798
  • 5
  • 46
  • 110
1
vote
0 answers

Airflow 2.4.0 ghost dag runs showing up in UI

After updating my Airflow setup from 2.3.3 to 2.4.0, ghost dag_runs appeared on some of my dags. These dag_runs have seemingly random schedule dates from past and they can't be found from the db using the details view search and Run ID. What are…
MrBronson
  • 582
  • 11
  • 21
1
vote
1 answer

airflow.decorators not being recognized

I am a new user of Airflow. I successfully set up the airflow locally. But when I try to import modules such as from airflow.decorators or airflow.operators.pythonOperators I receive an error as Import could not be resolved. Below is the screenshot…
Aditya
  • 59
  • 7
1
vote
2 answers

"Google Cloud" connection not listed when adding new connection on Airflow Webserver

I installed the google provider with pip install 'apache-airflow[google]' and also tried pip install apache-airflow-providers-google But I can not find Google Cloud option when adding a new connection via airflow webserver, as shown below: All…
Blaze Tama
  • 10,828
  • 13
  • 69
  • 129
1
vote
1 answer

Cloud composer: Unable to view Admin menu in airflow UI even after rbac=True & rbac_user_registration_code=Op

I am setting up a composer environment, I have this IAM access: Composer Adminstator I have marked rbac=True & rbac_user_registration_code=Op but I am unable to see admin menu in airflow UI & also if I go to my profile in airflow UI it shows the…
scz34323
  • 11
  • 1
1
vote
0 answers

Airflow webser doesnt redirect https

The airflow created by official helm chart is not redirect https. It is running behind LoadBalancer with ingress control service. Here is my ingress controller apiVersion: extensions/v1beta1 kind: Ingress metadata: annotations: …
Felix G.
  • 11
  • 2
1
vote
0 answers

How to import external class inside the PythonVirtualEnvOperator Airflow?

I have a dag like below, from datetime import datetime from airflow import DAG from airflow.operators.python_operator import PythonOperator from airflow.operators.python_operator import PythonVirtualenvOperator import platform def get_info(): …
Dharsini
  • 11
  • 1
1
vote
1 answer

Add RUN_ID as part of airflow logs

I have been reading a lot about logging in to Airflow and experimenting a lot but could not achieve what I am looking for. I want to customize the logging for Airflow. I have a lot of DAGs. Each DAG has multiple tasks. EAch DAG has multiple tasks…
Abhishek Soni
  • 289
  • 2
  • 4
  • 15
1
vote
0 answers

Error while running the command airflow webserver

I am trying to run airflow apache in virtual environment in Ubuntu. I am using the below version Python 3.10.4 PIP 22.1 Airflow 2.3.0 I have installed airflow via WSL I am currently facing the issue when running the following command airflow…
Nikitha Singh
  • 11
  • 5
1
vote
0 answers

There's no DAGs on Airflow UI, even they are listed

I'm setting up Airflow 2.3.0 (Python 3.9.13) cluster on 4 nodes (on-prem infrastructure): node1, node2 run: web UI, scheduler, flower, and worker. node3, node4 run: worker. Now, I'm facing with problem that Web UI does not display any DAGs, even…
Hung Nguyen Thuan
  • 11
  • 5
1
vote
0 answers

Airflow webserver is getting terminated every few minutes

Airflow is configured in my ubuntu server and its webserver and scheduler are configured to run as system daemons. I am recently facing an issue where Airflow webserver is failing to start and is getting killed minutes after restart with the…
maverick_27
  • 25
  • 4
1
vote
0 answers

How does apache airflow schedule DAGs?

I created DAG with schedule_interval equals: "0 0 * * *". I know that airflow shows time in UTC format, but I see next run date in past and my DAGs didn`t run. Please help) I write this post at 15.03.2022 in 9.30. enter image description here DAG`s…
Макар Погорелов
  • 11
  • 1
1
vote
1 answer

Is there a way to view the 2 connected DAGs in one graph view?

I have 2 DAGS: example_call_other_dag which looks: The second task (TriggerDagRunOperator) call the second DAG: example_xcom which looks: Is there a way to view the 2 connected DAGs in one graph view ? I wan't see something like: Is it…
user3668129
  • 4,318
  • 6
  • 45
  • 87
1
vote
0 answers

Airflow DAG exists in dag_list, but cannot be triggered and does not show up in UI

I have the directory for my dag in the airflow/dags directory, and when calling airflow dags list while logged into the webserver, the dag's ID shows up in the list. However, calling airflow dags list while logged into the scheduler returns the…
green_rose
  • 11
  • 3
1 2
3
…
10 11