I'm new to airflow, really appreciate any help for the following problem. I tried to run the airflow webserver on my laptop.
Theoratically, I set the start_time=datetime.now(), it should be run successfully when I manually ran the dag on the webserver,but it changed overtime, it had been either queued or successful. Sometimes it was successful(but runtime is 00:00:00, and obviously my dag hadn't been ran), and sometimes it was just queued.
Here's the code in my DAG:
from datetime import datetime
from airflow import DAG
from airflow.models import Variable
from airflow.operators.python import PythonOperator
def get_var():
#a=Variable.get('abc')
print('abd')
with DAG(dag_id='test_var',start_date=datetime.now()) as dag:
task1=PythonOperator(task_id='var',python_callable=get_var)
However, every time when I check the Graph bar in the airflow webUI, it shows up as the below picture:
I'm not sure if it matters with the way I initialize airflow, I follow the below steps:
airflow webserver -p 8080
airflow db init --- These two steps worked, yet the third step ---
airflow scheduler
[2022-10-31 09:46:45,562] {scheduler_job.py:701} INFO - Starting the scheduler
[2022-10-31 09:46:45,562] {scheduler_job.py:706} INFO - Processing each file at most -1 times
[2022-10-31 09:46:45,565] {executor_loader.py:107} INFO - Loaded executor: SequentialExecutor
[2022-10-31 09:46:45,569] {manager.py:163} INFO - Launched DagFileProcessorManager with pid: 13315
[2022-10-31 09:46:45,570] {scheduler_job.py:1381} INFO - Resetting orphaned tasks for active dag runs
[2022-10-31 09:46:46,169] {settings.py:58} INFO - Configured default timezone Timezone('UTC')
[2022-10-31T09:46:46.172+0800] {manager.py:409} WARNING - Because we cannot use more than 1 thread (parsing_processes = 2) when using sqlite. So we set parallelism to 1.
[2022-10-31 09:46:46 +0800] [13314] [INFO] Starting gunicorn 20.1.0
[2022-10-31 09:46:46 +0800] [13314] [ERROR] Connection in use: ('::', 8793)
[2022-10-31 09:46:46 +0800] [13314] [ERROR] Retrying in 1 second.
[2022-10-31 09:46:47 +0800] [13314] [ERROR] Connection in use: ('::', 8793)
[2022-10-31 09:46:47 +0800] [13314] [ERROR] Retrying in 1 second.
[2022-10-31 09:46:48 +0800] [13314] [ERROR] Connection in use: ('::', 8793)
[2022-10-31 09:46:48 +0800] [13314] [ERROR] Retrying in 1 second.
[2022-10-31 09:46:49 +0800] [13314] [ERROR] Connection in use: ('::', 8793)
[2022-10-31 09:46:49 +0800] [13314] [ERROR] Retrying in 1 second.
[2022-10-31 09:46:50 +0800] [13314] [ERROR] Connection in use: ('::', 8793)
[2022-10-31 09:46:50 +0800] [13314] [ERROR] Retrying in 1 second.
[2022-10-31 09:46:51 +0800] [13314] [ERROR] Can't connect to ('::', 8793)
It turned out like this. Does this have anything to do with my DAG operation on the webUI? Thanks for your time and help!
I tried to search to another stackflow post about `[ERROR] Can't connect to ('::', 8793), but they only discussed about the webserver stuff, and also I'm not sure if the reason that my dag couldn't work is because of airflow scheduler
