Airflow2 gitSync DAG works for airflow namespace, but not alternate namespace

Question

I'm running minikube to develop with Apache Airflow2. I am trying to sync my DAGs from a private repo on GitLab, but have taken a few steps back just to get a basic example working. In the case of the default "airflow" namespace, it works, but when using the exact same file in a non-default name space, it doesn't.

I have a values.yaml file which has the following section:

dags:
  gitSync:
    enabled: true
    repo: "ssh://git@github.com/apache/airflow.git"
    branch: v2-1-stable
    rev: HEAD
    depth: 1
    maxFailures: 0
    subPath: "tests/dags"
    wait: 60
    containerName: git-sync
    uid: 65533
    extraVolumeMounts: []
    env: []
    resources: {}

If I run helm upgrade --install airflow apache-airflow/airflow -f values.yaml -n airflow, and then kubectl port-forward svc/airflow-webserver 8080:8080 --namespace airflow, I get a whole list of DAGs as expected at http://localhost:8080.

But if I run helm upgrade --install airflow apache-airflow/airflow -f values.yaml -n mynamespace, and then kubectl port-forward svc/airflow-webserver 8080:8080 --namespace mynamespace, I get no DAGs listed at http://localhost:8080.

This post would be 10 times longer if I listed all the sites I hit trying to resolve this. What have I done wrong??

UPDATE: I created a new namespace, test01, in case there was some history being held over and causing the problem. I ran helm upgrade --install airflow apache-airflow/airflow -f values.yaml -n test01. Starting the webserver and inspecting, I do not get a login screen, but it goes straight to the usual web pages, also does not show the dags list, but this time has a notice at the top of the DAG page:

The scheduler does not appear to be running.
The DAGs list may not update, and new tasks will not be scheduled.

This is different behaviour yet again (although the same as with mynamespace insofar as showing no DAGs via gitSync), even though it seems to suggest a reason why DAGs aren't being retrieved in this case. I don't understand why a scheduler isn't running if everything was spun-up and initiated the same as before.

Curiously, helm show values apache-airflow/airflow --namespace test01 > values2.yaml gives the default dags.gitSync.enabled: false and dags.gitSync.repo: https://github.com/apache/airflow.git. I would have thought that should reflect what I upgraded/installed from values.yaml: enable = true and the ssh repo fetch. I get no change in behaviour by editing values2.yaml to dags.gitSync.enabled: true and re-upgrading -- still the error note about scheduler no running, and no DAGs.

Airflow2 gitSync DAG works for airflow namespace, but not alternate namespace

0 Answers0