Files in AIRFLOW_HOME (which is an Azure File Share MOUNT) are created as root

Question

I have set up the airflow in Azure Cloud (Azure Container Apps) and attached an Azure File Share as an external mount/volume

1. I ran **airflow init service**, it had created the airflow.cfg and `'webserver_config.py'` file in the **AIRFLOW_HOME (/opt/airflow)**, which is actually an azure mounted file system
2. I ran **airflow webserver service**, it had created the `airflow-webserver.pid` file in the **AIRFLOW_HOME (/opt/airflow)**, which is actually an azure mounted file system

Now the problem is all the files created above are created with root user&groups, not as airflow user(50000), I have also set the env variable AIRFLOW_UID to 50000 during the creation of the container app. due to this my webservers are not starting, throwing the below error

 PermissionError: [Errno 1] Operation not permitted: '/opt/airflow/airflow-webserver.pid'

Note: Azure Containers Apps does not allow use root/sudo commands, otherwise I could solve this problem with simple chown commands

Another problem is the airflow configurations passed through environment variables are never picked up by Docker, eg

  - name: AIRFLOW__API__AUTH_BACKENDS
    value: 'airflow.api.auth.backend.basic_auth'

Attached screenshot for reference

Your help is much appreciated!

YAML file that I use to create my container app:

id: /subscriptions/1234/resourceGroups/<my-res-group>/providers/Microsoft.App/containerApps/<app-name>
identity:
  type: None
location: eastus2
name: webservice
properties:
  configuration:
    activeRevisionsMode: Single
    registries: []
  managedEnvironmentId: /subscriptions/1234/resourceGroups/<my-res-group>/providers/Microsoft.App/managedEnvironments/container-app-env
  template:
    containers:
    - command:
      - /bin/bash
      - -c
      - exec /entrypoint airflow webserver
      env:
      - name: AIRFLOW__API__AUTH_BACKENDS
        value: 'airflow.api.auth.backend.basic_auth'
      - name: AIRFLOW__CELERY__BROKER_URL
        value: redis://:@myredis.redis.cache.windows.net:6379/0
      - name: AIRFLOW__CELERY__RESULT_BACKEND
        value: db+postgresql://user:pass@postres-db-servconn.postgres.database.azure.com/airflow?sslmode=require
      - name: AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION
        value: 'true'
      - name: AIRFLOW__CORE__EXECUTOR
        value: CeleryExecutor
      - name: AIRFLOW__CORE__SQL_ALCHEMY_CONN
        value: postgresql+psycopg2://user:pass@postres-db-servconn.postgres.database.azure.com/airflow?sslmode=require
      - name: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
        value: postgresql+psycopg2://user:pass@postres-db-servconn.postgres.database.azure.com/airflow?sslmode=require
      - name: AIRFLOW__CORE__LOAD_EXAMPLES
        value: 'false'
      - name: AIRFLOW_UID
        value: 50000
      image: docker.io/apache/airflow:latest
      name: wsr
      volumeMounts:
      - volumeName: randaf-azure-files-volume
        mountPath: /opt/airflow
      probes: []
      resources:
        cpu: 0.25
        memory: 0.5Gi
    scale:
      maxReplicas: 3
      minReplicas: 1
    volumes:
    - name: randaf-azure-files-volume
      storageName: randafstorage
      storageType: AzureFile
resourceGroup: RAND
tags:
  tagname: ws-only
type: Microsoft.App/containerApps

Files in AIRFLOW_HOME (which is an Azure File Share MOUNT) are created as root

0 Answers0