6

I tried using ENV variable to configure connection urls, I have an ami that is preconfigured with alchemy_conn and broker_url etc, I have written environment variables to /etc/environment in the instances being spun up from the amis to override the properties in airflow.cfg file. I was able to access the variables from python code as well.

But it doesn't seem to take effect while running ariflow, tried restarting the process as well but didn't work, it still points to the one in airflow.cfg file

Wordsmyth
  • 443
  • 4
  • 16
Somasundaram Sekar
  • 5,244
  • 6
  • 43
  • 85
  • What are the actual variables you are using? Not their values, but their names. E.G. AIRFLOW_ALCHEMY_URL, CELERY_BROKER_URL etc. – jhnclvr Nov 09 '17 at 14:36

1 Answers1

8

The issue is probably that system services do not automatically pick up environment from /etc/environment that is just for interactive sessions.

If you are using systemd you can add EnvironmentFile=/etc/environment, though more "typical" would be to put the service specific config in /etc/default/airflow (on Debian/Ubuntu) or /etc/sysconfig/airflow (on Centos/Redhat)

The example systemd scripts from Airflow already have this for Centos: https://github.com/apache/incubator-airflow/blob/faa9a5266c0b2e68693dd106b5cb46d30770dadc/scripts/systemd/airflow-webserver.service#L20

Ash Berlin-Taylor
  • 3,879
  • 29
  • 34