0

I am trying to run Airflow Celery worker with redis backend but cant get the connection between the worker and redis to work.

Here is how the configuration looks like:

broker_url = redis://:<my-password>@localhost:6379/0

I am getting error:

ValueError: invalid literal for int() with base 10: 'my-password-string'

Anyone know any fix for this?

Itzblend
  • 107
  • 2
  • 9
  • Does this answer your question? [Celery beat + redis with password throws No Auth exception](https://stackoverflow.com/questions/49376540/celery-beat-redis-with-password-throws-no-auth-exception) – Ersoy May 31 '20 at 05:28

2 Answers2

1

That's because there're special characters in your password, you can simply do this: e.g. your password is my#pw#

  1. do url-encoding, then you get my%23pw%23
  2. append another % before % to allow airflow read the config successfully

then the final password is my%%23pw%%23, so e.g. the redis broker url in airflow.cfg should be like this:

broker_url = redis://:my%%23pw%%23@redis-host:6379/0

Lane Lee
  • 89
  • 2
  • 8
0

So i solved it now.

My password was generated from random characters piped through base64 and it was causing problems.

I changed my password to bit shorter and simplier one and airflow worker now runs normally.

Itzblend
  • 107
  • 2
  • 9