Sidekiq died due to the following error:NOAUTH Authentication required

Question

I deploy gitlab in centos7.1, when in start gitlab there is an error occur: "The GitLab Unicorn web server with pid 9307 is running. The GitLab Sidekiq job dispatcher is not running." I check the sidekiq.log as follow:

2016-01-22T12:52:37.630Z 9990 TID-osgvasil4 INFO: [Sidetiq] Sidetiq v0.6.3 - Copyright (c) 2012-2013, Tobias Svensson <tob@tobiassvensson.co.uk>
2016-01-22T12:52:37.631Z 9990 TID-osgvasil4 INFO: [Sidetiq] Sidetiq is covered by the 3-clause BSD license.
2016-01-22T12:52:37.631Z 9990 TID-osgvasil4 INFO: [Sidetiq] See LICENSE and http://opensource.org/licenses/BSD-3-Clause for licensing details.
2016-01-22T12:52:37.631Z 9990 TID-osgvasil4 INFO: [Sidetiq] Sidetiq::Supervisor start
2016-01-22T12:52:37.632Z 9990 TID-osgvy7rgw INFO: [Sidetiq] Sidetiq::Actor::Clock id: 69937594813880 initialize
2016-01-22T12:52:37.633Z 9990 TID-osgvy6rww INFO: [Sidetiq] Sidetiq::Actor::Handler id: 69937594769280 initialize
2016-01-22T12:52:37.634Z 9990 TID-osgvy6kqk INFO: [Sidetiq] Sidetiq::Actor::Handler id: 69937594759400 initialize
2016-01-22T12:52:38.498Z 9990 TID-osgvasil4 INFO: Booting Sidekiq 3.3.0 with redis options {:url=>"redis://:REDACTED@127.0.0.1:6379", :namespace=>"resque:gitlab"}
2016-01-22T12:52:40.445Z 9990 TID-osgvasil4 INFO: Running in ruby 2.1.7p400 (2015-08-18 revision 51632) [x86_64-linux]
2016-01-22T12:52:40.445Z 9990 TID-osgvasil4 INFO: See LICENSE and the LGPL-3.0 for licensing details.
2016-01-22T12:52:40.445Z 9990 TID-osgvasil4 INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org/pro
2016-01-22T12:52:40.534Z 9990 TID-osgwye4lc ERROR: heartbeat: ERR invalid password
2016-01-22T12:52:40.534Z 9990 TID-osgvjnc5c ERROR: Error fetching message: ERR invalid password
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis/client.rb:113:in `call'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis/client.rb:95:in `block in connect'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis/client.rb:279:in `with_reconnect'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis/client.rb:93:in `connect'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis/client.rb:262:in `with_socket_timeout'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis/client.rb:194:in `call_with_timeout'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis.rb:1064:in `block in _bpop'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis.rb:37:in `block in synchronize'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /usr/local/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis.rb:37:in `synchronize'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis.rb:1061:in `_bpop'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis.rb:1106:in `brpop'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-namespace-1.5.1/lib/redis/namespace.rb:392:in `call_with_namespace'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-namespace-1.5.1/lib/redis/namespace.rb:289:in `method_missing'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/fetch.rb:102:in `block in retrieve_work'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/connection_pool-2.1.0/lib/connection_pool.rb:58:in `with'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq.rb:72:in `redis'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/fetch.rb:102:in `retrieve_work'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/fetch.rb:37:in `block in fetch'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/util.rb:15:in `watchdog'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/fetch.rb:33:in `fetch'
2016-01-22T12:52:40.535Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `public_send'
2016-01-22T12:52:40.536Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `dispatch'
2016-01-22T12:52:40.536Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:122:in `dispatch'
2016-01-22T12:52:40.536Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in `block in invoke'
2016-01-22T12:52:40.536Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in `block in task'
2016-01-22T12:52:40.536Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in `block in task'
2016-01-22T12:52:40.536Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in `block in initialize'
2016-01-22T12:52:40.536Z 9990 TID-osgvjnc5c ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in `block in create'
2016-01-22T12:52:45.535Z 9990 TID-osgwye4lc ERROR: heartbeat: NOAUTH Authentication required.
2016-01-22T12:52:50.536Z 9990 TID-osgwye4lc ERROR: heartbeat: NOAUTH Authentication required.
2016-01-22T12:52:53.526Z 9990 TID-osgvjkvm4 ERROR: NOAUTH Authentication required.
2016-01-22T12:52:53.526Z 9990 TID-osgvjkvm4 ERROR: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis/client.rb:113:in `call'
2016-01-22T12:52:53.527Z 9990 TID-osgvjkvm4 WARN: {:context=>"scheduling poller thread died!"}
2016-01-22T12:52:53.527Z 9990 TID-osgvjkvm4 WARN: NOAUTH Authentication required.
2016-01-22T12:52:53.527Z 9990 TID-osgvjkvm4 WARN: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis/client.rb:113:in `call'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis.rb:1320:in `block in smembers'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis.rb:37:in `block in synchronize'
/usr/local/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis.rb:37:in `synchronize'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis.rb:1319:in `smembers'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-namespace-1.5.1/lib/redis/namespace.rb:392:in `call_with_namespace'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-namespace-1.5.1/lib/redis/namespace.rb:289:in `method_missing'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/api.rb:557:in `block in cleanup'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/connection_pool-2.1.0/lib/connection_pool.rb:58:in `with'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq.rb:72:in `redis'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/api.rb:556:in `cleanup'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/api.rb:549:in `initialize'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/scheduled.rb:79:in `new'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/scheduled.rb:79:in `poll_interval'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/scheduled.rb:58:in `block in poll'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/util.rb:15:in `watchdog'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/scheduled.rb:23:in `poll'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `public_send'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `dispatch'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:122:in `dispatch'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in `block in invoke'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in `block in task'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in `block in task'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in `block in initialize'
/home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in `block in create'
2016-01-22T12:52:53.527Z 9990 TID-osgwye73c WARN: Sidekiq died due to the following error, cannot recover, process exiting
2016-01-22T12:52:53.527Z 9990 TID-osgwye73c WARN: NOAUTH Authentication required.
2016-01-22T12:52:53.527Z 9990 TID-osgwye73c WARN: /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/redis-3.2.1/lib/redis/client.rb:113:in `call'

I really have no idea how to fix it. THANKS!

Answer 1

Did you verify that the redis password is correct, also it looks like it supposed to be set differently than you have it.

configure redis auth on sidekiq

Answer 2

I had the same problem on my server. Aptitude reported that the package gitlab can't be installed correctly because of the error "NOAUTH Authentication required." when trying to call the Redis server.

Since I run a public server, I will not use the NOAUTH authentification. I want to use a password for Redis. My config file /etc/redis/redis.conf states (beside other lines):

protected-mode yes
requirepass MyPasswordHere1234567890

I solved following problems:

Problem 1

In the logfile /var/log/syslog I could see following error which prevented aptitude to install the package:

Mar 19 17:11:21 server1 gitlab-sidekiq[647]: 2018-03-19T16:11:21.041Z 647 TID-gtf4ftbyw INFO: Booting Sidekiq 4.2.3 with redis options {:host=>"localhost", :port=>6379, :password=>nil, :namespace=>"resque:gitlab", :url=>nil}
Mar 19 17:11:21 server1 gitlab-sidekiq[647]: NOAUTH Authentication required.

To Fix, I edited /etc/gitlab/initializers/sidekiq.rb and added the line redis_config_hash[:password] = 'MyPasswordHere1234567890' like so:

# Custom Redis configuration
redis_config_hash = Gitlab::Redis.params
redis_config_hash[:namespace] = Gitlab::Redis::SIDEKIQ_NAMESPACE
redis_config_hash[:password] = 'MyPasswordHere1234567890'

Sidekiq.configure_server do |config|
  config.redis = redis_config_hash

Problem 2

After the package was installed, I got another warning:

Running /usr/share/gitlab-shell/bin/check
Check GitLab API access: OK
Access to /usr/share/gitlab/.ssh/authorized_keys: OK
Send ping to redis server: /usr/lib/ruby/vendor_ruby/vendor/redis/lib/redis/client.rb:121:in `call': NOAUTH Authentication required. (Redis::CommandError)
        from /usr/lib/ruby/vendor_ruby/vendor/redis/lib/redis.rb:149:in `block in ping'
        from /usr/lib/ruby/vendor_ruby/vendor/redis/lib/redis.rb:58:in `block in synchronize'
        from /usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
        from /usr/lib/ruby/vendor_ruby/vendor/redis/lib/redis.rb:58:in `synchronize'
        from /usr/lib/ruby/vendor_ruby/vendor/redis/lib/redis.rb:148:in `ping'
        from /usr/share/gitlab-shell/bin/check:36:in `<main>'
gitlab-shell self-check failed
  Try fixing it:
  Make sure GitLab is running;
  Check the gitlab-shell configuration file:
  sudo -u gitlab -H editor /usr/share/gitlab-shell/config.yml
  Please fix the error above and rerun the checks.

To fix it, I edited /usr/share/gitlab-shell/config.yml and added the parameter pass in the redis section, like so:

redis:
  bin: /usr/bin/redis-cli
  host: 127.0.0.1
  port: 6379
  # pass: redispass # Allows you to specify the password for Redis
  pass: MyPasswordHere1234567890
  database: 0
  # socket: /var/run/redis/redis.sock # Comment out this line if you want to use TCP
  namespace: resque:gitlab