2

I have Gitlab 8.6 running on an Ubuntu 14.04 server that seems to have gotten messed up. I consistently get a 502 error when accessing the site. The server likely has not been restarted since installing Gitlab initially, and a power outage caused the server to reboot. Now, I cannot start/restart Gitlab due to what appears to be port conflicts.

I installed Gitlab via source, I don't have any custom port configurations, and am using NGINX. nginx -t shows that the configuration appears to be correct syntax-wise.

When I run netstat -tupln, I see that Unicorn & a Gitlab instance is already running on :8080 and :80 respectively at boot up. I suspect that a 2nd instance of Gitlab was installed which is being run at boot and that is causing the proper instance to have port conflicts when I try to run it via service gitlab restart. I'm not even sure if that's possible, but I can't seem to figure out where to go from here. Every time I run sudo gitlab-ctl reconfigure or service gitlab start, it fails and the unicorn.stderror.log shows bind errors to the :8080 port. I tried moving the Unicorn service to :8081 as well, but I still receive the port binding error.

Does anyone know how I can detect if there are multiple Gitlab instances running, and maybe if there is a way to remove a duplicated one if it's possible? Thank you!

EDIT: Here is what is in the /etc/gitlab/gitlab.rb file. Everything else is commented out.

## Url on which GitLab will be reachable external_url 'http://my-gitlab-instance.domain.com'

EDIT 2: My /home/git/gitlab/ directory is mapped to https://gitlab.com/gitlab-org/gitlab-ce.git, and is on the 8-7-stable branch. gitlab-shell and gitlab-workhorse are on the correct versions according to https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/update/8.6-to-8.7.md

EDIT 3: I have gotten to a point where the Gitlab seems to self-check okay by removing the gitlab-ce package (https://gitlab.com/gitlab-org/omnibus-gitlab/issues/135), but the server returns a 404. NGINX, Unicorn, Sidekiq, and gitlab-workhorse all say that they're running. I see that unicorn.rb is listening on :8080, and nginx is listening on 0.0.0.0:80 and :::80. I guess now I'm troubleshooting this 404 and hopefully I will be back to my install-from-source.

CSchulz
  • 10,882
  • 11
  • 60
  • 114
jmastic
  • 109
  • 1
  • 10
  • Can you show /etc/gitlab/gitlab.rb? – SLY May 03 '16 at 10:58
  • Added what is in gitlab.rb. Everything else, including the port overrides are commented out. The `external_url` is the only thing enabled. Obviously, this is just an example, but the proper one is in my gitlab.rb file. – jmastic May 03 '16 at 13:48
  • Sounds very strange. Maybe you should try to update your gitlab to latest stable? – SLY May 03 '16 at 16:32
  • Added another edit. I tried upgrading to 8.7, and I at least am on the correct Git branches. – jmastic May 03 '16 at 18:30
  • Add to your gitlab.rb the following lines `external_url 'http://:'` `unicorn['port'] = ` Run `sudo gitlab-ctl reconfigure` and show full output – SLY May 03 '16 at 18:39
  • Since I installed from source, should I still use `gitlab-ctl reconfigure`? I haven't had to do that in the past. It is failing consistently on the commands containing `psql`, eg `psql: could not connect to server: No such file or directory`. Also, the server is located on a virtual machine which I can't copy/paste from which makes posting logs/files very difficult. – jmastic May 03 '16 at 20:55
  • Some of gitlab.rb changes require reconfiguration. Usually you can run `gitlab-ctl reconfigure` on working instance, this operation should not break anything. – SLY May 04 '16 at 01:41

1 Answers1

1

What I have found is that there were 2 issues causing the errors I had.

First, I removed a "gitlab-ce" package that was installed, following the instructions here: https://gitlab.com/gitlab-org/omnibus-gitlab/issues/135. For some reason, when I restart the machine now I have to restart these services, in order, for Gitlab to run properly redis-server, gitlab, nginx. However, Gitlab does start responding properly after that.

Second, the 404 error was due to a different server that was also listening on that IP address, causing a conflict.

I will likely move to using the omnibus package on a fresh, new server going forward, but at least the immediate issues appear resolved. Thanks for your help, SLY!

jmastic
  • 109
  • 1
  • 10