0

Update:

I did end up finding a solution to the issue, but it doesn't explain why this happened. I am leaving this question up in case anyone can answer why and how to avoid it in the future.

Here are the errors I was getting

Fetching changes with git depth set to 20... Reinitialized existing Git repository in /home/gitlab-runner/builds/i3fxSC2v/0/occ/groupsyncer/.git/ fatal: git fetch-pack: expected shallow list fatal: The remote end hung up unexpectedly Cleaning up project directory and file based variables 00:00

Since the last successful run, the server hosting the runner had rebooted. I removed the build dir and re-ran the job and it worked. I don't know why other than potentially cache had changed? The gitlab instance had also been restarted since during that time frame.

The last successful run was 10 days ago.

Runner toml:

listen_address = "0.0.0.0:9103"
concurrent = 6
check_interval = 0
log_level = "warning"

[session_server]
  session_timeout = 1800

[[runners]]
  name = "shell-runner"
  url = "https://gitlab.example.com/"
  token = "REDACTED"
  executor = "shell"
  pre_build_script = "mkdir -p $HOME/.docker/ && echo \"{ \\\"proxies\\\": { \\\"default\\\": { \\\"httpProxy\\\": \\\"$HTTP_PROXY\\\", \\\"httpsProxy\\\": \\\"$HTTPS_PROXY\\\", \\\"noProxy\\\": \\\"$NO_PROXY\\\" } } }\" > $HOME/.docker/config.json"
  environment = ["https_proxy=http://squidproxy.example.com:3128", "http_proxy=http://squidproxy.example.com:3128", "HTTPS_PROXY=squidproxy.example.com:3128", "HTTP_PROXY=squidproxy.example.com:3128", "no_proxy=gitlab.example.com", "NO_PROXY=gitlab.example.com,docker,thealias"]
  clone_url = "https://gitlab.example.com"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
Brandon Kauffman
  • 1,515
  • 1
  • 7
  • 33
  • I'm not sure this is needed for shell executor, but have you tried setting the `pre_clone_script` [as described in the proxy documentation](https://docs.gitlab.com/runner/configuration/proxy.html#adding-the-proxy-to-the-docker-containers) – sytech Aug 26 '22 at 16:54
  • @sytech The toml is working, it just failed oddly when there were items in the cache dir – Brandon Kauffman Aug 26 '22 at 17:01
  • 1
    Ah, thanks for clearing that up for me. [This question](https://stackoverflow.com/questions/56663096/gitlab-runner-doesnt-work-on-a-specific-project) may be related... Without knowing what changed between when it worked and when it did not, it's hard to say exactly why it stopped working before and is now working after the changes you made. Possibly a manual configuration that does not persist through reboot or a change that was applied on reboot (like an upgrade of the system OS/packages)... Could have even been a temporary outage of the proxy. Beats me at this point. – sytech Aug 26 '22 at 17:10
  • 1
    @sytech Thanks! I voted to close as a duplicate. That was the issue. I re-encountered it after a few successful builds – Brandon Kauffman Aug 26 '22 at 17:16

0 Answers0