How to solve Git fetch status code 128 with "Your account is suspended"

Question

I have a Jenkins master with two build agents out of which one is a Windows box and the other is a Mac. The following Jenkinsfile attempts to check out a git repository from a Github Enterprise server.

properties([
    parameters([
        string(
            name: 'GIT_URL',
            defaultValue: params.GIT_URL?:'https://github.com/',
            description: 'URL to clone the Git repo.'
        ),
        string(
            name: params.GIT_SELECTOR?:'GIT_SELECTOR',
            defaultValue: params.GIT_SELECTOR?:'refs/heads/master',
            description: 'Branch, tag or commit.'
        ),
        [
            $class: 'CredentialsParameterDefinition',
            name: 'GIT_CREDENTIALS',
            credentialType: 'com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl',
            defaultValue: params.GIT_CREDENTIALS?:'',
            description: 'User and password to clone the Git repository',
            required: true
        ]
    ])
])

node('build'){

    stage('Clean'){
        cleanDir()
        sh('''
            pwd
            ls -la
        ''')
    }

    stage('Identify host'){
        def host = command("hostname")
        def hostOS = isUnix()?'mac':'win'
        currentBuild.description = " ${hostOS}-host"
    }

    stage('Check out'){
        checkout([
            $class: 'GitSCM',
            branches: [
                [name: params.GIT_SELECTOR]
            ],
            extensions: [
                [
                    $class: 'RelativeTargetDirectory',
                    relativeTargetDir: 'app'
                ]
            ],
            userRemoteConfigs: [
                [
                    credentialsId: params.GIT_CREDENTIALS,
                    url: GIT_URL
                ]
            ]
        ])
    }
}

It succeeds when the build goes to the Windows box, but it recently started failing on the Mac for no apparent reason. The error I'm getting is this:

Cloning the remote Git repository
Cloning repository https://github.acme.com/Acme/foo-project.git
 > git init /Jenkins/workspace/.tests/test-checkout/app # timeout=10
Fetching upstream changes from https://github.acme.com/Acme/foo-project.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials my-git-credentials
 > git fetch --tags --progress https://github.acme.com/Acme/foo-project.git +refs/heads/*:refs/remotes/origin/*
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "git fetch --tags --progress https://github.acme.com/Acme/foo-project.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: remote: Your account is suspended. Please check with your installation administrator.
fatal: unable to access 'https://github.acme.com/Acme/foo-project.git/': The requested URL returned error: 403

    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1877)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1596)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:71)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:348)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:545)
    at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
    at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
    at hudson.remoting.UserRequest.perform(UserRequest.java:153)
    at hudson.remoting.UserRequest.perform(UserRequest.java:50)
    at hudson.remoting.Request$2.run(Request.java:332)
    at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at hudson.remoting.Engine$1$1.run(Engine.java:85)
    at java.lang.Thread.run(Thread.java:745)
    at ......remote call to Channel to /208.83.1.25(Native Method)
    at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545)
    at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
    at hudson.remoting.Channel.call(Channel.java:830)
    at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
    at sun.reflect.GeneratedMethodAccessor1126.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
    at com.sun.proxy.$Proxy75.execute(Unknown Source)
    at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1067)
    at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1107)
    at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)
    at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:83)
    at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:73)
    at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
    at hudson.security.ACL.impersonate(ACL.java:260)
    at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

Notice the bit about GitException and the git fetch error with message status code 128 and The requested URL returned error: 403. Of course, since it does work on the Windows box I know it's definitely not the credentials. What could possibly be the cause of this? What should I look at?

Edit: I've tried hitting the Github API from both slaves using curl -X GET https://github.acme.com/api/v3/user -H 'Authorization: token *****...****' and while on the Windows box it worked fine, on the Mac one I got:

{
    "message": "Must authenticate to access this API.",
    "documentation_url": "https://developer.github.com/enterprise/2.11/v3"
}

score 2 · Answer 1 · answered Apr 07 '18 at 08:29

There's a couple of things you can try:

Your GitHub enterprise instance will have logs detailing the failed authentication. Ask your administrator for those, as they may have the answer straight away.
If you have permissions to do so with your user, try hitting an authenticated endpoint of the GitHub REST API from the problem box using basic auth (e.g https://github.acme.com/api/v3/user) - if that fails with a 403 too you should get some information about the reason for it in the response body and headers.

score 0 · Accepted Answer · answered Aug 07 '18 at 10:39

After much troubleshooting the only thing we found that solves this issue is a hard reboot of the Mac build agent. After bouncing the box, all jobs sent to it are able to access the Git server again.

I haven't been able to find the reason why this occurred so this is the best answer so far. If anyone has any suggestions on what could potentially be the cause for such behavior I'd be inclined to mark that as the answer instead of this.

How to solve Git fetch status code 128 with "Your account is suspended"

2 Answers2