2

I'm using the latest Jenkins (2.319) as well as the latest version of the Bitbucket Branch Source Plugin. Jenkins is on a local machine behind a firewall, and Bitbucket Server v6.10.1 is connected to it through the plugin.

I'm very frequently experiencing the following error with the build failing within <100ms:

javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
    at java.base/sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:526)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHostname(SSLConnectionSocketFactory.java:504)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:437)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
    at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:859)
Caused: java.io.IOException: Communication error for url: /rest/api/1.0/projects/Sample/repos/sample-project/branches?start=0&limit=200
    at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:889)
    at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getResources(BitbucketServerAPIClient.java:835)
    at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getServerBranches(BitbucketServerAPIClient.java:588)
    at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getBranches(BitbucketServerAPIClient.java:579)
    at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:795)
    at jenkins.scm.api.SCMSource.fetch(SCMSource.java:582)
    at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:100)
    at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:309)
    at hudson.model.ResourceController.execute(ResourceController.java:100)
    at hudson.model.Executor.run(Executor.java:433)
Finished: FAILURE

This error also commonly appears when scanning multibranch pipeline or when triggering hook events - basically any kind of communication between Jenkins and Bitbucket. It often resolves itself after a retry, though sometimes not even retries help and I need to wait 2-3 minutes before the "push" goes through.

I realize this could be a problem with the Bitbucket Server servers, but I have no idea how to pinpoint or troubleshoot the issue. The fact that this is not consistently reproducible and only happens 30-40% of the time doesn't help.

Any ideas and pointers that could help with pinning down the issue are welcome.

Sharp
  • 1,335
  • 1
  • 12
  • 27
  • Can you target a specific Bitbucket server and then troubleshoot by trying each one individually in the pool. It may just be that one of the Bitbucket servers has a cert issue, hence the issue is intermittent for you. I guess you declare the Bitbucket url via the plugin, so instead of the DNS loadbalanced url, use a specific server. Just an idea, to see if that helps – djmonki Nov 12 '21 at 13:04
  • I logged HttpClient/SSL stuff and noticed this issue appears with all IP addresses it connects to, but (somewhat) rarely. When I scan the organization folder for example, I can see that 20 requests to the same IP succeed and then 21st fails (and then 22nd again succeeds) – Sharp Nov 12 '21 at 16:29
  • Ah ok. In that case, notice that the scan via the api has the following: ```/rest/api/1.0/projects/Sample/repos/sample-project/branches?start=0&limit=200```. I wonder if the webhook with Bitbucket (from the Bitbucket server farm perpsective) is limited to 20 connections. Then jenkins has to wait for the threads from the connection pool to be released before you get another successful connection ? This issue seems to have been reported for the bitbucket plugin previously [jenkinsci/bitbucket-branch-source-plugin/issues/386](https://github.com/jenkinsci/bitbucket-branch-source-plugin/issues/386) – djmonki Nov 12 '21 at 16:48
  • How would I go around testing that? I see we have different errors on the HTTP client level, but I suppose that could be due to a different server setup? – Sharp Nov 13 '21 at 10:51
  • @Sharp you could possibly point the webhook at https://requestcatcher.com/ and force 25 changes, then look at the output. May or may not help – Chris Maggiulli Nov 17 '21 at 12:38

0 Answers0