I am using jenkins swarm plugin to connect a slave node to a master node. However, it is giving Connection Refused error.
I am using docker swarm to deploy this on a multi node cluster. Right now I am testing with a single node setup only and here is my compose file for the setup:
version: '3.1'
services:
viz:
image: manomarks/visualizer
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
ports:
- "8090:8080"
master:
image: myproject-jenkins-master
ports:
- "8080:8080"
- "50000:50000"
volumes:
- ./jenkins_home:/var/jenkins_home
nginx:
image: myproject-jenkins-nginx
ports:
- "80:80"
linuxagent:
image: myproject-jenkins-linuxagent
And its being deployed using the docker stack deploy command so each of the above service gets its own internal load balancer.
Jenkins is served on web via nginx. But I've also kept port 8080 exposed for debugging. If I open my browser and hit either of http://localhost or http://localhost:8080, I can access the jenkins web interface. On master, I have installed the jenkins swarm plugin. If I get inside the bash shell from my linuxagent container, which is intended to be a slave node, I can ping the other docker services but here is what happens when I try to run the swarm-client-3.3.jar file.
If I try to access jenkins, I can do so by accessing http://myproject_master:8080
root@d139902be5de:~# curl http://myproject_master:8080
<html><head><meta http-equiv='refresh' content='1;url=/login?from=%2F'/><script>window.location.replace('/login?from=%2F');</script></head><body style='background-color:white; color:white;'>
Authentication required
<!--
You are authenticated as: anonymous
Groups that you are in:
Permission you need to have (but didn't): hudson.model.Hudson.Read
... which is implied by: hudson.security.Permission.GenericRead
... which is implied by: hudson.model.Hudson.Administer
-->
</body></html> root@d139902be5de:~#
Also, I can access the same via http://myproject_nginx
root@d139902be5de:~# curl http://myproject_nginx
<html><head><meta http-equiv='refresh' content='1;url=/login?from=%2F'/><script>window.location.replace('/login?from=%2F');</script></head><body style='background-color:white; color:white;'>
Authentication required
<!--
You are authenticated as: anonymous
Groups that you are in:
Permission you need to have (but didn't): hudson.model.Hudson.Read
... which is implied by: hudson.security.Permission.GenericRead
... which is implied by: hudson.model.Hudson.Administer
-->
</body></html> root@d139902be5de:~#
If I try to launch the jar file for swarm, I get this error:
root@d139902be5de:~# java -jar swarm-client-3.3.jar -username mandeep -password 12213 -master http://myproject_nginx
Apr 09, 2017 5:52:44 PM hudson.plugins.swarm.Client main
INFO: Client.main invoked with: [-username mandeep -password 12213 -master http://myproject_nginx]
Apr 09, 2017 5:52:44 PM hudson.plugins.swarm.Client run
INFO: Discovering Jenkins master
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Apr 09, 2017 5:52:44 PM hudson.plugins.swarm.Client run
INFO: Attempting to connect to http://myproject_nginx/ bc1be8e7-eaf0-47ff-8aeb-36f75d6ba081 with ID c84ce43b
Apr 09, 2017 5:52:45 PM hudson.remoting.jnlp.Main createEngine
INFO: Setting up slave: d139902be5de-c84ce43b
Apr 09, 2017 5:52:45 PM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Apr 09, 2017 5:52:45 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://myproject_nginx/]
Apr 09, 2017 5:52:45 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
Agent address: myproject_nginx
Agent port: 50000
Identity: 86:5b:f3:77:84:92:21:87:95:4c:4b:0e:f7:4e:e5:1d
Apr 09, 2017 5:52:45 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Apr 09, 2017 5:52:45 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to myproject_nginx:50000
Apr 09, 2017 5:52:55 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to myproject_nginx:50000 (retrying:2)
java.io.IOException: Failed to connect to myproject_nginx:50000
at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:243)
at hudson.remoting.Engine.connect(Engine.java:500)
at hudson.remoting.Engine.innerRun(Engine.java:364)
at hudson.remoting.Engine.run(Engine.java:287)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect0(Native Method)
at sun.nio.ch.Net.connect(Net.java:454)
at sun.nio.ch.Net.connect(Net.java:446)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:204)
... 3 more
Seems like the issue is that the slave expects master to be myproject_nginx and hence hits the port 50000 which fails because this service does not have port 50000 exposed. So for that purpose, I have exposed port 8080 and 50000 on the myproject_master service. But when I try that url, then I get a different error
root@d139902be5de:~# java -jar swarm-client-3.3.jar -username mandeep -password 12213 -master http://myproject_master:8080
Apr 09, 2017 5:57:01 PM hudson.plugins.swarm.Client main
INFO: Client.main invoked with: [-username mandeep -password 12213 -master http://myproject_master:8080]
Apr 09, 2017 5:57:01 PM hudson.plugins.swarm.Client run
INFO: Discovering Jenkins master
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Apr 09, 2017 5:57:01 PM hudson.plugins.swarm.Client run
SEVERE: IOexception occurred
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at java.net.Socket.<init>(Socket.java:434)
at java.net.Socket.<init>(Socket.java:286)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at hudson.plugins.swarm.SwarmClient.discoverFromMasterUrl(SwarmClient.java:224)
at hudson.plugins.swarm.Client.run(Client.java:115)
at hudson.plugins.swarm.Client.main(Client.java:88)
This is where I am completely stuck and unable to understand how to fix the issue. Can there be a problem because the slave is running behind a load balancer ? Or is there anything fundamentally wrong in the architecture that I am trying to accomplish ? I want to be able to scale the linuxagent service dynamically so that each of the node behaves as a slave agent and just connects with the master node whenever it is launched. I read about the swarm plugin for jenkins and found that it can be used to achieve this kind of setup
