service registry ping failed, will retry. Error: Get "http://localhost:8040/access/api/v1/system/ping": dial tcp [::1]: connect: connection refused

Question

service registry ping failed, will retry. Error: Get "http://localhost:8040/access/api/v1/system/ping": dial tcp [::1]:8040: connect: connection refused

The application is running on remote machine and the log is showing localhost in URLs at multiple places. We tried to change the system.yaml file and add the IP for node also. We tried to change the IP and PORT in system.yaml file as well as bundle.js of server inside frontend folder.

There is no entry of localhost in hosts files too.

We checked the artifactory.xml and access.xml too, and nothing weird found in it. At one place, someone asked to changes the owner to 'artifactory' but I am not getting to change the owner of the files as the owner of the file is the system owner /logged-in user to Linux machine.

The previous question similar to it is answered yet, without any resolution. Please share, if any more inputs or attempts required to fix it.

logs

2:12:29.162L[org.apache.coyote.http11.Http11NioProtocol] [org.apache.coyote.AbstractProtocol init] - Initializing ProtocolHandle"http-nio-8081"] 
2021-07-27T12:12:29.258L[org.apache.cat.util.net.NioSelectorPool] [org.apache.cat.util.net.NioSelectorPool getSharedSelector] - Using a shared selector for servlet write/read 
2021-07-27T12:12:29.430L[org.apache.coyote.http11.Http11NioProtocol] [org.apache.coyote.AbstractProtocol init] - Initializing ProtocolHandle"http-nio-127.0.0.1-8091"] 
2021-07-27T12:12:29.431L[org.apache.cat.util.net.NioSelectorPool] [org.apache.cat.util.net.NioSelectorPool getSharedSelector] - Using a shared selector for servlet write/read 
2021-07-27T12:12:29.450L[org.apache.coyote.http11.Http11NioProtocol] [org.apache.coyote.AbstractProtocol init] - Initializing ProtocolHandle"http-nio-127.0.0.1-8040"] 
2021-07-27T12:12:29.451L[org.apache.cat.util.net.NioSelectorPool] [org.apache.cat.util.net.NioSelectorPool getSharedSelector] - Using a shared selector for servlet write/read 
2021-07-27T12:12:29.485L[org.apache.catalina.core.StandardService] [org.apache.catalina.core.StandardService startInternal] - Starting service [Catalina] 
2021-07-27T12:12:29.486L[org.apache.catalina.core.StandardEngine] [org.apache.catalina.core.StandardEngine startInternal] - Starting Servlet engine: [Apache cat/8.5.66] 
2021-07-27T12:12:29.609L[org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - Deploying deployment descripto/localhome/myUser/jfrog/artifactory/app/artifactory/cat/conf/Catalina/localhost/artifactory.xml] 
2021-07-27T12:12:29.609L[org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - Deploying deployment descripto/localhome/myUser/jfrog/artifactory/app/artifactory/cat/conf/Catalina/localhost/access.xml] 
2021-07-27T12:12:29.730L5m[ct] [WARNING]  ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - A docBase [/localhome/myUser/jfrog/artifactory/app/artifactory/cat/webapps/access.war] inside the host appBase has been specified, and will be ignored 
2021-07-27T12:12:29.731L5m[ct] [WARNING]  ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - A docBase [/localhome/myUser/jfrog/artifactory/app/artifactory/cat/webapps/artifactory.war] inside the host appBase has been specified, and will be ignored 
2021-07-27T1
0:12:45.207Z [o.a.c.h.HaNodeProperties:65] [ocalhost-startStop-2] - Artifactory is running in non-clustered mode.
2021-07-27T10:12:46.531Z [security_keys.go: ]ain ] - Master key is missing. Pending for 25 seconds with s eouinit]
2021-07-27T10:12:46.854Zain ] - pinging artifactory, attempt number 10
2021-07-27T10:12:46.863Zain ] - pinging artifactory attempt number failed with code : ECONNREFUSED
2021-07-27T10:13:17.857Z server.go:5]ain ] - Starting server on localhost:8049
2021-07-27T10:13:17.857Z server.go:5]ain ] - Starting server on :8082
2021-07-27T10:13:17.857Z server.go:5]ain ] - Starting server on localhost:8046
2021-07-27T10:13:17.857Z provider_aggregator.go: ]ain ] - Starting provider *file.Provider {"Watch":true,"Filename":"","Constraints":null,"Trace":false,"TemplateVersion":0,"DebugLogGeneratedTemplate":false,"Directory":"/localhome/myUser/jfrog/artifactory/var/data/router/traefik/routes","TraefikFile":""}
2021-07-27T10:13:17.860Z server_configuration.go:6  ]ain ] - Server configuration reloaded on :8082
2021-07-27T10:13:17.860Z server_configuration.go:6  ]ain ] - Server configuration reloaded on localhost:8046
2021-07-27T10:13:17.860Z server_configuration.go:6  ]ain ] - Server configuration reloaded on localhost:8049
2021-07-27T10:13:17.860Z ain.go:47  ]ain ] - Router (jfrou) service initialization completed in 57.650 seconds. Listening on port: 8082
2021-07-27T10:13:18.094Z ERROR][22 ] ]ain ] - ArtifactoryClient::http [get] request /api/system/ping failed with4 code
2021-07-27T10:13:18.725Z [.AccessWithRouterRegistrar:111] [pool-13-thread-  ] - Access Server successfully registered on Router on URL: http://localhost:8046
2021-07-27T10:13:19.108Z ERROR][22 ] ]ain ] - ArtifactoryClient::http [get] request /api/system/ping failed with4 code
ERROR][22 ] ]ain ] - ArtifactoryClient::http [get] request /api/system/ping failed with4 code

Is this a clean install or an upgrade? Which version of Artifactory is it? Can you please attach the full log? — Maxim Y, Jul 27 '21 at 12:01
Yes @maxyor6040, it's a clean new install and it's version is 7.21. — Zafar, Jul 27 '21 at 12:52
Please share your `system.yaml`. It appears router and access has started successfully. The logs are incomplete can you, filter only ERRORs and send ? Also pls send the response to `http://localhost:8046/access/api/v1/topology` — Prasanna Raghavendra, Jul 28 '21 at 01:15
sytem.yaml `configVersion: 1 shared: node: ip: 127.0.0.1 database:` — Zafar, Jul 28 '21 at 08:34

score 0 · Answer 1 · edited Feb 06 '23 at 18:33

we had a system running fine for the last 4 months, we see there was some "update detected in the logs " - we are assuming that JFROG has done an update to the image.

we have the following errors (from docker logs) over and over again.

the only thing we can see is that the ":localhost" is trying to connect on ::1 - rather than use ip4 like everything else does -

2021-08-31T16:24:43.534Z [jffe ] [INFO ] [                ] [                              ] [main                ] - pinging artifactory, attempt number 40
2021-08-31T16:24:43.536Z [jffe ] [INFO ] [                ] [                              ] [main                ] - pinging artifactory attempt number 40 failed with code : ECONNREFUSED
2021-08-31T16:24:44.206Z [jfac ] [WARN ] [e6b16cb1bbc84c40] [o.j.c.ExecutionUtils:165      ] [pool-6-thread-2     ] - Retry 60 Elapsed 29.12 secs failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again
2021-08-31T16:24:47.090Z [jfevt] [INFO ] [3e5b610962c1f4aa] [access_thin_client.go:103     ] [main                ] - Cluster join: Retry 45: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [startup]
2021-08-31T16:24:47.471Z [jfmd ] [INFO ] [2908f702d61ab45b] [accessclient.go:57            ] [main                ] - Cluster join: Retry 45: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [access_client]
2021-08-31T16:24:49.214Z [jfac ] [WARN ] [e6b16cb1bbc84c40] [o.j.c.ExecutionUtils:165      ] [pool-6-thread-2     ] - Retry 70 Elapsed 34.13 secs failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again
2021-08-31T16:24:52.093Z [jfevt] [INFO ] [3e5b610962c1f4aa] [access_thin_client.go:103     ] [main                ] - Cluster join: Retry 50: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [startup]
2021-08-31T16:24:52.275Z [jfrou] [INFO ] [5bfc52e1713f01a5] [join_executor.go:165          ] [main                ] - Cluster join: Retry 5: Service registry ping failed, will retry. Error: Get "http://localhost:8040/access/api/v1/system/ping": net/http: request canceled (Client.Timeout exceeded while awaiting headers)
2021-08-31T16:24:52.473Z [jfmd ] [INFO ] [2908f702d61ab45b] [accessclient.go:57            ] [main                ] - Cluster join: Retry 50: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [access_client]
2021-08-31T16:24:53.563Z [jffe ] [INFO ] [                ] [                              ] [main                ] - pinging artifactory, attempt number 50
2021-08-31T16:24:53.564Z [jffe ] [INFO ] [                ] [                              ] [main                ] - pinging artifactory attempt number 50 failed with code : ECONNREFUSED
2021-08-31T16:24:54.225Z [jfac ] [WARN ] [e6b16cb1bbc84c40] [o.j.c.ExecutionUtils:165      ] [pool-6-thread-2     ] - Retry 80 Elapsed 39.14 secs failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again
2021-08-31T16:24:57.095Z [jfevt] [INFO ] [3e5b610962c1f4aa] [access_thin_client.go:103     ] [main                ] - Cluster join: Retry 55: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [startup]
2021-08-31T16:24:57.476Z [jfmd ] [INFO ] [2908f702d61ab45b] [accessclient.go:57            ] [main                ] - Cluster join: Retry 55: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [access_client]
2021-08-31T16:24:59.235Z [jfac ] [WARN ] [e6b16cb1bbc84c40] [o.j.c.ExecutionUtils:165      ] [pool-6-thread-2     ] - Retry 90 Elapsed 44.15 secs failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again
2021-08-31T16:25:02.098Z [jfevt] [INFO ] [3e5b610962c1f4aa] [access_thin_client.go:103     ] [main                ] - Cluster join: Retry 60: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [startup]
2021-08-31T16:25:02.478Z [jfmd ] [INFO ] [2908f702d61ab45b] [accessclient.go:57            ] [main                ] - Cluster join: Retry 60: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [access_client]
2021-08-31T16:25:03.589Z [jffe ] [INFO ] [                ] [                              ] [main                ] - pinging artifactory, attempt number 60
2021-08-31T16:25:03.590Z [jffe ] [INFO ] [                ] [                              ] [main                ] - pinging artifactory attempt number 60 failed with code : ECONNREFUSED
2021-08-31T16:25:04.246Z [jfac ] [WARN ] [e6b16cb1bbc84c40] [o.j.c.ExecutionUtils:165      ] [pool-6-thread-2     ] - Retry 100 Elapsed 49.16 secs failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again
2021-08-31T16:25:07.100Z [jfevt] [INFO ] [3e5b610962c1f4aa] [access_thin_client.go:103     ] [main                ] - Cluster join: Retry 65: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [startup]
2021-08-31T16:25:07.481Z [jfmd ] [INFO ] [2908f702d61ab45b] [accessclient.go:57            ] [main                ] - Cluster join: Retry 65: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [access_client]
2021-08-31T16:25:09.258Z [jfac ] [WARN ] [e6b16cb1bbc84c40] [o.j.c.ExecutionUtils:165      ] [pool-6-thread-2     ] - Retry 110 Elapsed 54.17 secs failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again
2021-08-31T16:25:12.103Z [jfevt] [INFO ] [3e5b610962c1f4aa] [access_thin_client.go:103     ] [main                ] - Cluster join: Retry 70: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [startup]
2021-08-31T16:25:12.483Z [jfmd ] [INFO ] [2908f702d61ab45b] [accessclient.go:57            ] [main                ] - Cluster join: Retry 70: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [access_client]
2021-08-31T16:25:13.615Z [jffe ] [INFO ] [                ] [                              ] [main                ] - pinging artifactory, attempt number 70
2021-08-31T16:25:13.617Z [jffe ] [INFO ] [                ] [                              ] [main                ] - pinging artifactory attempt number 70 failed with code : ECONNREFUSED
2021-08-31T16:25:14.270Z [jfac ] [WARN ] [e6b16cb1bbc84c40] [o.j.c.ExecutionUtils:165      ] [pool-6-thread-2     ] - Retry 120 Elapsed 59.19 secs failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again
2021-08-31T16:25:17.105Z [jfevt] [INFO ] [3e5b610962c1f4aa] [access_thin_client.go:103     ] [main                ] - Cluster join: Retry 75: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [startup]
2021-08-31T16:25:17.486Z [jfmd ] [INFO ] [2908f702d61ab45b] [accessclient.go:57            ] [main                ] - Cluster join: Retry 75: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [access_client]
2021-08-31T16:25:19.282Z [jfac ] [WARN ] [e6b16cb1bbc84c40] [o.j.c.ExecutionUtils:165      ] [pool-6-thread-2     ] - Retry 130 Elapsed 1.07 minutes failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again
2021-08-31T16:25:22.108Z [jfevt] [INFO ] [3e5b610962c1f4aa] [access_thin_client.go:103     ] [main                ] - Cluster join: Retry 80: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [startup]
2021-08-31T16:25:22.488Z [jfmd ] [INFO ] [2908f702d61ab45b] [accessclient.go:57            ] [main                ] - Cluster join: Retry 80: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [access_client]
2021-08-31T16:25:23.642Z [jffe ] [INFO ] [                ] [                              ] [main                ] - pinging artifactory, attempt number 80
2021-08-31T16:25:23.644Z [jffe ] [INFO ] [                ] [                              ] [main                ] - pinging artifactory attempt number 80 failed with code : ECONNREFUSED
2021-08-31T16:25:24.289Z [jfac ] [WARN ] [e6b16cb1bbc84c40] [o.j.c.ExecutionUtils:165      ] [pool-6-thread-2     ] - Retry 140 Elapsed 1.15 minutes failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again
2021-08-31T16:25:27.110Z [jfevt] [INFO ] [3e5b610962c1f4aa] [access_thin_client.go:103     ] [main                ] - Cluster join: Retry 85: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [startup]
^C

then we went an looked at the log files for access - its complaining that it cant talk to router - so access fails on port 8040 and its trying to get to router on port 8046.

these boxes were "touched" and the issue appeared spontaneously. Restarting the container doesnt seem to do anything, we have tried adding IP in system.yaml and it makes no different (since the issue as we can see it is that router doesnt actually start - port 8046 doesnt show up as a started.

catalina logs show nothing

tried to open support with JFROG since these are licensed boxes and didnt get much help there either (other than upgrade to 8????)

any change others have go and figured out "why" the router/access are always using localhost (when the license and other configuration are hostname bound - and then not being smart enough to use the IPV4 entry it finds ?) - pushing it to docker and saying "get rid of that entry" is not going to cut it - this was working fine until last week.. and then boom, stopped.

I suspect is something buried i.e license, rather than a instant random configuration issue

I am also getting similar log, which is having localhost appearing at many places. I searched for the localhost in all configuration files (.xml) , and replace them by 0.0.0.0 .You can try it and may be it will solve your problem. — Zafar, Sep 01 '21 at 04:53

score 0 · Answer 2 · answered Apr 10 '23 at 22:16

0

setting up ip as 0.0.0.0 worked for me in windows.

answered Apr 10 '23 at 22:16

sanju cena

1

score -1 · Answer 3 · answered Jul 27 '21 at 15:11

The attached logs don't contain the error you're referring to but I assume it's something similar to:

2021-07-27T15:04:22.880Z [jfevt] [INFO ] [61e09fae00e74a7 ] [access_thin_client.go:103     ] [main                ] - Cluster join: Retry 5: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp 127.0.0.1:8046: connect: connection refused [startup]

If you see the following log at the bottom the startup is successful and you can ignore this log message:

###############################################################
###   All services started successfully in 43.304 seconds   ###
###############################################################

It's just part of the startup flow - services waiting for access service to finish starting.

Are you experiencing any other issues?

Connection Refused, as I wrote the title of this problem. I am still getting the same error. — Zafar, Jul 28 '21 at 12:03

service registry ping failed, will retry. Error: Get "http://localhost:8040/access/api/v1/system/ping": dial tcp [::1]: connect: connection refused

3 Answers3