Connection refused can not use retry when use ribbon

Question

info:

spring-cloud.version=Dalston.SR5
spring-boot-starter-parent=1.5.7.RELEASE
zuul-core=1.3.0

When one client's status is down(I mean kill it),but zuul can also have cache of instance info, so request will route to down client, but can't have a correct response.so zuul do retry, route request to another client and get correct response. This condition like restart my client.

so I have some configurations:

ribbon:
    MaxAutoRetries: 1
    MaxAutoRetriesNextServer: 2
    OkToRetryOnAllOperations: true
    ReadTimeout: 1000
    ConnectTimeout: 250
    ServerListRefreshInterval: 1000
zuul:
  retryable: true

And I use spring-retry jar:

<dependency>
    <groupId>org.springframework.retry</groupId>
    <artifactId>spring-retry</artifactId>
</dependency>

but it doesn't work as I think. so I debug the code. RetryableRibbonLoadBalancingHttpClient will be used when zuul route request. the exception throw on

org.springframework.cloud.netflix.ribbon.apache.RetryableRibbonLoadBalancingHttpClient#execute
final HttpResponse httpResponse = RetryableRibbonLoadBalancingHttpClient.this.delegate.execute(httpUriRequest);

I found the retry code after it, so it can't go to retry logic. Can you give some suggests to fix this problem? Exception info :

Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 172.16.73.151:8701 [/172.16.73.151] failed: Connection refused
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.3.jar:4.5.3]
    at org.springframework.cloud.netflix.ribbon.apache.RetryableRibbonLoadBalancingHttpClient$1.doWithRetry(RetryableRibbonLoadBalancingHttpClient.java:89) ~[spring-cloud-netflix-core-1.3.6.RELEASE.jar:1.3.6.RELEASE]
    at org.springframework.cloud.netflix.ribbon.apache.RetryableRibbonLoadBalancingHttpClient$1.doWithRetry(RetryableRibbonLoadBalancingHttpClient.java:71)

It should work. Could you post original exception that was caused by 'org.apache.http.conn.HttpHostConnectException' — yongsung.yoon, May 31 '18 at 04:14
The problem is very obvious. This code throw a exception: `final HttpResponse httpResponse = RetryableRibbonLoadBalancingHttpClient.this.delegate.execute(httpUriRequest);` The code is in RetryableRibbonLoadBalancingHttpClient. And this code can't be execute： `this.executeWithRetry(request, retryPolicy, retryCallback);` — Hope Dc, Jun 02 '18 at 15:44
I guess that you misunderstood the code. `....this.delegate.execute` code exist inside `RetryCallback retryCallback` and this will be executed by `this.executeWithRetry`. And it will execute this retryCallback with RetryTemplate. That is how retry works in Spring Cloud Zuul. — yongsung.yoon, Jun 04 '18 at 09:44
You are right!The exception is happen when retry request.I found the issue https://github.com/spring-cloud/spring-cloud-netflix/issues/2667 and thank very much.because you said,I read codes and test many times and found this bug codes. — Hope Dc, Jun 07 '18 at 02:11

score 1 · Answer 1 · answered Jun 02 '18 at 09:00

I recommend you to add follow configuration and check if it works :

@Configuration
public class RibbonConfig {

@Value("${ribbon.OkToRetryOnAllOperations}")
private Boolean retryOnAllOperations;

@Value("${ribbon.MaxAutoRetries}")
private Integer maxAutoRetries;

@Value("${ribbon.MaxAutoRetriesNextServer}")
private Integer maxAutoRetriesNextServer;

@Bean
public RetryHandler retryHandler() {
    IClientConfig clientConfig = DefaultClientConfigImpl
            .Builder
            .newBuilder()
            .withMaxAutoRetries(maxAutoRetries)
            .withMaxAutoRetriesNextServer(maxAutoRetriesNextServer)
            .withRetryOnAllOperations(retryOnAllOperations)
            .build();

    RetryHandler retryHandler = new DefaultLoadBalancerRetryHandler(maxAutoRetries, maxAutoRetriesNextServer, retryOnAllOperations);
    return new RequestSpecificRetryHandler(true, true, retryHandler, clientConfig );
 }

@Bean
@Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
public IRule ribbonRule() {
    return new RetryRule();
 }
}

Connection refused can not use retry when use ribbon

1 Answers1