Rabbitmq HA synchronization

Question

I am trying to tune a 3 node rabbitmq cluster (don't have a separate load balancer in the configuration) by setting ha-sync-batch-size. After playing around with it I am observing that the latency of the failover seems to actually increase with the batch size setting set. It seems that the default value (every message) works better with faster switchover to a new master node. Is this the general observation or are there other considerations for setting the batch size?
To test I am using a load tester with 20-50 concurrent users. And shut down one node at a time starting with the master node. Typically, a few messages error out and then the new master node kicks in. Is there a better way of reducing the window for the new master node election? Any feedback would be appreciated.

This question is very unclear - I can't at all figure out how the answer relates to the question. — theMayer, Feb 02 '16 at 19:39
I suppose I should have split the question into 2 parts. Part 1 should be how to optimize the sync via configuration. Part 2 should have been - Since my observation in trying to optimize with batch size did not work in reducing the failover time what should be done. The solution I found was not via tweaking the configuration instead used the client to retry multiple times while the failover was taking place. Hope that clarifies the question. — sharman, Feb 02 '16 at 21:51
It might - I'm still having trouble understanding what you mean by "optimize the sync via configuration" - the problem you have is not easy to figure out. Is there a way you could clarify the specific problem? — theMayer, Feb 03 '16 at 19:52

score 0 · Answer 1 · answered Feb 01 '16 at 23:00

I think I found an answer. I failed to post that I am using RabbitTemplate. I wired a RetryTemplate into it as follows:

public @Bean RabbitTemplate templateFactory(){
    log.debug("Creating an template  factory.....");
    RabbitTemplate r=new RabbitTemplate(connectionFactory);
    r.setExchange(rabbitExchange);
    r.setRoutingKey(rabbitBinding);
    r.setRetryTemplate(retryTemplate());
    return r;
}

@Bean RetryTemplate retryTemplate(){
    RetryTemplate retryTemplate = new RetryTemplate();
    ExponentialBackOffPolicy backOffPolicy = new ExponentialBackOffPolicy();
    backOffPolicy.setInitialInterval(500);
    backOffPolicy.setMultiplier(10.0);
    backOffPolicy.setMaxInterval(5000);
    retryTemplate.setBackOffPolicy(backOffPolicy);
    SimpleRetryPolicy policy=new SimpleRetryPolicy();
    policy.setMaxAttempts(3);

    retryTemplate.setRetryPolicy(policy);
    return retryTemplate;
}

And, spring quite reliably retries (3 times in this case) enough number of times to send the request through. If anybody has a better idea please post. Thanks

Rabbitmq HA synchronization

1 Answers1