8

I'm trying to implement an asynchronous REST method of sending a message to Kafka in Spring MVC. Everything works, but when the server is unavailable, the onFailure event is processed for a long time. How to limit the response time in ListenableFuture for example to three seconds.

Here's my code:

@Autowired
KafkaTemplate<String, String> kafkaTemplate;

@Value("${spring.kafka.topic}")
String topic;

@RequestMapping("/test")
DeferredResult<ResponseEntity<?>> test(
        @RequestParam(value = "message") String message
) {

    DeferredResult<ResponseEntity<?>> deferredResult = new DeferredResult<>();
    ListenableFuture<SendResult<String, String>> future = kafkaTemplate.send(topic, "testKey", message);

    future.addCallback(new ListenableFutureCallback<SendResult<String, String>>() {

        @Override
        public void onSuccess(SendResult<String, String> sendResult) {
            ResponseEntity<String> responseEntity = new ResponseEntity<>("SUCCESS", HttpStatus.OK);
            deferredResult.setResult(responseEntity);
        }

        @Override
        public void onFailure(Throwable ex) {
            ResponseEntity<String> responseEntity = new ResponseEntity<>("FAILURE", HttpStatus.OK);
            deferredResult.setResult(responseEntity);
        }

    });

    return deferredResult;
}

I tried to use a REQUEST_TIMEOUT_MS_CONFIG property of Kafka and .get(long timeout, TimeUnit unit) method of ListenableFuture but havn't got desired result.

V. Perfilev
  • 438
  • 1
  • 7
  • 18
  • The `get()` should fail after the timeout; what is happening in that case? – Gary Russell Mar 26 '18 at 12:58
  • @GaryRussell If I add `future.get(1000, TimeUnit.MILLISECONDS);` before `return` I get following error in 60 seconds. Without this get-method I get correct "FAILURE" response in 60 seconds. `HTTP Status 500 - Request processing failed; nested exception is java.util.concurrent.ExecutionException: org.springframework.kafka.core.KafkaProducerException: Failed to send; nested exception is org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.` – V. Perfilev Mar 26 '18 at 13:57
  • See my answer; you can reduce `max.block.ms`. – Gary Russell Mar 26 '18 at 14:49

1 Answers1

8

That's because the producer blocks for 60 seconds (by default).

See max.block.ms in the KafkaDocumentation for producer configuration.

max.block.ms The configuration controls how long KafkaProducer.send() and KafkaProducer.partitionsFor() will block.These methods can be blocked either because the buffer is full or metadata unavailable.Blocking in the user-supplied serializers or partitioner will not be counted against this timeout.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179