5

Sorry for the question being too generic, but someone has some tutorial or guide on how to perform producer and consumer testing with kafka embedded. I've tried several, but there are several versions of dependencies and none actually works =/

I'm using spring cloud stream kafka.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
Tiago Costa
  • 1,004
  • 4
  • 17
  • 31

1 Answers1

12

We generally recommend using the Test Binder in tests but if you want to use an embedded kafka server, it can be done...

Add this to your POM...

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka-test</artifactId>
    <scope>test</scope>
</dependency>

Test app...

@SpringBootApplication
@EnableBinding(Processor.class)
public class So43330544Application {

    public static void main(String[] args) {
        SpringApplication.run(So43330544Application.class, args);
    }

    @StreamListener(Processor.INPUT)
    @SendTo(Processor.OUTPUT)
    public byte[] handle(byte[] in){
        return new String(in).toUpperCase().getBytes();
    }

}

application.properties...

spring.cloud.stream.bindings.output.destination=so0544out
spring.cloud.stream.bindings.input.destination=so0544in
spring.cloud.stream.bindings.output.producer.headerMode=raw
spring.cloud.stream.bindings.input.consumer.headerMode=raw
spring.cloud.stream.bindings.input.group=so0544

Test case...

@RunWith(SpringRunner.class)
@SpringBootTest
public class So43330544ApplicationTests {

    @ClassRule
    public static KafkaEmbedded embeddedKafka = new KafkaEmbedded(1);

    @Autowired
    private KafkaTemplate<byte[], byte[]> template;

    @Autowired
    private KafkaProperties properties;

    @BeforeClass
    public static void setup() {
        System.setProperty("spring.kafka.bootstrap-servers", embeddedKafka.getBrokersAsString());
    }

    @Test
    public void testSendReceive() {
        template.send("so0544in", "foo".getBytes());
        Map<String, Object> configs = properties.buildConsumerProperties();
        configs.put(ConsumerConfig.GROUP_ID_CONFIG, "test0544");
        configs.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
        ConsumerFactory<byte[], byte[]> cf = new DefaultKafkaConsumerFactory<>(configs);
        Consumer<byte[], byte[]> consumer = cf.createConsumer();
        consumer.subscribe(Collections.singleton("so0544out"));
        ConsumerRecords<byte[], byte[]> records = consumer.poll(10_000);
        consumer.commitSync();
        assertThat(records.count()).isEqualTo(1);
        assertThat(new String(records.iterator().next().value())).isEqualTo("FOO");
    }

}
Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • very good, Gary Russell finally, a test that works hahah. Ty very much. – Tiago Costa Apr 11 '17 at 17:40
  • Now I have tests that use the context of spring, but do not use kafka, simple tests of comparison between objects for example, this tests, throws connection refused erro with kafka when I execute all the testes. can i decide which tests will use kafka embedded and which ones will not use? – Tiago Costa Apr 11 '17 at 20:00
  • 1
    Not if you are testing a Spring Boot app with `EnableBinding`; the binder needs to connect. You have to isolate those tests from the boot environment. This is one reason why using the test binder is preferable. – Gary Russell Apr 11 '17 at 20:09
  • Note that the zkNodes is no longer required (since 2.0) since we no longer need to connect to zookeeper to provision topics. – Gary Russell Oct 05 '18 at 17:53
  • 1
    Can we write junit tests using TestSupportBinder for 'spring-cloud-stream-binder-kafka-streams' in which we process data using KStream not MessageChannel? – TimeTraveler Feb 06 '20 at 08:39
  • Don't ask new questions on old answers; no; the test binder doesn't support the KStream binder; you have to use an embedded Kafka broker. – Gary Russell Feb 06 '20 at 14:10