Embedded Kafka Refresh Messages Between Tests

Question

I'm using Spring Kafka with the the @EmbeddedKafka annotation to start the Embedded kafka instance.

Dependency:

<dependency>
  <groupId>org.springframework.kafka</groupId>
  <artifactId>spring-kafka-test</artifactId>
  <scope>test</scope>
</dependency>

And I've configured my application to connect to it:

spring:
  kafka:
    bootstrap-servers: ${spring.embedded.kafka.brokers}

I am successfully sending messages and consuming messages.

My problem is that the messages are retained somewhere and may appear in proceeding test runs. And the logs are printing these:

Successfully joined group with generation 117

The high generation number indicates that Kafka is retaining information between test reruns that i do not want.

How do I completely clean up Embedded Kafka and start fresh?

EDIT:

The issue I had was I was using Spring Profiles incorrectly. I had a custom annotation annotation with my embedded Kafka configuration. The problem was I was setting @ActiveProfiles("kafka") in the compound annotation, and also setting @ActiveProfiles("dev") in the actual test class. I've updated the compound annotation to this:

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
// @ActiveProfiles("kafka") REMOVED
@TestPropertySource(properties = "spring.kafka.bootstrap-servers=${spring.embedded.kafka.brokers}")
@EmbeddedKafka(/* config properties */)
public @interface CustomEmbeddedKafka{
}

Artem Bilan · Accepted Answer · 2019-04-10T15:21:02.597

5

You need to use @DirtiesContext to clean an application context in between tests. Together with that the embedded Kafka broker is going to be destroyed as well.

If the problem is between one test class methods, then you need to rethink the logic over there do not interfere with the same topics in different methods.

You also can consider to use @DirtiesContext(classMode = DirtiesContext.ClassMode.BEFORE_EACH_TEST_METHOD), but this is going to be some kind of performance degradation, since an application context together with an embedded Kafka is going to be recreated before every test method.

UPDATE

Try this suggestion instead of application.properties modification:

static {
    System.setProperty(EmbeddedKafkaBroker.BROKER_LIST_PROPERTY,
            "spring.kafka.bootstrap-servers");
}

See https://docs.spring.io/spring-kafka/docs/2.2.5.RELEASE/reference/#kafka-testing-embeddedkafka-annotation

edited Apr 10 '19 at 15:21

answered Apr 10 '19 at 15:08

Artem Bilan

113,505
11
91
118

`@DirtiesContext` does not work. The Embedded Kafka is retained even outside of the `applicationContext` for some reason. It's not just between tests in a test group. Messages are picked up like a day after the previous test. – George Apr 10 '19 at 15:13
Then it looks like you connect to the real Kafka outside of your project... – Artem Bilan Apr 10 '19 at 15:18
See an UPDATE in my answer. – Artem Bilan Apr 10 '19 at 15:21
Looks like this was the case. I was just using Spring Profiles incorrectly. – George Apr 10 '19 at 15:28

Embedded Kafka Refresh Messages Between Tests

1 Answers1