0

I have spring application with some embedded kafka tests @EmbeddedKafka. For each test I need a separate topic.

@ExtendWith(SpringExtension::class)
@SpringBootTest(
    classes = [
        Application::class
    ]
)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@EmbeddedKafka(
    partitions = 1,
    topics = [
        "topicName"
    ]
)
@ActiveProfiles("test")
@Import(value = [ApplicationListenerCaseTest.TestConfig::class])
class ApplicationListenerCaseTest {

    @Autowired
    private lateinit var testKafkaTemplate: KafkaTemplate<String, String>

    @Autowired
    private lateinit var kafkaListenerEndpointRegistry: KafkaListenerEndpointRegistry

    @Autowired
    private lateinit var kafkaEmbeddedKafka: EmbeddedKafkaBroker

    override fun getTestKafkaTemplate() = testKafkaTemplate

    @BeforeEach
    fun setup() {
        for (msgListenerContainer in kafkaListenerEndpointRegistry.listenerContainers) {
            ContainerTestUtils.waitForAssignment(msgListenerContainer, kafkaEmbeddedKafka.partitionsPerTopic)
        }
    }

    @Test
    fun test() {
         // ...     
    }

    @TestConfiguration
    class TestConfig {

        @Bean
        @Primary
        fun testKafkaTemplate(
            @Value("topicName") topic: String,
            broker: EmbeddedKafkaBroker
        ) = KafkaTemplate(DefaultKafkaProducerFactory<String, String>(KafkaTestUtils.producerProps(broker)))
            .apply { defaultTopic = topic }
    }
}

There are several such tests and they take a very long time.I think the long execution is due to the fact that a separate kafka broker is created for each test.

Is there a way to run all tests in one broker and to avoid wasting time creating a new context for every test?

olga1234
  • 1

1 Answers1

0

See the documentation: https://docs.spring.io/spring-kafka/docs/current/reference/html/#same-broker-multiple-tests

You can use the same broker for multiple test classes with something similar to the following:

public final class EmbeddedKafkaHolder {

    private static EmbeddedKafkaBroker embeddedKafka = new EmbeddedKafkaBroker(1, false)
            .brokerListProperty("spring.kafka.bootstrap-servers");

    private static boolean started;

    public static EmbeddedKafkaBroker getEmbeddedKafka() {
        if (!started) {
            try {
                embeddedKafka.afterPropertiesSet();
            }
            catch (Exception e) {
                throw new KafkaException("Embedded broker failed to start", e);
            }
            started = true;
        }
        return embeddedKafka;
    }

    private EmbeddedKafkaHolder() {
        super();
    }

}

This assumes a Spring Boot environment and the embedded broker replaces the bootstrap servers property.

Then, in each test class, you can use something similar to the following:

static {
    EmbeddedKafkaHolder.getEmbeddedKafka().addTopics("topic1", "topic2");
}

private static final EmbeddedKafkaBroker broker = EmbeddedKafkaHolder.getEmbeddedKafka();

...

The preceding example provides no mechanism for shutting down the broker(s) when all tests are complete. This could be a problem if, say, you run your tests in a Gradle daemon. You should not use this technique in such a situation, or you should use something to call destroy() on the EmbeddedKafkaBroker when your tests are complete.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • I have kafka cluster configuration in application.yaml. For test I set bootstrap-servers: ${spring.kafka.bootstrap-servers}, but there is exception `Invalid url in bootstrap.servers: ${spring.kafka.bootstrap-servers}` – olga1234 Aug 29 '23 at 10:52
  • `bootstrap.servers` (with a `.`) is a Kafka property. Kafka knows nothing about Spring or its property placeholders. You don't need anything in the YAML, the `.brokerListProperty("spring.kafka.bootstrap-servers")` above sets the Boot property to the address of the embedded broker; boot then sets `bootstrap.servers` with that value. – Gary Russell Aug 29 '23 at 13:33