0

I have some spring boot tests that are using Kafka internally. For Kafka functionality, I am using @EmbeddedKafka annotation on each of the test classes that are using Kafka (with same server on each test, localhost:9092).

@SpringBootTest
@DirtiesContext
@EmbeddedKafka(partitions = 1, brokerProperties = { "listeners=PLAINTEXT://localhost:9092", "port=9092" })
class SampleIntegrationTest extends Base integration test {
   // code
}

The tests are passing when I run individually or sequentially in test suite but failing when I run the test suite parallelly. In junit-platform.properties setting:

junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = same_thread
junit.jupiter.execution.parallel.mode.classes.default = concurrent

Getting resource access exception on some of the tests. Is there someway these tests can be run parallelly with @EmbeddedKafka?

Also just to mention, some of these Kafka related integration tests internally sends data on same topics.

Thanks.

rMonteiro
  • 1,371
  • 1
  • 14
  • 37
dh1
  • 97
  • 1
  • 9
  • 1
    See if this one helps you somehow: https://docs.spring.io/spring-kafka/reference/html/#same-broker-multiple-tests – Artem Bilan Apr 12 '23 at 16:58

1 Answers1

0

When running your tests in parallel, you are encountering resource access exceptions because multiple tests are trying to use the same Kafka broker port (9092) and the same topics. To avoid these conflicts, you can create a unique Kafka broker and topics for each test.

To achieve this, you can use dynamic ports and unique topics for each test.

Update the @EmbeddedKafka annotation to use a dynamic port:

@EmbeddedKafka(partitions = 1, brokerProperties = { "listeners=PLAINTEXT://localhost:0" })

Inject the EmbeddedKafkaBroker instance into your test class and configure your Kafka producer and consumer with the dynamically assigned port:

@SpringBootTest
@DirtiesContext
@EmbeddedKafka(partitions = 1, brokerProperties = { "listeners=PLAINTEXT://localhost:0" })
class SampleIntegrationTest extends BaseIntegrationTest {

    @Autowired
    private EmbeddedKafkaBroker embeddedKafkaBroker;

    @BeforeEach
    void setUp() {
        Map<String, Object> producerProps = KafkaTestUtils.producerProps(embeddedKafkaBroker);
        // configure your Kafka producer using producerProps

        Map<String, Object> consumerProps = KafkaTestUtils.consumerProps("testGroup", "true", embeddedKafkaBroker);
        // configure your Kafka consumer using consumerProps
    }

    // code
}

Use unique topics for each test. Instead of using hardcoded topic names, you can generate unique topic names for each test:

class SampleIntegrationTest extends BaseIntegrationTest {

    private String uniqueTopic;

    @BeforeEach
    void setUp() {
        uniqueTopic = UUID.randomUUID().toString();
        // ...
    }

    // code
}
rMonteiro
  • 1,371
  • 1
  • 14
  • 37
  • Using different ports and topics for each test might not be possible since the kafka service logic within is pushing messages to a pre-defined Kafka host and with pre-defined topics. Is there some other way round to this? With or without @EmbeddedKafka? – dh1 Apr 13 '23 at 07:39