StreamsException when running @QuarkusTest with kafka streams in quarkus development mode

Question

Quarkus 2.15.3,

Problem When running my test case via gradle build or IDE (IntelliJ), everything is fine, test OK.
When starting application in Quarkus Dev Mode, test fails with following exception:

Caused by: org.apache.kafka.streams.errors.StreamsException: Unable to initialize state, this can happen if multiple instances of Kafka Streams are running in the same state directory (current state directory is [C:\Users\username\AppData\Local\Temp\kafka-streams\my-service]
        at org.apache.kafka.streams.processor.internals.StateDirectory.initializeProcessId(StateDirectory.java:191)

TopologyProducer, builds a simple topology, creating only a global state store

@ApplicationScoped
public class TopologyProducer {

    @ConfigProperty(name = "sometopic")
    String SOME_TOPIC;
    
    @Produces
    public Topology buildTopology() {
        final var builder = new StreamsBuilder();
    
        var keySerDe = new Serdes.StringSerde();
        var valueSerDe = new ObjectMapperSerde<>(MyValue.class);
    
        builder.globalTable(SOME_TOPIC, Consumed.with(keySerDe, valueSerDe), Materialized.as(MyStateStoreService.STORE_NAME));
        return builder.build();
    }

}

MyStateStoreService, reads a value from the global state store by key

@ApplicationScoped
public class MyStateStoreService {
    public static final String STORE_NAME = "MyStore";

    @Inject
    KafkaStreams streams;

    public MyValue valueById(String id) {
// Taken this from official documentation but tried with try-catch and streams.start() in catch-block too. Same result.
        while (true) {
            try {
                return (MyValue) streams.store(StoreQueryParameters
                        .fromNameAndType(STORE_NAME, QueryableStoreTypes.keyValueStore())).get(id);
            } catch (InvalidStateStoreException e) {
                // Ignore, store not ready yet
            }
        }
    }

}

MyStateStoreServiceTest, should test, that MyStateStoreService returns the newest version of a value for a specific key

@QuarkusTest
@QuarkusTestResource(KafkaCompanionResource.class)
public class MyStateStoreServiceTest {
    @InjectKafkaCompanion
    KafkaCompanion companion;

    @ConfigProperty(name = "sometopic")
    String SOME_TOPIC;
    
    @Inject
    MyStateStoreService myStateStoreService;
    
    @BeforeEach
    public void setUp() {
        companion.registerSerde(String.class, new Serdes.StringSerde());
        companion.registerSerde(MyValue.class, new ObjectMapperSerde<>(MyValue.class));
    }
    
    @Test
    public void valueUpdatesCorrectly() {
        // Given
        var key= "valueUpdatesCorrectly";
        var value = new MyValue("dummyValue");
        var valueUpdated= new MyValue("dummyValueUpdated");
    
        // When
        companion
                .produce(String.class, MyValue.class)
                .fromRecords(
                        new ProducerRecord<>(SOME_TOPIC, key, value),
                        new ProducerRecord<>(SOME_TOPIC, key, valueUpdated))
                .awaitCompletion();
    
        // Then
        assertEquals(valueUpdated, myStateStoreService.valueById(key));
    }

}

By now, in my application there was no problem in using Dev Mode, @QuarkusTestResource, KafkaCompanion and a Topology without state stores together. Other tests, using this approach (taken from Testing using a Kafka broker), run fine.

As soon as this global state store is added to my topology, I get above exception ONLY during continuous tests in dev mode (for readability reasons I removed the code for the rest of my topology from TopologyProducer)

Is there something wrong in my test setup, misunderstanding of dev mode and kafka companion from my site, or something else I missed?

StreamsException when running @QuarkusTest with kafka streams in quarkus development mode

0 Answers0