0

I've start dipping toes in Spring Cloud Stream and it's Cloud Function support.

I've uploaded a sample project to elucidate this problem here -> https://github.com/nmarquesantos/spring-cloud-function-kafka

I have a project that exposes a few functions in a reactive manner, with kafka as the message broker.

The function receives a Flux and proceeds to save the elements via the Reactive Mongo library. It then returns the updated resource via another flux.

@Service
public class ExampleCloudFunction {

@Autowired
private PlayerRepository playerRepository;

@Bean
public Function<Flux<Player>, Flux<Player>> playerUpdate() {
    return flux -> flux.flatMap(player -> playerRepository.save(player)).log("Saved player");
}

@PollableBean
public Supplier<Flux<Player>> playerFeeder() {
   return () -> Flux.just(new Player(UUID.randomUUID().toString(), "Ronaldo"));
}
}

The playerUpdate function is where the errors occur. playerFeeder is simply a function I created to send data to reproduce the problem. In real life, this would come from a different service.

Here's a snippet of the errors by running the sample project i mentioned above:

2020-05-21 22:13:40.842 ERROR 1884 --- [container-0-C-1] onfiguration$FunctionToDestinationBinder : Failed to process the following content which will be dropped: Context1{reactor.onNextError.localStrategy=reactor.core.publisher.OnNextFailureStrategy$ResumeStrategy@2c3e726}

org.springframework.transaction.reactive.TransactionContextManager$NoTransactionInContextException: No transaction in context

2020-05-21 22:13:41.853 ERROR 1884 --- [container-0-C-1] onfiguration$FunctionToDestinationBinder : Failed to process the following content which will be dropped: Context1{reactor.onNextError.localStrategy=reactor.core.publisher.OnNextFailureStrategy$ResumeStrategy@2c3e726}

org.springframework.transaction.reactive.TransactionContextManager$NoTransactionInContextException: No transaction in context

I'm struggling to understand what I've done wrong and couldn't find much information in my search.

nmarques
  • 151
  • 12

1 Answers1

0

Spring for Apache Kafka does not currently support reactive transactions.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • so is this a case of having to avoid using spring kafka together with reactive repositories? – nmarques May 21 '20 at 14:37
  • 1
    Async processing of kafka data is tricky because discrete messages are not acknowledged, only an offset per partition is maintained. We just haven't investigated it yet. I opened an issue. – Gary Russell May 21 '20 at 14:39
  • ok thanks a lot Gary :) for now I'll investigate using non-reactive repositories. – nmarques May 21 '20 at 14:43
  • Oh; sorry; I assumed you were trying to use reactive transactions with Kafka; if you just want reactive transactions with Mongo; it looks like you need to start a transaction. See [here](https://docs.spring.io/spring-data/mongodb/docs/3.0.0.RELEASE/reference/html/#mongo.transactions.reactive). (My comments about async and Kafka still apply though) – Gary Russell May 21 '20 at 15:25
  • i was just trying to chain the publishers, i.e. chain the flux that's coming from kafka and then receive the output from reactive mongo to publish it onto another topic – nmarques May 21 '20 at 15:36