0

I am working to build a producer..consumer pattern using .NET reactive. Producer reads messages from Kafka message bus. Once the message is read, it needs to be handed over to the consumer to process the message.

I was able to do that using .NET reactive extensions (observable and observer). However, I will like to handle a scenario where the messages are read faster from the bus and consumer is falling behind. I will like the observable to react to back-pressure i.e. if observable indicates that it is still processing earlier messages, slow down the observable.

// Create a subject of my custom object and make it observable
 private readonly Subject<MyMessage> messageSubject = new Subject<MyMessage>();

messageSubject.AsObservable();

// add observer to the subject
_onMessageObservable.ObserveOn(NewThreadScheduler.Default)

// On receive of message, notify the observer
messageSubject.OnNext(msg.Value);
Ashish Bhatia
  • 161
  • 1
  • 6
  • Your code here doesn't make sense. Calling `messageSubject.AsObservable();` does nothing unless you assign the result to something. Also `_onMessageObservable.ObserveOn(NewThreadScheduler.Default)` doesn't add an observer to the subject. This just says what scheduler should be used to observe the values and again you must assign the result of this call to make it useful. – Enigmativity Aug 24 '17 at 00:27
  • Next, this is an observer pattern. For it to apply back-pressure it would have to be an "interferer" pattern. You can certainly apply techniques to consume chunks of the source to keep it up-to-date, but you'd need to specify how you want to do that. You can also write code with side-effects, but how we would do that with a Kafka message bus I don't know. You need to provide a [mcve] for us to work with. – Enigmativity Aug 24 '17 at 00:30

0 Answers0