0

What is the best possible way to create a consolidated event (output) from 2 different asynchronous input event in the realtime?

Take for example

"input_event_A" : { "correlation_id" : "abc_123", "payload_event_a": { some event A data } }

"input_event_B" : { "correlation_id" : "abc_123", "payload_event_b": { some event B data } }

So when the event B arrives, i need to consolidate the payload data from A and B based on the same correlation_id

The output I am looking for is lets say for event C

"output_event_C" : { "correlation_id" : "abc_123", "payload_event_C": { some event A+B data } }

The challenge is that event A and B are asynchronous and in the event listener either A or B can be picked up at any given time and since the consolidation process should be triggered only when event B is processed, there can be possibility that during that time the event A has not yet arrived, how to solve for that?

The current approach i am using is

I am saving both the events asynchronously in 2 separate event stores. When the event B comes in, i am searching the DB for correlation_id in event Store for A, and if its there i pull the data and consolidate the event and on the flip side, if the data is not there in event store A, i am retrying infinitely till the data in event A becomes available. This retry process can create race conditions and use lot of resources and hence i am looking for a better way to deal with this consolidation.

Note: The 2 input events come on a JMS Q for which i have a Java based listener to process these events.

user1697113
  • 683
  • 2
  • 14
  • 24

1 Answers1

0

CEP engine would be the best.

But something like Guava Cache is adequate in this case as the only thing needed is global map.

class Example {

    class MessageListener1 {
        void onMessage(Message msg) {
            Event e = toEvent(msg);
            fireOn(e);
        }
    }

    class MessageListener2 {
        void onMessage(Message msg) {
            Event e = toEvent(msg);
            fireOn(e);
        }
    }


   Cache<String, Event> eventStorage = CacheBuilder.newBuilder()
            .expireAfterWrite(1, TimeUnit.HOURS)
            .build();

    void fireOn(Event e1) {
        Event e2 = eventStorage.get(e1.correlId(), () -> e1);
        if (e1.equals(e2)) {
            // nothing to do yet
            return;
        }
        
        // we have two corelated events 
        Event e3 = doSomething(e1, e2);        
    }
}

Guava Cache is nice as expiry will take care of freeing memory, and get with loader method takes care of inherit race conditions.

Talijanac
  • 1,087
  • 9
  • 20