disruptor producer too fast for consumer

Question

I am using a disruptor for some business logic which publishes to another disruptor that handles IO. The events published to the IO disruptor can arrive too fast to construct and validate the IO. Well, that's kind of the point...

The IO disruptor is setup like this:

disruptor = new Disruptor<>(factory, RING_SIZE, executor, ProducerType.SINGLE, new BlockingWaitStrategy());
disruptor.handleEventsWith(new Logic(disruptor, io));

Then the Logic event handler is setup like this:

public void onEvent(FixEvent event)
{
     quickfix.Message ioMessage = event.message;
     quickfix.SessionID receiver = event.session;

     Log.debug("message: " + event.message.toString());

     SessionID id = new SessionID(receiver.getBeginString(), "MYFX", receiver.getTargetCompID());
     Session session = Session.lookupSession(id);

     Log.debug("message: " + ioMessage.toString());

     session.send (ioMessage);
}

and what's happening is by the time you get to send (ioMessage) there's been a new event which is somehow overwriting the ioMessage, and therefore duplicate messages are being sent out.

What can you suggest?

I think you need to add some more details on what happens inside send() and the "do a little bit of work" section. Based on what you have shown above, you should not be having issues. Disruptor ensures you have only a single thread processing the ioMessage, but if you are sending that ioMessage out somewhere else you may be seeing issues. — jasonk, Sep 02 '15 at 21:26
@jasonk ok, filled it out some more. Maybe the issue is the log, in the first instance I am looking at the event.message and in the second instance looking at the ioMessage variable. This is where I am sometimes seeing different FIX event messages under load but nevertheless the problem is there are duplicate FIX messages sent to the receiver one after the other. When 2 events are published right after each other, the 2nd of them somehow overwrites the first and is sent out twice... weird. — rupweb, Sep 03 '15 at 07:09

rupweb · Answer 1 · 2015-09-17T11:06:32.610

-1

The answer looked like it was to finalize the event handler variables and use a synchronized final lock on the event as follows:

private final Object lock = new Object();

public void onEvent(final FixEvent event, final long sequence, final boolean endOfBatch)
{   
    synchronized(lock)
    {
        ...

It didn't work unless the lock was final. It didn't work with a static final lock. It didn't work synchronizing on anything else other than a final lock, ie synchronizing on the final event didn't work.

Then it worked 1 time, at band camp, then stopped working...

edited Sep 17 '15 at 11:06

answered Sep 14 '15 at 14:07

rupweb

3,052
1
30
57

related to http://stackoverflow.com/questions/32566752/disruptor-onevent-handler-appears-to-be-too-fast-for-java-synchronized-lock – rupweb Sep 14 '15 at 14:09
The disruptor has already ensured the event handler is running on its own thread, the problem has to exist further down the stack. – Sam Turtel Barker Sep 15 '15 at 03:23
@SamTurtelBarker well what's inside the synchronized(lock) block is to find the session id and send the message but somehow the message is being overwritten. – rupweb Sep 15 '15 at 16:35

disruptor producer too fast for consumer

1 Answers1