0

In our ActivePivot solution we have written a post processor that computes the price of an stock option depending on the price of the stock (and a volatility parameter). When it is evaluated the post processor connects (for now) to the Google Finance service to retrieve the stock price on the fly. So each time a user makes a query on ActivePivot the aggregates are computed in real-time with the latest prices.

But we would also like to leverage the continuous queries in ActivePivot and have the aggregates that changed pushed to the users in real-time (instead of periodically hitting the refresh button of ActivePivot Live). We know that it is usually implemented by writing a continuous handler, one that would propagate price change events to ActivePivot and let ActivePivot compute the impact on subscribed queries. But Google Finance does not offer a push API, and if we hammer the service with periodic polling on hundreds of stocks we will get banned.

What mechanism do you recommend in ActivePivot to workaround this issue?

Jack
  • 145
  • 1
  • 1
  • 11

1 Answers1

0

When the data source of a post processor is too unpredictable (like a remote web service with no push capability like in your case), the best way to go is to create a stream and a handler for this post processor that tell ActivePivot the whole result of this post processor has changed every N second.

You can create a TickingStream, which send an (empty) event with a fixed period like this:

package com.quartetfs.pivot.sandbox.postprocessor.impl;

import java.util.Timer;
import java.util.TimerTask;

import com.quartetfs.biz.pivot.IActivePivotSession;
import com.quartetfs.biz.pivot.query.aggregates.IAggregatesContinuousQueryEngine;
import com.quartetfs.biz.pivot.query.aggregates.impl.AStream;
import com.quartetfs.fwk.QuartetExtendedPluginValue;
import com.quartetfs.fwk.types.impl.ExtendedPluginInjector;

/**
 * Stream sending an event at a regular rate.
 *
 */
@QuartetExtendedPluginValue(interfaceName = "com.quartetfs.biz.pivot.query.aggregates.IStream", key = TickingStream.PLUGIN_KEY)
public class TickingStream extends AStream<Void> {

    private static final long serialVersionUID = 1L;

    public static final String PLUGIN_KEY = "TICKING";

    /** The default ticking period, in ms. **/
    protected static final long DEFAULT_PERIOD = 1000;

    /** The ticking period, in ms. **/
    protected long period = DEFAULT_PERIOD;

    /** The task responsible for sending the ticking events. */
    protected final TimerTask sendEventTask;

    /** The timer that schedules the {@link #sendEventTask}. */
    protected Timer tickingTimer;

    /**
     * Create a ticking stream.
     *
     * @param engine
     * @param session
     */
    public TickingStream(IAggregatesContinuousQueryEngine engine,
            IActivePivotSession session) {
        super(engine, session);

        // Create the task that will send the events.
        sendEventTask = new TimerTask() {

            @Override
            public void run() {
                sendEvent(null);
            }
        };
        // Schedule this task with the default period:
        rescheduleTask();
    }

    /**
     * Schedule the {@link #sendEventTask} with the {@link #period} period.
     * Removes also all previous scheduling of this task.
     */
    protected void rescheduleTask() {
        if (tickingTimer != null) {
            tickingTimer.cancel();
        }
        tickingTimer = new Timer();
        tickingTimer.schedule(sendEventTask, 0, period);
    }

    /**
     * Change the ticking period of this stream. This will reschedule the task
     * according to this new period. This setter will be called via
     * {@link ExtendedPluginInjector extended plugin injection}
     *
     * @param period the period to set. Must be strictly positive.
     *
     * @throws IllegalArgumentException if period is smaller or equal to 0.
     */
    public void setPeriod(long period) {
        if (period <= 0) {
            throw new IllegalArgumentException("Non-positive period.");
        }
        this.period = period;
        rescheduleTask();
    }

    /** {@inheritDoc} */
    @Override
    public Class<Void> getEventType() {
        return Void.class;
    }

    /** {@inheritDoc} */
    @Override
    public String getType() {
        return PLUGIN_KEY;
    }
}

And the handler that ask to refresh the part of all continuous queries related to the post processor having this handler at each tick:

package com.quartetfs.pivot.sandbox.postprocessor.impl;

import java.util.Collections;

import com.quartetfs.biz.pivot.IActivePivot;
import com.quartetfs.biz.pivot.ILocation;
import com.quartetfs.biz.pivot.query.aggregates.IImpact;
import com.quartetfs.biz.pivot.query.aggregates.impl.AAggregatesContinuousHandler;
import com.quartetfs.biz.pivot.query.aggregates.impl.Impact;
import com.quartetfs.fwk.QuartetExtendedPluginValue;

/**
 * The handler associated with a {@link TickingStream}.
 *
 * This handler asks for a full refresh of the locations queried on
 * post-processors with this handler each time it receives a tick.
 * <p>
 * This is the handler to use for post processors that have unpredictable data
 * sources which prevent the creation of a stream and a handler that can decide
 * which subset of the currently queried locations should be updated in a
 * continuous query.
 *
 */
@QuartetExtendedPluginValue(interfaceName = "com.quartetfs.biz.pivot.query.aggregates.IAggregatesContinuousHandler", key = TickingStream.PLUGIN_KEY)
public class TickingHandler extends AAggregatesContinuousHandler<Void> {

    private static final long serialVersionUID = 1L;

    /**
     * @param pivot
     */
    public TickingHandler(IActivePivot pivot) {
        super(pivot);
    }

    /**
     * {@inheritDoc}
     * <p>
     * The impact on a queried location is the whole location since there is no
     * way for us to know which part of the location should be updated or not.
     */
    @Override
    public IImpact computeImpact(ILocation location, Void event) {
        return new Impact(location, Collections.singleton(location), Collections.singleton(location));
    }

    /** {@inheritDoc} */
    @Override
    public String getStreamKey() {
        // This handler is made to be used with the TickingStream.
        return TickingStream.PLUGIN_KEY;
    }

    /** {@inheritDoc} */
    @Override
    public String getType() {
        return TickingStream.PLUGIN_KEY;
    }
}

And you configure your post processor to use this handler this way:

<measure name="..." folder="..." aggregationFunctions="...">
            <postProcessor pluginKey="yourPPpluginKey">
                <properties>
                    <entry key="continuousQueryHandlerKeys" value="TICKING" />
                </properties>
            </postProcessor>
        </measure>

In a way, you will be polling Google Finance service since your post processor will be called every second (default period). But this will only happen if there is a user making a continuous query using your post processor, and your post processor will only be called on the locations that are queried by the users, so hopefully you will only need a very small subset of Google Finance information. Besides, calls to your post processor will be shared among multiple users, so that you issue the minimum amount of queries against Google Finance.

jolivier
  • 7,380
  • 3
  • 29
  • 47