Exception in PredictionEngineBase when using Time Series model with PredictionEnginePool (ML.NET)

Question

I've created a Time Series model using the method described here resulting in this code:

var data = items.ToArray();
var trainData = mlContext.Data.LoadFromEnumerable(data);

var estimator = mlContext.Forecasting.ForecastBySsa(
    nameof(FooPrediction.BarPrediction),
    nameof(FooInput.Bar),
    12,
    data.Length,
    data.Length,
    2,
    confidenceLowerBoundColumn: nameof(FooPrediction.ConfidenceLowerBound),
    confidenceUpperBoundColumn: nameof(FooPrediction.ConfidenceUpperBound));

var transformer = estimator.Fit(trainData);
using var engine = transformer.CreateTimeSeriesEngine<FooInput, FooPrediction>(mlContext);
engine.CheckPoint(mlContext, "model.zip");

where items is IEnumerable<FooInput>. These are my model classes:

public class FooPrediction
{
    public float[] BarPrediction { get; set; }

    public float[] ConfidenceLowerBound { get; set; }

    public float[] ConfidenceUpperBound { get; set; }
}

public class FooInput
{
    public float Bar { get; set; }

    public float Baz { get; set; }
}

In my Startup, I add a PredictionEnginePool thus:

services.AddPredictionEnginePool<FooInput, FooPrediction>().FromFile(String.Empty, "model.zip", true);

In my middleware service, I inject the PredictionEnginePool and then call:

var prediction = items.Select(i => predictionEnginePool.Predict(i));

where items is IEnumerable<FooInput>.

This results in an ArgumentOutOfRangeException being thrown in PredictionEngineBase.TransformerChecker:

Must be a row to row mapper (Parameter 'transformer')

Debugging into the code, I can see there is a check for IsRowToRowMapper on the ITransformer object being true. However, when the model is created, an SsaForecastingTransformer is created which has this property set to false.

Am I doing something wrong, or does PredictionEnginePool not support Time Series models?

I've also tried this with AddPredictionEnginePool<IEnumerable<FooInput>, FooPrediction> and then calling predictionEnginePool.Predict(items), but this also results in the same exception.

Answer 1

This code has been added after my time, so I don't have firsthand knowledge.

However, as far as I know ML.NET, the answer is yes: most likely, PredictionEnginePool does not support the time series prediction.

The reason is, the time-series prediction engine is actually a 'state machine'. You need to feed all the data, in the correct sequence, to one prediction engine, so that it correctly reacts to this 'time series'.

Prediction engine pool is solving a completely different scenario: if you have truly stateless models, you may instantiate a handful of interchangeable instances (a pool) of prediction engines, and the predictions will be handled with whatever engine is currently free.

These 'stateless' models are represented by a 'row to row mapper' concept in the codebase. Basically, the prediction of this model is determined exclusively, and solely, on one row of data.