Can't get proper transformer for PFI

Question

I'm trying to follow the example for PFI. The last transform of my pipeline is a ColumnCopyingTransformer, so pipeline.Fit(trainingDataView).LastTransformer is that type. However, MLContext.MulticlassClassification.PermutationFeatureImportance expects IPredictionTransformer<IPredictor>. It feels like the example was crafted with a very specific pipeline in mind.

I got this tip via the ML.Net gitter channel:

You can iterate over your transformer chain pipeline.Fit(trainingDataView) until you find something which is IPredictionTransformer

However, I don't see how to perform that iteration. I tried the following but get null for predictor.

var trainedModel = (TransformerChain<ITransformer>)mlContext.Model.Load(stream);
var predictor = trainedModel.OfType<IPredictionTransformer<IPredictor>>().FirstOrDefault();

What am I missing?

Answer 1

Apparently, transformer chains can have child transformer chains as of ML.Net 0.10. The following returns the desired transformer:

public static IPredictionTransformer<IPredictor> FindPredictionTransformer(this TransformerChain<ITransformer> chain)
{
    var xf = chain.OfType<IPredictionTransformer<IPredictor>>().FirstOrDefault();
    if (xf != null) return xf;

    foreach (var t in chain.OfType<TransformerChain<ITransformer>>())
    {
        var xfChild = t.OfType<IPredictionTransformer<IPredictor>>().FirstOrDefault();
        if (xfChild != null) return xfChild;
    }

    return null;
}

Usage:

TransformerChain<ITransformer> trainedModel;

using (var stream = new FileStream(modelPath, FileMode.Open, FileAccess.Read, FileShare.Read))
{
    trainedModel = (TransformerChain<ITransformer>)mlContext.Model.Load(stream);
}

var predictionTransformer = trainedModel.FindPredictionTransformer();

var featureImportance = PfiHelper.CalculatePfi(mlContext, predictionTransformer, trainingDataView);

It's unclear to me why it was necessary to iterate child TransformerChain<ITransformer> or whether it might be necessary to search deeper (grandchildren, etc) in some cases. I'd appreciate a comment or additional answer clarifying that issue.