c# AutoML Regression training wont show Rsquared or MeanAbsoluteError

Question

I am using c# AutoML to train a model for Regression and i cant see the Rsquared or MeanError for any of the algorithms

        //loading train data through Text Loader
        var trainData = loader.Load(_filePath);
        Console.WriteLine("created train data");

        var settings = new RegressionExperimentSettings
        {
            MaxExperimentTimeInSeconds = 10,
            //OptimizingMetric = RegressionMetric.MeanAbsoluteError
        };

        var progress = new Progress<RunDetail<RegressionMetrics>>(p =>
        {
            if (p.ValidationMetrics != null)
            {
                Console.WriteLine($"Current Result - {p.TrainerName}, {p.ValidationMetrics.RSquared}, {p.ValidationMetrics.MeanAbsoluteError}");
            }
        });

        var experiment = context.Auto().CreateRegressionExperiment(settings);
        //find best model

        var labelColumnInfo = new ColumnInformation()
        {
            LabelColumnName = "median_house_value"
        };

        var result = experiment.Execute(trainData, labelColumnInfo, progressHandler: progress);
        Console.WriteLine(Environment.NewLine);
        Console.WriteLine("Best run:");
        Console.WriteLine($"Trainer name - {result.BestRun.TrainerName}");
        Console.WriteLine($"RSquared - {result.BestRun.ValidationMetrics.RSquared}");
        Console.WriteLine($"MAE - {result.BestRun.ValidationMetrics.MeanAbsoluteError}");
        Console.ReadLine();

When i run the console application the outputs i get are 0 -infinite or not a number

score 0 · Answer 1 · answered Feb 14 '20 at 21:04

I've gotten similar results when my dataset has been too small.

AutoML is using 10 fold cross validation if I recall correctly. This can lead to the test data set being too small to get any usable metrics out of it.

So if your dataset is small, you could try with a bigger one and see if it gets better metrics, at least to rule out that case.

c# AutoML Regression training wont show Rsquared or MeanAbsoluteError

1 Answers1