How To Evaluate Predictive Neural Network in Encog

Question

We're a creating a neural network for predicting typhoon occurrence using several typhoon parameters as input. So far, we have been able to generate data and train the neural network using Encog 3.2. Right now, we need to evaluate the results of the training.

We're using the ForestCover project (in Encog 3.2 examples) as reference, however the said project's Evaluation code is for a classification neural network. Thus, we can't evaluate our neural network following said project's code.

We also checked the PredictMarket project (in Encog 3.2 examples) since it is a predictive neural network. But we're having a difficulty in using the MLData.

MLData output = network.compute(inputData);

We want to extract the contents of output and compare it to the contents of the evaluation.csv for neural-network evaluation.

Is there a way we can extract/convert the output variable into a normalized value which we can then compare to the normalized evaluation.csv?

or

Is there a way we can modify the ForestCover Evaluate.java file to be able to evaluate a predictive neural network?

Thank you.

@YuraZaletskyy We normalized our data using the built-in data generation in the ForestCover example in Encog 3.2 via OutputFieldRangeMapped. It automatically divides the data (75%-25%) into the training.csv and evaluation.csv files. — Pem Reyes, Jan 25 '16 at 05:07

score 1 · Answer 1 · answered Jan 20 '16 at 02:10

Here is a C# example (Java should be similar) which writes out a .csv file (TestResultsFile) with both the de-normalized expected and actual results so you can compare them with an Excel graph.

var evaluationSet = (BasicMLDataSet)EncogUtility.LoadCSV2Memory(Config.EvaluationNormalizedFile.ToString(),
    network.InputCount, network.OutputCount, true, CSVFormat.English,
        false);

var analyst = new EncogAnalyst();
analyst.Load(Config.NormalizationAnalystFile);

// Change this to whatever your output field index is
int outputFieldIndex = 29;

using (var resultsFile = new System.IO.StreamWriter(Config.TestResultsFile.ToString()))
{
    foreach (var item in evaluationSet)
    {
        var normalizedActualOuput = (BasicMLData)network.Compute(item.Input);
        var actualOutput = analyst.Script.Normalize.NormalizedFields[outputFieldIndex].DeNormalize(normalizedActualOuput.Data[0]);

        var idealOutput = analyst.Script.Normalize.NormalizedFields[outputFieldIndex].DeNormalize(item.Ideal[0]);


        var resultLine = String.Format("{0},{1}", idealOutput, actualOutput);
        resultsFile.WriteLine(resultLine);

    }
}

Much of this came from ideas from Abishek Kumar's Pluralsight Course

If you really want to compare Normalized values, just remove the two calls to "Denormalize".

Thank you very much for this insight! We will try following this code, and we will update you of the results. — Pem Reyes, Jan 25 '16 at 05:10

How To Evaluate Predictive Neural Network in Encog

1 Answers1