How to make predictions with Syncfusion PMML from a Neural Network trained with sklearn MLPClassifier?

Question

I trained a model in Python using sklearn.neural_network.MLPClassifier (0.20.3) and saved it in PMML format using sklearn2pmml (0.48.0). The saved PMML model works as expected when loaded in Java using org.jpmml:pmml-evaluator:1.4.14.

I now want to load the PMML model and make predictions in C# using the Syncfusion package:

      <ItemGroup>
        <PackageReference Include="Syncfusion.PMML.AspNet" Version="17.4.0.44" />
      </ItemGroup>

using System;
using Syncfusion.PMML;

namespace myprogram
{
    class Program
    {
        static void Main(string[] args)
        {

            var predictors = new           
                {                
                predictor_1 = 0.05,                
                predictor_2 = 203.0,               
                predictor_3 = 400.0,
                predictor_4 = 22.0,
                predictor_5 = 9.01         
                };

            string PmmlFilePath = “/project/model.pmml";  

            //Create instance for PMML Document            
            PMMLDocument pmmlDocument = new PMMLDocument(PmmlFilePath);            

            //Create instance for Mining model            
            NeuralNetworkModelEvaluator neuralNetworkModel = new NeuralNetworkModelEvaluator(pmmlDocument);            

            //Gets the predicted result            
            PredictedResult predictedResult = neuralNetworkModel.GetResult(predictors, null);
        }
    }
}

but the last line of the above code raises the following exception:

Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at Syncfusion.PMML.NeuralNetworkModelEvaluator.ComputeResult(Dictionary`2 fieldValuePair, NeuralNetworkModel neuralNetworkModel)
   at Syncfusion.PMML.NeuralNetworkModelEvaluator.GetResult(Object obj, IModelOptions modelOptions)
   at myprogram.Program.Main(String[] args) in /project/Program.cs:line 66

model.pmml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<PMML xmlns="http://www.dmg.org/PMML-4_3" xmlns:data="http://jpmml.org/jpmml-model/InlineTable" version="4.3">
    <Header>
        <Application name="JPMML-SkLearn" version="1.5.20"/>
        <Timestamp>2020-20-15T03:42:46Z</Timestamp>
    </Header>
    <DataDictionary>
        <DataField name="target_state" optype="categorical" dataType="string">
            <Value value="RED"/>
            <Value value="GREEN"/>
        </DataField>
        <DataField name="predictor_1" optype="continuous" dataType="double"/>
        <DataField name="predictor_2" optype="continuous" dataType="double"/>
        <DataField name="predictor_3" optype="continuous" dataType="double"/>
        <DataField name="predictor_4" optype="continuous" dataType="double"/>
        <DataField name="predictor_5" optype="continuous" dataType="double"/>
    </DataDictionary>
    <TransformationDictionary/>
    <MiningModel functionName="classification">
        <MiningSchema>
            <MiningField name="target_state" usageType="target"/>
            <MiningField name="predictor_1"/>
            <MiningField name="predictor_2"/>
            <MiningField name="predictor_3"/>
            <MiningField name="predictor_4"/>
            <MiningField name="predictor_5"/>
        </MiningSchema>
        <Segmentation multipleModelMethod="modelChain" x-missingPredictionTreatment="returnMissing">
            <Segment id="1">
                <True/>
                <RegressionModel functionName="regression">
                    <MiningSchema>
                        <MiningField name="predictor_2"/>
                        <MiningField name="predictor_5"/>
                        <MiningField name="predictor_1"/>
                        <MiningField name="predictor_3"/>
                        <MiningField name="predictor_4"/>
                    </MiningSchema>
                    <Output>
                        <OutputField name="decisionFunction" optype="continuous" dataType="double" isFinalResult="false"/>
                    </Output>
                    <LocalTransformations>
                        <DerivedField name="robust_scaler(predictor_1)" optype="continuous" dataType="double">
                            <Apply function="/">
                                <Apply function="-">
                                    <FieldRef field="predictor_1"/>
                                    <Constant dataType="double">38.0</Constant>
                                </Apply>
                                <Constant dataType="double">36.0</Constant>
                            </Apply>
                        </DerivedField>
                        <DerivedField name="robust_scaler(predictor_3)" optype="continuous" dataType="double">
                            <Apply function="/">
                                <Apply function="-">
                                    <FieldRef field="predictor_3"/>
                                    <Constant dataType="double">29.5</Constant>
                                </Apply>
                                <Constant dataType="double">15.5</Constant>
                            </Apply>
                        </DerivedField>
                        <DerivedField name="robust_scaler(predictor_4)" optype="continuous" dataType="double">
                            <Apply function="/">
                                <Apply function="-">
                                    <FieldRef field="predictor_4"/>
                                    <Constant dataType="double">-2.0</Constant>
                                </Apply>
                                <Constant dataType="double">11.0</Constant>
                            </Apply>
                        </DerivedField>
                    </LocalTransformations>
                    <RegressionTable intercept="0.4485538242235567">
                        <NumericPredictor name="robust_scaler(predictor_1)" coefficient="0.09187667567720746"/>
                        <NumericPredictor name="predictor_2" coefficient="1.002293414783222337"/>
                        <NumericPredictor name="robust_scaler(predictor_3)" coefficient="-0.1790001566845147"/>
                        <NumericPredictor name="robust_scaler(predictor_4)" coefficient="-0.20065445270398309"/>
                        <NumericPredictor name="predictor_5" coefficient="-0.08789985419968031"/>
                    </RegressionTable>
                </RegressionModel>
            </Segment>
            <Segment id="2">
                <True/>
                <RegressionModel functionName="classification" normalizationMethod="softmax">
                    <MiningSchema>
                        <MiningField name="target_state" usageType="target"/>
                        <MiningField name="decisionFunction"/>
                    </MiningSchema>
                    <Output>
                        <OutputField name="probability(RED)" optype="continuous" dataType="double" feature="probability" value="RED"/>
                        <OutputField name="probability(GREEN)" optype="continuous" dataType="double" feature="probability" value="GREEN"/>
                    </Output>
                    <RegressionTable intercept="0.0" targetCategory="RED">
                        <NumericPredictor name="decisionFunction" coefficient="-1.0"/>
                    </RegressionTable>
                    <RegressionTable intercept="0.0" targetCategory="GREEN">
                        <NumericPredictor name="decisionFunction" coefficient="1.0"/>
                    </RegressionTable>
                </RegressionModel>
            </Segment>
        </Segmentation>
        <ModelVerification recordCount="1">
            <VerificationFields>
                <VerificationField field="predictor_1" column="data:predictor_1"/>
                <VerificationField field="predictor_2" column="data:predictor_2"/>
                <VerificationField field="predictor_3" column="data:predictor_3"/>
                <VerificationField field="predictor_4" column="data:predictor_4"/>
                <VerificationField field="predictor_5" column="data:predictor_5"/>
                <VerificationField field="probability(RED)" column="data:probability_RED" precision="1.0E-13" zeroThreshold="1.0E-13"/>
                <VerificationField field="probability(GREEN)" column="data:probability_GREEN" precision="1.0E-13" zeroThreshold="1.0E-13"/>
            </VerificationFields>
            <InlineTable>
                <row>
                    <data:predictor_1>595.0</data:predictor_1>
                    <data:predictor_2>0.0</data:predictor_2>
                    <data:predictor_3>201.0</data:predictor_3>
                    <data:predictor_4>-2.0</data:predictor_4>
                    <data:predictor_5>0.1</data:predictor_5>
                    <data:probability_RED>0.2555804919272633</data:probability_RED>
                    <data:probability_GREEN>0.9974195080727367</data:probability_GREEN>
                </row>
            </InlineTable>
        </ModelVerification>
    </MiningModel>
</PMML>

Can someone please help me to find where the problem is?

Kavin Raj · Accepted Answer · 2020-01-23T11:49:46.087

6

We have checked sample PMML file using NeuralNetworkModelEvaluator and we couldn’t reproduce the issue. Can you share your PMML file to check our side and provide you the solution sooner.

Also, we would suggest you to try the below code,

        string pmmlFilePath = “/project/model.pmml”;  

        //Create instance for PMML Document
        PMMLEvaluator PMMLEvaluator = new PMMLEvaluatorFactory().GetPMMLEvaluatorInstance(pmmlFilePath);

        //Gets the predicted result            
        PredictedResult predictedResult = PMMLEvaluator.GetResult(anonymousType, null);

Note: Syncfusion PMML library works by matching the schema defined in dmg.org and you can check Syncfusion help documentation for supported models and user guide.

For any further queries, please create a new incident (under your account) from our support website to provide solution quickly. Please find the support website link below. https://www.syncfusion.com/support/directtrac/incidents/newincident

Note : I work for Syncfusion.

edited Jan 23 '20 at 11:49

answered Jan 23 '20 at 11:41

Kavin Raj

159
1
8

Kavin, when I run it I get this warning: ```warning NU1701: Package 'Syncfusion.PMML.AspNet 17.4.0.44' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8' instead of the project target framework '.NETCoreApp,Version=v3.1'. This package may not be fully compatible with your project. ```. Could this be the issue? – Javide Jan 24 '20 at 04:08
I have added the model.pmml to my question. – Javide Jan 24 '20 at 04:23
I also tried a different algorithm (MLPRegressor) both in Java (https://github.com/javide/pmml-demo2) and in C# (https://github.com/javide/pmml-syncfusion) and, while it works as expected in Java, in C# I get: ```Unhandled exception. System.Exception: Unsupported activation function```. The model is based on a rectifier (relu) activation function. – Javide Jan 24 '20 at 04:24
@`Kavin Raj`, I tried loading the model in the way you suggested, but the call to predict throws exactly the same error. – Javide Jan 25 '20 at 21:44
@Javide In Your PMML , feature ( http://dmg.org/pmml/v4-4/Output.html#xsdType_RESULT-FEATURE ) attribute is missed in the Output field of first segmentation. So that it throws error as “unsupported feature type” in GetResult(). Note : Please make sure you haven’t missed any properties when creating the model and Feature attribute in the output field must be specified – Kavin Raj Jan 27 '20 at 09:40
@Javide As checked , you have created PMML with MiningModel but creating instance for Neural Network model which causes NullReferenceException issue. Please check the missing feature attribute in your model and create instance for MiningModel instead of Neural Network model. – Kavin Raj Jan 27 '20 at 09:41
You are correct, if I add `feature="predictedValue"` to the OutputField in the first Segment it works. This is how I fixed it: ``. However, this attribute is not defined as required by the PMML XSD, which defines a default instead. I don't know whether I can programmatically set this attribute via the Python library. Are you sure this is not a defect of the Syncfusion package? – Javide Jan 27 '20 at 11:47
I raised this issue with the maintainer of the sklearn2pmml library as well and he suggested a couple of workarounds: https://github.com/jpmml/sklearn2pmml/issues/205 – Javide Jan 27 '20 at 22:43
1

We also check to handle the case if feature attribute not specified". If it is the proper case, we will move the fix in next release by considering default value as "predictedValue". @Javide – Kavin Raj Jan 28 '20 at 06:21

How to make predictions with Syncfusion PMML from a Neural Network trained with sklearn MLPClassifier?

1 Answers1