How to correctly export Weight and Bias value of Backpropagation neural network into another programming language (Java)

Question

I created backpropagation Neural Network using Matlab. I tried to implement XOR gate using Matlab, then getting its weight and bias to create neural network in java. Network consist of 2 input neuron, 2 hidden layer each using 2 neuron and 1 output neuron. After train network, i got following weight and bias :

clear;
clc;
i = [0 0 1 1; 0 1 0 1];
o = [0 1 1 0];
net = newff(i,o,{2,2},{'tansig','logsig','purelin'});
net.IW{1,1} = [
    -5.5187   -5.4490;
     3.7332    2.7697
];
net.LW{2,1} = [
   -2.8093   -3.0692;
   -1.6685    6.7527
];
net.LW{3,2} = [
    -4.9318   -0.9651
];
net.b{1,1} = [
    2.1369;
    2.6529
];
net.b{2,1} = [
    -0.2274;
    -4.9512
];
net.b{3,1} = [
    1.4848
];

input  = net.IW{1,1};
layer  = net.LW{2,1};
output = net.LW{3,2};

biasinput = net.b{1,1};
biaslayer = net.b{2,1};
biasoutput= net.b{3,1};


a = sim(net,i);
a;

I simulate it using 1 and 1 as input got following result :

>> f = [1;1]

f =

     1
     1

>> sim(net,f)

ans =

   -0.1639

Then I tried to make simple java code to count this neural network. My code :

public class Xor {

    //Value of neuron
    static double[] neuroninput    = new double[2];
    static double[] neuronhidden1  = new double[2];
    static double[] neuronhidden2  = new double[2];
    static double[] neuronoutput   = new double[2];

    //Weight variable init
    //For first hidden layer
    static double[] weighthidden11 = new double[2];
    static double[] weighthidden12 = new double[2];

    //for second hidden layer
    static double[] weighthidden21 = new double[2];
    static double[] weighthidden22 = new double[2];

    //for output layer
    static double[] weightoutput   = new double[2];
    //End of weight variable init

    //Bias value input
    static double[] biashidden1    = new double[2];
    static double[] biashidden2    = new double[2];
    static double[] biasoutput     = new double[1];

    public static void main(String[] args) {
        neuroninput[0] = 1;
        neuroninput[1] = 1;

        weighthidden11[0] = -5.5187;
        weighthidden11[1] = -5.4490;
        weighthidden12[0] =  3.7332;
        weighthidden12[1] =  2.7697;

        weighthidden21[0] = -2.8093;
        weighthidden21[1] = -3.0692;
        weighthidden22[0] = -1.6685;
        weighthidden22[1] =  6.7527;

        weightoutput[0]    = -4.9318;
        weightoutput[1]    = -0.9651;

        biashidden1[0] = 2.1369;
        biashidden1[1] = 2.6529;

        biashidden2[0] = -0.2274;
        biashidden2[1] = -4.9512;

        biasoutput[0]  = 1.4848;

        //Counting each neuron (Feed forward)
        neuronhidden1[0] = sigma(neuroninput,weighthidden11,biashidden1[0]);
        neuronhidden1[0] = tansig(neuronhidden1[0]);

        neuronhidden1[1] = sigma(neuroninput,weighthidden12,biashidden1[1]);
        neuronhidden1[1] = tansig(neuronhidden1[1]);


        neuronhidden2[0] = sigma(neuronhidden1,weighthidden21,biashidden2[0]);
        neuronhidden2[0] = logsig(neuronhidden2[0]);

        neuronhidden2[1] = sigma(neuronhidden1,weighthidden22,biashidden2[1]);
        neuronhidden2[1] = logsig(neuronhidden2[1]);

        neuronoutput[0] = sigma(neuronhidden2,weightoutput,biasoutput[0]);
        neuronoutput[0] = purelin(neuronoutput[0]);
        System.out.println(neuronoutput[0]);
    }

    static double tansig(double x) {
        double value = 0;
        value = (Math.exp(x) - Math.exp(-x)) / (Math.exp(x) + Math.exp(-x));
        return value;
    }

    static double logsig(double x) {
        double value = 0;
        value = 1 / (1+Math.exp(-x));
        return value;
    }

    static double purelin(double x) {
        double value = x;
        return value;
    }

    static double sigma(double[] val, double[] weight, double hidden) {
        double value = 0;
        for (int i = 0; i < val.length; i++) {
            value += (val[i] * weight[i]);
            //System.out.println(val[i]);
        }
        value += hidden;
        return value;
    }
}

But it got result as following :

-1.3278721528152158

My question, is there any error or my mistake in exporting weight and bias value from matlab to java? Maybe I made mistake in my java program? Thank you verymuch..

I'm not going to debug your code for you. I'd suggest doing that yourself though: step through with a debugger and identify the first point at which the results start to look different. — mikera, Jan 09 '13 at 06:33
I have debugged my java code and counted manual in paper. But it give me the same result. I have very low understanding about Matlab. my problem is how exactly bringing Matlab's weight and bias to another programming language — justmyfreak, Jan 09 '13 at 06:39

score 2 · Accepted Answer · answered Apr 22 '13 at 16:36

2

I think the problem is the normalization: http://www.mathworks.com/matlabcentral/answers/14590

If you work with 0,1 inputs, you have to use the f(x)=2*x-1 normalization function, which transforms the values to the [-1; 1] interval, then g(x)=(x+1)/2 to transform back the output to [0; 1]. Pseudocode:

g( java_net( f(x), f(y) ) ) = matlab_net(x, y)

I tried this with an other network and worked for me.

answered Apr 22 '13 at 16:36

sebesbal

71
4

How if input and output is not 0,1? Is it same normalization function? – justmyfreak May 04 '13 at 15:01
If you work with [a;b] interval, you need two linear functions which satisfy the following conditions: f(a)=-1, f(b)=1, g(-1)=a, g(1)=b. The solution: f(x)=2*(x-a)/(b-a)-1, g(x)=(x+1)*(b-a)/2 + a – sebesbal Nov 04 '16 at 10:48

KlausCPH · Answer 2 · 2013-01-10T17:42:11.187

0

Your problem is most surely related to your JAVA version of the Matlab sim() command.

It is a complex Matlab command with many settings affecting the architecture of the network to be simulated. To make debugging easier, try and implement the sim() command yourself in Matlab. Possibly reduce the number of layers until you have a match in Matlab between sim()-builtin and your own sim version. When that is working convert to JAVA.

EDIT:

The Reason for re-implementing the sim() function in Matlab, is that if you can't implement it here, you won't be able to properly implement it in JAVA either. Feed forward networks are quiet easy to implement using Matlab vector notation.

edited Jan 10 '13 at 17:42

answered Jan 09 '13 at 20:24

KlausCPH

1,816
11
14

It means I need to create my own sim() function to check whether my input is recognized by network or not? – justmyfreak Jan 10 '13 at 12:24
is sim() I have to recreated just feed-forward? – justmyfreak Jan 13 '13 at 01:04
Yes, you only need to recreate the calculation of the classification based on the weights and network topology you already have – KlausCPH Jan 13 '13 at 14:16

How to correctly export Weight and Bias value of Backpropagation neural network into another programming language (Java)

2 Answers2