Neural Network returning same output for every input

Question

I have written a simple Artificial Neural Network in Java as part of a project. When I begin training the data (Using a training set i gathered) The error count in each epoch quickly stabilizes (to around 30% accuracy) and then stops. When testing the ANN all outputs for any given input are EXACTLY the same.

I am trying to output a number between 0 and 1 (0 to classify a stock as a faller and 1 to classify a riser - 0.4-0.6 should indicate stability)

When adding the same training data into RapidMiner Studios a proper ANN with much greater (70+%) Accuracy is created, therefore I know that the dataset is fine. There must be some problem in the ANN logic.

Below is the code for running and adjusting the weights. Any and all help appreciated!

    public double[] Run(double[] inputs) {
    //INPUTS
    for (int i = 0; i < inputNeurons.length; i++) {
        inputNeurons[i] = inputs[i];
    }

    for (int i = 0; i < hiddenNeurons.length; i++) {
        hiddenNeurons[i] = 0;
    } //RESET THE HIDDEN NEURONS

    for (int e = 0; e < inputNeurons.length; e++) {
        for (int i = 0; i < hiddenNeurons.length; i++) {
            //Looping through each input neuron connected to each hidden neuron

            hiddenNeurons[i] += inputNeurons[e] * inputWeights[(e * hiddenNeurons.length) + i];
            //Summation (with the adding of neurons)  - Done by taking the sum of each (input * connection weight)
            //The more weighting a neuron has the more "important" it is in decision making
        }
    }

    for (int j = 0; j < hiddenNeurons.length; j++) {
        hiddenNeurons[j] = 1 / (1 + Math.exp(-hiddenNeurons[j]));
        //sigmoid function transforms the output into a real number between 0 and 1
    }

    //HIDDEN
    for (int i = 0; i < outputNeurons.length; i++) {
        outputNeurons[i] = 0;
    } //RESET THE OUTPUT NEURONS

    for (int e = 0; e < hiddenNeurons.length; e++) {
        for (int i = 0; i < outputNeurons.length; i++) {
            //Looping through each hidden neuron connected to each output neuron

            outputNeurons[i] += hiddenNeurons[e] * hiddenWeights[(e * outputNeurons.length) + i];
            //Summation (with the adding of neurons) as above
        }
    }

    for (int j = 0; j < outputNeurons.length; j++) {
        outputNeurons[j] = 1 / (1 + Math.exp(-outputNeurons[j])); //sigmoid function as above
    }

    double[] outputs = new double[outputNeurons.length];
    for (int j = 0; j < outputNeurons.length; j++) {
        //Places all output neuron values into an array
        outputs[j] = outputNeurons[j];
    }
    return outputs;
}

public double[] CalculateErrors(double[] targetValues) {
    //Compares the given values to the actual values
    for (int k = 0; k < outputErrors.length; k++) {
        outputErrors[k] = targetValues[k] - outputNeurons[k];
    }
    return outputErrors;
}

    public void tuneWeights() //Back Propagation
{
    // Start from the end - From output to hidden
    for (int p = 0; p < this.hiddenNeurons.length; p++)     //For all Hidden Neurons
    {
        for (int q = 0; q < this.outputNeurons.length; q++)  //For all Output Neurons
        {
            double delta = this.outputNeurons[q] * (1 - this.outputNeurons[q]) * this.outputErrors[q];
            //DELTA is the error for the output neuron q
            this.hiddenWeights[(p * outputNeurons.length) + q] += this.learningRate * delta * this.hiddenNeurons[p];
            /*Adjust the particular weight relative to the error
             *If the error is large, the weighting will be decreased
             *If the error is small, the weighting will be increased
             */
        }
    }

    // From hidden to inps -- Same as above
    for (int i = 0; i < this.inputNeurons.length; i++)       //For all Input Neurons
    {
        for (int j = 0; j < this.hiddenNeurons.length; j++)  //For all Hidden Neurons
        {
            double delta = this.hiddenNeurons[j] * (1 - this.hiddenNeurons[j]);
            double x = 0;       //We do not have output errors here so we must use extra data from Output Neurons
            for (int k = 0; k < this.outputNeurons.length; k++) {
                double outputDelta = this.outputNeurons[k] * (1 - this.outputNeurons[k]) * this.outputErrors[k];
                //We calculate the output delta again
                x = x + outputDelta * this.hiddenWeights[(j * outputNeurons.length) + k];
                //We then calculate the error based on the hidden weights (x is used to add the error values of all weights)
                delta = delta * x;
            }
            this.inputWeights[(i * hiddenNeurons.length) + j] += this.learningRate * delta * this.inputNeurons[i];
            //Adjust weight like above
        }
    }
}

Answer 1

After long coversation I think that you may find an answer to your question in the following points :

1. Bias is really important. Actually - one of the most popular SO questions about neural network is about bias :) : Role of Bias in Neural Networks
2. You should babysit your learning process. It's good to keep track of your test on accuracy and validation set and to use appropriate learning rate during training. What I advise you is to use simpler dataset when you know that it is easy to find true solution (for example - a triangle or square - use 4 - 5 hidden units then). I also advise you to use the following playgroud :

http://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=circle&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=4,2&seed=0.36368&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification