PHP Fann gives same result despite different inputs

Question

I am trying out the FANN PHP module and I was able to successfully run the example here http://php.net/manual/en/fann.examples-1.php

I modified it to be able to handle 5 inputs with an arbitrary function for output. I generated 1000 training data and run the training for the neural network. However, upon testing the output has duplicate results for different inputs.

This is a fragment of the training data. The function is $x = round($a + $b * $c / $d - $e, 2). so 35 + 33 * 31 / 25 - 48 = 27.92

1000 5 1
35 33 31 25 48
27.92
74 3 1 26 94
-19.88
7 62 86 48 71
47.08
31 73 68 94 95
-11.19
100 87 44 75 43
108.04
72 25 62 39 57
54.74
...

Here is my training code. I used FANN_LINEAR because the other activation functions have outputs of 0, 1, or -1. I read that FANN_LINEAR is unbounded. So this should be applicable, right?

<?php
$num_input = 5;
$num_output = 1;
$num_layers = 6;
$num_neurons_hidden = 4;
$desired_error = 0.0001;
$max_epochs = 500000;
$epochs_between_reports = 1000;

$ann = fann_create_standard($num_layers, 5, 5, 5, 5, 5, 1);

if ($ann) {
    fann_set_activation_function_hidden($ann, FANN_LINEAR);
    fann_set_activation_function_output($ann, FANN_LINEAR);

    $filename = dirname(__FILE__) . "/xor.data";
    if (fann_train_on_file($ann, $filename, $max_epochs, $epochs_between_reports, $desired_error))
        fann_save($ann, dirname(__FILE__) . "/xor_float.net");

    fann_destroy($ann);
}

Here is my testing code

<?php
$train_file = (dirname(__FILE__) . "/xor_float.net");
if (!is_file($train_file))
    die("The file xor_float.net has not been created! Please run simple_train.php to generate it");

$ann = fann_create_from_file($train_file);
if (!$ann)
    die("ANN could not be created");

$a = mt_rand(1, 100);
$b = mt_rand(1, 100);
$c = mt_rand(1, 100);
$d = mt_rand(1, 100);
$e = mt_rand(1, 100);

echo "Expecting $a $b $c $d $e => ".round($a + $b * $c / $d - $e, 2)."\n\n";

$input = array($a, $b, $c, $d, $e);
$calc_out = fann_run($ann, $input);
echo "Result: ".print_r($calc_out, true);
fann_destroy($ann);

This is where it gets weird. I tried running this code multiple times, but the result is the same

fann$ php test2.php 
Expecting 94 67 95 40 85 => 168.13

Result: Array
(
    [0] => 89.329223632812
)
fann$ php test2.php 
Expecting 53 43 56 64 64 => 26.63

Result: Array
(
    [0] => 89.329223632812
)
fann$ php test2.php 
Expecting 18 85 57 94 30 => 39.54

Result: Array
(
    [0] => 89.329223632812
)

Can you give me some pointers to achieve my goal, that is, to approximate an arbitrary function using FANN. Do I have to increase my training data? Increase layers, or nodes per layer? Do I use another activation function.

score 2 · Answer 1 · answered May 31 '16 at 14:59

At a first glance, it seems like your network got stuck at certain weight configuration. That means that the weights, which are normally initialized to small values around 0, start changing their values to better fit the desired outputs. Since these changes are small, it is possible that, after a number of training epochs, they either don't «move enough» to find a nice combination of values, or they remain at a local minimum. There are a number of possible reasons (or just things to think about) for that:

The input & output values that the network and the training algorithm have to process are relatively big (~range [-20,100]), compared with the initial values of the weights (~range [-1,1]). The easiest solution here is to normalize the inputs & outputs to be A) small, B) centered around 0.
The activation functionFANN_LINEAR has no upper and lower limits. That may be good, but it can also lead to divergence. One option is to use FANN_SIGMOID_SYMMETRIC, which has limits, but also a transition zone where it is nearly linear. It works with normalized inputs and outputs.
Sometimes the values of momentum and learning rate can lead to a bad learning. To make some diagnostics, you may want to plot a learning error curve of the error against the training epoch. Of course it is a bit more laborious, since you need to train each epoch separately (fann_train_epoch instead of fann_train_on_file).
Lastly. I have no experience with big networks, but I could imagine that a single hidden layer could learn a problem like this. Why not giving it a try? It is already difficult enough to fix the number of 1 hidden layer, to just put more and more parameters in the equation ;)

Well, I hope that helps :) Have fun with your nets!

PHP Fann gives same result despite different inputs

1 Answers1