Encog: BasicNetwork: Online learning without preconstructed dataset

Question

I am trying to use the encog library as a function approximator for a reinforcement learning problem. To be more precise, I am trying to get a multi layer perceptron (BasicNetwork) up and running. Since my agent will somehow explore the world based on whatever RL-algorithm I chose I cannot prebuild any BasicNeuralDataSet as shown in the XOR example. Probably, I have to use the pause() and resume() functions but since I cannot find any documentation or examples on these I am somewhat lost in how to use these features (if they even work in my version. I'm not quite sure after reading the answer to the question in the second link).

I am using Java and the encog-core-2.5.3 jar. My current approach looks like this:

BasicNetwork network = new BasicNetwork();
network.addLayer(new BasicLayer(null, true,2));
network.addLayer(new BasicLayer(new ActivationTANH(), true,4));
network.addLayer(new BasicLayer(new ActivationTANH(), true,1));
network.getStructure().finalizeStructure();
network.reset();

TrainingContinuation cont = null;
double error = 0;

do {
    int rnd = random.nextInt(trainInputs.length);
    NeuralDataSet trainingSet = new BasicNeuralDataSet(
        new double[][] { trainInputs[rnd] },
        new double[][] { trainOutputs[rnd] });

    Backpropagation train = new Backpropagation(network, trainingSet);

    // train the neural network
    if (cont != null) {
        train.resume(cont);
    }

    train.iteration();
    cont = train.pause();

    error = train.getError();
} while (error > 0.01);

This is obviously a minimal example where I am just drawing random datapoints from a toy sample (XOR). What happens is that the MLP does not converge. Logging is showing me completely random errors so I assume that somewhat the trainer is being reset and that my pause/resume approach is not correctly implemented.

P.S.: Since I am not bound to Encoq but can use any framework there is I also appreciate sample code that fulfills my requirements. So far I tried Weka and Neuroph but both seem to lack real online learning where one can just trigger the training whenever a new sample is available (It has to be possible to classify samples during any time as well)

JeffHeaton · Accepted Answer · 2014-01-07T19:05:59.733

8

Sorry about the slow response. Basically, it sounds like you are asking for online training. That is you just present a single case and the neural network weights are immediately updated. That way it would not be necessary to create an entire training set, you just train as needed. Unfortunately, Encog does not have good support of this. It has become a frequently asked question, and I plan to add it in the next release.

For now, about the only way you could do it is to create a training set with a single item and then train for a single iteration.

EDIT Online training has been added as of Encog 3.2. See this FAQ for more info.

http://www.heatonresearch.com/faq/5/3

edited Jan 07 '14 at 19:05

answered Apr 20 '13 at 20:01

JeffHeaton

3,250
1
22
33

2

Yes it would be nice to add online support to Encog, I need that as well for something I am working on. – Miley May 24 '13 at 16:27
You do not seem to be the only one. I am planning it in the next release. – JeffHeaton May 24 '13 at 16:32
1

Hi Jeff, thanks for the library! Do you have an idea of when the next release might be available? – nicolaspanel Oct 09 '13 at 19:46
See above edit, but this will be added in Encog 3.2 which should be released by Feb 2014. The changes for online training are already in GitHub. – JeffHeaton Jan 07 '14 at 19:06
2

This hasn't completly non desciption of usage, wtf? URL are not available, examples are only for XOR problem.... – murt Jan 12 '16 at 00:07
@JeffHeaton, where can the FAQ be found? – Matsemann Nov 09 '16 at 20:43

Encog: BasicNetwork: Online learning without preconstructed dataset

1 Answers1

Linked