PyBrain Predicting Error

Question

I am trying to predict the sale price of a given set of goods. I am using RecurrentNetwork and BackpropTrainer in pybrain. Here is my code,

def nnet(train, target, valid):

    ds = SupervisedDataSet(52-len(NU)+5, 1)

    for i in range(len(train)):
        ds.appendLinked(train[i], target[i])

    n = RecurrentNetwork()

    n.addInputModule(LinearLayer(52-len(NU)+5, name='in'))
    n.addModule(SigmoidLayer(3, name='hidden'))
    n.addOutputModule(LinearLayer(1, name='out'))

    n.addConnection(FullConnection(n['in'], n['hidden'], name='c1'))
    n.addConnection(FullConnection(n['hidden'], n['out'], name='c2'))
    n.addRecurrentConnection(FullConnection(n['hidden'], n['hidden'], name='c3'))

    n.sortModules()

    t = BackpropTrainer(n,learningrate=0.001,verbose=True)
    t.trainOnDataset(ds, 20)

    prediction = np.zeros((11573, 1), dtype = int)
    for i in range(11573):
        prediction[i] = n.activate(valid[i])

    return prediction

Here train and target, which are numpy arrays, are used to train the model and 52-len(NU)+5 is the number of attributes(features).For each item in valid we have to predict the sale price. The problem is, for every item in valid, I get the same sale price except for the first one. What have I done wrong? Thanks in advance.

Array dimensions are as follows,

train - 401125, 52-len(NU)+5

target - 401125, 1

valid - 11573, 52-len(NU)+5

Taking into account that nobody has answered yet, I suggest asking here: https://github.com/pybrain/pybrain/issues — pablofiumara, Mar 27 '14 at 03:54
I don't know PyBrain but for these kind of task I suggest you to take a look at http://scikit-learn.org/stable/. — blueSurfer, Apr 04 '14 at 15:57
Maybe this is unrelated, but have you tried training more than 20 epochs (your learning rate seems a bit low)? maybe using the `trainUntilConvergence` method? — agmangas, Apr 09 '14 at 16:10

score 3 · Answer 1 · answered May 20 '14 at 21:07

I'm not sure about the specific implementation details of PyBrain, but I see two possibilities.

1) Backpropagation does not work with linear activation functions. Depending on the implementation details of PyBrain, changing both instances of "LinearLayer" to "SigmoidLayer" could fix this issue.

2) With recurrent neural networks, you have to use backpropagation through time (an algorithm specifically adapted for RNNs) instead of normal backpropagation. Depending on the implementation details of PyBrain, there might be a separate class for this variant. It's worth checking out.

PyBrain Predicting Error

1 Answers1