Pybrain regression is not converging to reasonable values

Question

I am a beginner to PyBrain (and fairly new to ANN's), so to familiarize with using PyBrain I have tried to train on a sin function. My outputs have been making little sense --- for each data point I get an output of 0, -0, or some fixed real number ( in the example below I get a real number). This indicates to me that I am not training correctly.

I have been trying to train by creating a dataset with one input and one output via SupervisedDataSet(1,1) and training using BackPropTrainer(). I have read the PyBrain documentation extensively and it is rather sparse and the examples are not great. My input is an integer in the range (0,1000) and my output/targets are simply the sin of that integer multiplied by some constant (so it evaluates between zero and 2pi). Code is below:

def add_samples():

    ds = SupervisedDataSet(1,1)

    for j in  range(0,1000):
            ds.addSample(j,math.sin((j*math.pi)/500))

    print ds
    return ds

def FeedForward():
    n= FeedForwardNetwork()

    #construct input, hiddent, and output Layers
    inLayer=LinearLayer(1)
    hiddenLayer = SigmoidLayer(3)
    outLayer=LinearLayer(1)


    # add layers to the network
    n.addInputModule(inLayer)
    n.addModule(hiddenLayer)
    n.addOutputModule(outLayer)

    # make the connections between the layers
    in_to_hidden = FullConnection(inLayer, hiddenLayer)
    hidden_to_out = FullConnection(hiddenLayer, outLayer)

    # explicitly adding the connections to the network
    n.addConnection(in_to_hidden)
    n.addConnection(hidden_to_out)

    # some internal organization
    n.sortModules()
    print n
    return n
def backprop():
    n=FeedForward()
    ds=add_samples()

    trainer = BackpropTrainer(n, ds)
    trainer.trainOnDataset(ds,100)
    trainer.testOnData(verbose=True)



def main():
    backprop()

if __name__ == '__main__':
    main()

My outputs look like this:

  error:  0.00084029
  out:     [0.016 ]
  correct: [-0.019]
  error:  0.00060250
  out:     [0.016 ]
  correct: [-0.013]
  error:  0.00040415
  out:     [0.016 ]
  correct: [-0.006]
  error:  0.00024526

All 1000 outputs evaluated to 0.016. Does anyone have any suggestions or can point me to a good example ? I have been spinning in circle for awhile. I'd imagine I am missing something trivial or there is some fundamental concept to ANN's or machine learning in general I am missing. Please let me know if it would help if I provide more information. Thanks!

Pybrain regression is not converging to reasonable values

0 Answers0