Lasagne - error

Question

i am trying to learn neural nets / lasagne by using the code in Robert Layton's Learning Data Mining with Python. I think i am following the code to the letter, but i get the following error message. any hints or intuition what i am doing wrong much appreciated;

Traceback (most recent call last):

  File "<ipython-input-78-3ff2950373de>", line 3, in <module>
    updates=lasagne.updates.sgd(loss,all_params,learning_rate=0.01)

  File "C:\Users\WouterD\Anaconda\lib\site-packages\lasagne\updates.py", line 134, in sgd
    grads = get_or_compute_grads(loss_or_grads, params)

  File "C:\Users\WouterD\Anaconda\lib\site-packages\lasagne\updates.py", line 110, in get_or_compute_grads
    return theano.grad(loss_or_grads, params)

  File "C:\Users\WouterD\Anaconda\lib\site-packages\theano-0.7.0-py2.7.egg\theano\gradient.py", line 551, in grad
    handle_disconnected(elem)

  File "C:\Users\WouterD\Anaconda\lib\site-packages\theano-0.7.0-py2.7.egg\theano\gradient.py", line 538, in handle_disconnected
    raise DisconnectedInputError(message)

DisconnectedInputError: grad method was asked to compute the gradient with respect to a variable that is not part of the computational graph of the cost, or is used only by a non-differentiable operator: W
Backtrace when the node is created:
  File "C:\Users\WouterD\Anaconda\lib\site-packages\theano-0.7.0-py2.7.egg\theano\compile\sharedvalue.py", line 248, in shared
    utils.add_tag_trace(var)

code below:

from sklearn.datasets import load_iris
iris=load_iris()
X=iris.data.astype(np.float32)
y_true=iris.data.astype(np.int32)

from sklearn.cross_validation import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,y_true,random_state=14)

import lasagne
input_layer=lasagne.layers.InputLayer(shape=(10,X.shape[1]))

hidden_layer=lasagne.layers.DenseLayer(input_layer,num_units=12,nonlinearity=lasagne.nonlinearities.sigmoid)

output_layer=lasagne.layers.DenseLayer(hidden_layer,num_units=3,nonlinearity=lasagne.nonlinearities.softmax)

import theano.tensor as T
net_input=T.matrix('net_input')
net_output=output_layer.get_output_for(net_input)
true_output=T.ivector("true_output")

loss=T.mean(T.nnet.categorical_crossentropy(net_output,true_output))
all_params=lasagne.layers.get_all_params(output_layer)
updates=lasagne.updates.sgd(loss,all_params,learning_rate=0.01)

Answer 1

The problem is that you're not computing the loss with respect to the real input variable. net_input=T.matrix('net_input') is your own symbolic input to the network but Lasagne had already created one for you when you created the InputLayer. You also don't need to get the output with respect to a particular input, just get the output with respect to the input layer.

So, replace the two lines

net_input=T.matrix('net_input')
net_output=output_layer.get_output_for(net_input)

with the single line

net_output=lasagne.layers.get_output(output_layer)

Anticipating the next issue you'll have, you can get the input variable Lasagne created for you via input_layer.input_var so you can compile your training function like this:

import theano
f = theano.function([input_layer.input_var, true_output], outputs=loss, updates=updates)

Answer 2

input_layer=lasagne.layers.InputLayer(shape=(10,X.shape[1]),input_var=input)

while the input is the tensor you defined before