Multi-class regression in nolearn?

Question

I'm trying to build a Neural Network using nolearn that can do regression on multiple classes.

For example:

net = NeuralNet(layers=layers_s,
             input_shape=(None, 2048),
             l1_num_units=8000,
             l2_num_units=4000,
             l3_num_units=2000,
             l4_num_units=1000,
             d1_p = 0.25,
             d2_p = 0.25,
             d3_p = 0.25,
             d4_p = 0.1,
             output_num_units=noutput,
             output_nonlinearity=None,
             regression=True,
             objective_loss_function=lasagne.objectives.squared_error,
             update_learning_rate=theano.shared(float32(0.1)),
             update_momentum=theano.shared(float32(0.8)),
             on_epoch_finished=[
                    AdjustVariable('update_learning_rate', start=0.1, stop=0.001),
                    AdjustVariable('update_momentum', start=0.8, stop=0.999),
                    EarlyStopping(patience=200),
                                    ],
             verbose=1,
             max_epochs=1000)

noutput is the number of classes for which I want to do regression, if I set this to 1 everything works. When I use 26 (the number of classes here) as output_num_unit I get a Theano dimension error. (dimension mismatch in args to gemm (128,1000)x(1000,26)->(128,1))

The Y labels are continues variables, corresponding to a class. I tried to reshape the Y labels to (rows,classes) but this means I have to give a lot of the Y labels a value of 0 (because the value for that class is unknown). Is there any way to do this without setting some y_labels to 0?

y_train has the following shape: (1082,). All these are continues variables corresponding to different (26) classes. — B100, Aug 30 '15 at 10:36
Are you sure they are continuous, or do they take discrete values (0, 1, ..., 25)? — P. Camilleri, Sep 01 '15 at 07:33
They are continuous so what I meant was; [Class, Value]: [A, 7.9], [B, 5.2] [A, 4.2] etc. So there are 26 different classes but 1082 continues values on which I want to do (multi class) regression. — B100, Sep 02 '15 at 09:02
So you want to ignore the class? I don't understand, could you include an example of expected output? And also the first few lines of `y`? — P. Camilleri, Sep 02 '15 at 10:31
Correct, I have different target classes (data sources) with a lot of different continues values, the number of classes should correspond to the num_output units, every output unit performs regression on one target. What I'm trying to do is very similair to joint DNN in this Article;(http://www.cs.toronto.edu/~gdahl/papers/deepQSARJChemInfModel2015.pdf). So the first few lines of y look like this; 7.9, 5.2, 4.2 — B100, Sep 02 '15 at 11:37

score 0 · Answer 1 · answered Aug 30 '15 at 11:10

If you want to do multiclass (or multilabel) regression with 26 classes, your output must not have shape (1082,), but (1082, 26). In order to preprocess your output, you can use sklearn.preprocessing.label_binarize which will transform your 1D output to 2D output.

Also, your output non linearity should be a softmax function, so that the rows of your output sum to 1.

Multi-class regression in nolearn?

1 Answers1