1

Is it possible to combine the softmax and linear activation functions in output layer in keras interface for R ? E.g. first 5 neurons will be softmax, because they should predict classes which are mutually exclusive, and the 6th and 7th neurons will be linear, because they should predict two continuous outcomes. Of course, the loss function for first 5 will be cross entropy and for the rest 2 will be MSE. I am asking this in context of autoencoders.

pikachu
  • 690
  • 1
  • 6
  • 17

1 Answers1

1

I would create a model with two outputs (as in keras_model(inputs = input, outputs = c(output1, output2)). output1 is a vector of length 5 that has had a softmax applied to it. output2 is a vector with the other two neurons. You can then compile the model with two separate loss functions, one for each output.

If you do not have experience with Keras' functional API, this page seems to provide a decent overview.

Note: I only know python, not R, so apologies for any syntax errors. Hopefully my code gets the point across.

The Guy with The Hat
  • 10,836
  • 8
  • 57
  • 75
  • Thanks a lot. Do you think this would work also if I need to use the softmax for more than one variable? I mean like first 5 variables will have softmax together, next 3 variables will have again softmax and next 2 variables would be linear or whatever. I would want the first 5 to point by softmax to one of the variable and second set of 3 softmax to point to one of the 3. Is there a way how to write this to keras? – pikachu Oct 22 '19 at 06:49
  • 1
    @pikachu If I'm understanding you correctly, then yeah, that should be easy! Just make a third output layer, apply softmax to that, and add it to the set of outputs when you create the model. – The Guy with The Hat Oct 22 '19 at 13:17