5

Maybe a very dumb question but I can't find an example how to use categorical_hinge in Keras. I do classification and my target is shape(,1) with values [-1,0,1] so I have 3 categories. Using the functional API I have set up my output layer like this:

output = Dense(1, name='output', activation='tanh', kernel_initializer='lecun_normal')(output1)

Then I apply:

model.compile(optimizer=adam, loss={'output': 'categorical_hinge'}, metrics=['accuracy'])

The result is that the model is converging but accuracy goes towards to 0. What do I do wrong?

Manngo
  • 829
  • 7
  • 24

2 Answers2

2

While [-1, 0, 1] is a valid target range for your tanh activation function, experience tells that Keras models don't work well with classification in a binary output. Consider using three one-hot vectors with a softmax classifier instead. If I interpret this bug report correctly, categorical hinge is built to work with one-hot vectors anyway.

So: Convert your labels to one-hots and change your output to something along the lines of:

output = Dense(3, name='output', activation='softmax', kernel_initializer='lecun_normal')(output1)
Huang_d
  • 144
  • 8
  • Answer is correct with the only mistake that [-1, 0, 1] is not a binary output. All the rest is correct. – Manngo Jan 18 '19 at 22:01
  • Do you need to add activation='softmax' in Keras? Mathematically, hinge loss applies on distance, which does not have to be between (0 and 1). While crossentropy loss applies on probability, so it requires softamx before using the loss function. – Diansheng Aug 27 '19 at 10:54
0

Use: 

model.compile(optimizer=adam, loss="categorical_hinge", metrics=['accuracy'])
Yunnosch
  • 26,130
  • 9
  • 42
  • 54
Neeraj Natu
  • 53
  • 1
  • 7