0

I am stuck with the nce_loss activation function in the word2vec model. I want to figure out what activation function it uses among all these listed here:

These include smooth nonlinearities (sigmoid, tanh, elu, softplus, and softsign), continuous but not everywhere differentiable functions (relu, relu6, crelu and relu_x), and random regularization (dropout).

I have searched for it in this function and somewhere else but failed to get any ideas. I suppose it is the relu* series. What can I try next?

halfer
  • 19,824
  • 17
  • 99
  • 186
Lerner Zhang
  • 6,184
  • 2
  • 49
  • 66

1 Answers1

1

None of those. It uses CrossEntropy.

fabrizioM
  • 46,639
  • 15
  • 102
  • 119