-1

I was trying to do a custom loss function but I get the error. This is supposed to be cyclegan for text This is the loss function:

def sloss(ytrue, ypred):
  nump = 0
  print(nump)
  for i in range(len(ypred[0])):
    if int(round(ypred[0][i])) != int(ytrue[0][i]):
      nump+=1
  return(nump)

Model:

models = tf.keras.models.Sequential()
models.add(Dense(100, activation='relu'))
models.add(Dense(100, activation='linear'))
models.add(Dense(100, activation='sigmoid'))

models.compile(loss=sloss, optimizer='adam')

models.fit(sx(1,100 np arr), randes(1,100 np arr), epochs=1)
Spock 360
  • 9
  • This function would have to be implemented using tensorflow ops, and also this is just accuracy which is not differentiable (cannot be used as a loss). – Dr. Snoopy Sep 25 '22 at 01:46
  • Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. – user11717481 Sep 25 '22 at 03:28

1 Answers1

0

The loss function needs to be calculated based on ypred, otherwise the gradient cannot be calculated. Your loss function is just a sum of multiple 1's:

nump += 1
nump += 1
...
nump += 1

No ypred here. That's why the gradient cannot be calculated and the error message. Another issue is that your loss function is constant on intervals, that is, the derivative is zero everywhere. The optimizer cannot work with a zero gradient.

AndrzejO
  • 1,502
  • 1
  • 9
  • 12
  • No, that is not the reason, have you even considered that gradient is not mathematically defined since the function is not continuous? – Dr. Snoopy Sep 25 '22 at 03:04
  • The `sign` function is not continous, still the gradient is defined almost everywhere, except not in `0`. Since a neural network consists of linear functions and very nice activation functions, the accuracy function will have gradients almost everywhere (meaning everywhere except a set of measure zero, where the function is not continous). – AndrzejO Sep 25 '22 at 04:16
  • The sign function was an example of a function which is not continous, but has gradients. In ML courses they simplify math a lot. The real issue is that wherever the gradient exist, it is zero, so it doesn't help. See how accuracy is calculated - it's a finite sum of constant functions. Which means you can divide the domain of accuracy into a finite number of sets, and on each set accuracy is constant, with a derivative 0. The only places where the derivative does not exist is the borders of these sets, where we have the 'steps' – AndrzejO Sep 25 '22 at 12:57
  • All those details are irrelevant, because when gradients cannot be computed, tensorflow will error with the exact message in the question title. – Dr. Snoopy Sep 25 '22 at 13:27
  • Of course it's irreleveant. The loss function needs not only to be computed, it needs to be a tensor dependent on the weights. Which is not the case here, `nump` is just a simple number – AndrzejO Sep 25 '22 at 13:55