0

I have tried to create a custom loss function using a numba jit-complied function as outlined here for a regression algorithm. It seems to work as a metric, but I have a strange error when used as a loss. I have a toy function that replicates the problem here:

@njit
def test_del(y_true, y_pred):
    cols = y_true.shape[1]
    out = 0
    for i in range(y_true.shape[1]):
        true_dam = np.abs(y_true[:, i]).max()  #toy
        pred_dam = np.abs(y_pred[:, i]).max()  #toy
        out += np.mean(np.abs(np.log(pred_dam / true_dam))**2)
    return out/cols

(yes I know this toy problem can be optimized to be more vectorized, but it follows the structure of my actual functions that can't be, so I'm leaving it)

Then I have a loss/metric function:

def del_loss(y_true, y_pred):
    return tf.numpy_function(test_del, [y_true, y_pred], tf.float64) +\
           K.cast(tf.keras.losses.mean_squared_error(y_true, y_pred), tf.float64)

Now, if I compile a model with del_loss as a metric (as long as I cast it to float64, which is weird but whatever), it works fine. But if I use it as a loss I get this strange string of errors:

Traceback (most recent call last):

  #removed my chain of objects resulting in a `model.compile(loss = del_loss)` call

  File "C:\ProgramData\Anaconda3\envs\MLEnv\lib\site-packages\keras\backend\tensorflow_backend.py", line 75, in symbolic_fn_wrapper
    return func(*args, **kwargs)

  File "C:\ProgramData\Anaconda3\envs\MLEnv\lib\site-packages\keras\engine\training.py", line 229, in compile
    self.total_loss = self._prepare_total_loss(masks)

  File "C:\ProgramData\Anaconda3\envs\MLEnv\lib\site-packages\keras\engine\training.py", line 692, in _prepare_total_loss
    y_true, y_pred, sample_weight=sample_weight)

  File "C:\ProgramData\Anaconda3\envs\MLEnv\lib\site-packages\keras\losses.py", line 73, in __call__
    losses, sample_weight, reduction=self.reduction)

  File "C:\ProgramData\Anaconda3\envs\MLEnv\lib\site-packages\keras\utils\losses_utils.py", line 166, in compute_weighted_loss
    losses, None, sample_weight)

  File "C:\ProgramData\Anaconda3\envs\MLEnv\lib\site-packages\keras\utils\losses_utils.py", line 76, in squeeze_or_expand_dimensions
    elif weights_rank - y_pred_rank == 1:

TypeError: unsupported operand type(s) for -: 'int' and 'NoneType'

Now if I try to trace back that last step I get squeeze_or_expand_dimensions and realize I'm in an if block that only should fire if I have sample_weight - I don't. In any case, the code before it is:

y_pred_rank = K.ndim(y_pred)
weights_rank = K.ndim(sample_weight)
if weights_rank != 0:
    if y_pred_rank == 0 and weights_rank == 1:
        y_pred = K.expand_dims(y_pred, -1)
    elif weights_rank - y_pred_rank == 1:
        sample_weight = K.squeeze(sample_weight, -1)
    elif y_pred_rank - weights_rank == 1:
        sample_weight = K.expand_dims(sample_weight, -1)

There shouldn't be any way for y_pred_rank or weights_rank to end up None (and even if weights gets set to 1 earlier (as it appears to be in compute_weighted_loss), weights_rank should end up 0), but apparently it is. And how that relates to my new loss function is beyond me

Daniel F
  • 13,620
  • 2
  • 29
  • 55

1 Answers1

1

this dummy example on my machine without numba works:

def test_del(y_true, y_pred):
    cols = y_true.shape[1]
    out = 0
    for i in range(y_true.shape[1]):
        true_dam = np.abs(y_true[:, i]).max()  #toy
        pred_dam = np.abs(y_pred[:, i]).max()  #toy
        out += np.mean(np.abs(np.log(pred_dam / true_dam))**2)
    return out/cols

def del_loss(y_true, y_pred):
    return tf.numpy_function(test_del, [y_true, y_pred], tf.float64) +\
           K.cast(tf.keras.losses.mean_squared_error(y_true, y_pred), tf.float64)


inp = Input((10,))
x = Dense(30)(inp)
out = Dense(10)(x)
model = Model(inp, out)
model.compile('adam', del_loss)

model.fit(np.random.uniform(0,1, (3,10)), np.random.uniform(0,1, (3,10)), epochs=3)
Marco Cerliani
  • 21,233
  • 3
  • 49
  • 54