TypeError: Keras symbolic inputs/outputs do not implement `len`. When using custom loss function

Question

I am trying to develop an autoencoder that also uses the labels of the greyscale images I am trying to reconstruct. For that, I define a custom loss function.

However, when I try to run my code, I get this error:

TypeError: Keras symbolic inputs/outputs do not implement __len__. You may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model. This error will also get raised if you try asserting a symbolic input/output directly.

I confirmed, that it's being produced at return total_loss (see below)

I already tried the solutions provided here (trying to disable eager execution) and here (changing math operations to tf.math versions + disabling eager execution). Similar questions provide more or less the same answers or use additional libraries which I don't. However, none of the solutions work for me.

Here is the code I'm working with:

import tensorflow as tf
from tensorflow.keras.callbacks import EarlyStopping
from tensorflow import keras


def joint_loss(imgs_true, imgs_pred, y_true, y_pred, reconstruction_weight, 
               classification_weight): 
  # imgs_true = original images (= keras.input)
  # imgs_pred = reconstructed images of my autoencoder
  # y_true = true labels of my data (= keras.input)
  # y_pred = predicted labels from my bottleneck layer
  # reconstruction_weight/classification_weight = explanation below in "hyperparameters"
  reconstruction_loss = tf.reduce_mean(tf.square(imgs_true - imgs_pred))
  classification_loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(y_true, y_pred))
  total_loss = (tf.math.scalar_mul(reconstruction_weight, reconstruction_loss) + 
            tf.math.scalar_mul(classification_weight, classification_loss))
  
  return total_loss



# define function of the autoencoder that's to be optimized; give back validation loss
def create_and_train_autoencoder(encoding_dim, f1, f2, f3, f4, f5, k1, k2, 
                                 num_epochs, bat_size, learning_rate, 
                 recon_weight, class_weight):
    # explanation for parameters in "hyperparameter" below

    # input for images and labels
    input_img = keras.Input(shape=(64, 128, 1))
    input_label = keras.Input(shape=(1,))

    # Encoding layers
    x = keras.layers.Conv2D(f1, (k1, k2), activation='relu', padding='same')(input_img)
    x = keras.layers.MaxPooling2D((2, 2), padding='same')(x)
    x = keras.layers.Conv2D(f2, (k1, k2), activation='relu', padding='same')(x)
    x = keras.layers.MaxPooling2D((2, 2), padding='same')(x)
    x = keras.layers.Conv2D(f3, (k1, k2), activation='relu', padding='same')(x)
    x = keras.layers.MaxPooling2D((2, 2), padding='same')(x)


    # Bottleneck
    encoded = keras.layers.Conv2D(encoding_dim, (k1, k2), activation='relu', 
                                  padding='same')(x)


    # Define the classification branch
    encoded_flattened = keras.layers.Flatten()(encoded)
    encoded_flattened_dense1 = keras.layers.Dense(f4, activation='relu')(encoded_flattened)
    encoded_flattened_dense2 = keras.layers.Dense(f5, activation='relu')(encoded_flattened_dense1)
    label_output = keras.layers.Dense(1, activation='sigmoid')(encoded_flattened_dense2)


    # Decoding layers
    x = keras.layers.UpSampling2D((2, 2))(encoded)
    x = keras.layers.Conv2D(f3, (k1, k2), activation='relu', padding='same')(x)
    x = keras.layers.UpSampling2D((2, 2))(x)
    x = keras.layers.Conv2D(f2, (k1, k2), activation='relu', padding='same')(x)
    x = keras.layers.UpSampling2D((2, 2))(x)
    x = keras.layers.Conv2D(f1, (k1, k2), activation='relu', padding='same')(x)
    decoded = keras.layers.Conv2D(1, (k1, k2), activation='sigmoid', padding='same')(x)


    # create model and compile
    autoencoder = keras.Model([input_img, input_label], [decoded, label_output])

    autoencoder.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=learning_rate), 
                    loss=joint_loss(input_img, decoded, input_label, label_output,
                                    recon_weight, class_weight))
    
    
    # Create early stopping callback
    early_stopping = EarlyStopping(monitor='val_loss', patience=15, 
                                   restore_best_weights=True)
    
    
    # fit
    history = autoencoder.fit([x_train, y_train], [x_train, y_train],
                epochs=num_epochs,
                batch_size= bat_size,
                shuffle=True,
                validation_data=([x_test, y_test], [x_test, y_test]),
                callbacks=[early_stopping])

    
    return history.history['val_loss'][-1]


# execute the function
best_autoencoder = create_and_train_autoencoder(
    encoding_dim, f1, f2, f3, f4, f5, k1, k2, num_epochs, bat_size, 
    learning_rate, reconstr_weight, class_weight)

A little bit of background if needed:

I have 51 (64,128) greyscale images in a (51,64,128) array as my x_train. I have 13 (64,128) greyscale images in a (13,64,128) array as my x_test. I have 51 (1,) labels in a (51,) array as my y_train. I have 13 (1,) labels in a (13,) array as my y_test. My labels are [0,1].

For the sake of this example I'm using the following hyperparameters:

encoding_dim=14 # number filters for reduced dimension
f1 = 16 # number filters
f2=f3 = 8 # number filters
f4=f5 = 64 # number filters
k1=k2 = 3 # kernel size
num_epochs = 25 # number of epochs
bat_size = 32 # batch_size
learning_rate = 0.0001 # learning rate
reconstr_weight = 0.5 # parameter for manipulating joint loss
class_weight = 0.5 # parameter for manipulating joint loss

I'm using tensorflow 2.7. I'm not using Cuda.

Any help is much appreciated!

score 1 · Answer 1 · answered Feb 12 '23 at 13:10

1

In the function 'joint_loss' you are assigning a value to 'total_loss' variable, but returning the function itself (which doesn't make any sense). I guess instead of:

return joint_loss

You intention was:

return total_loss

answered Feb 12 '23 at 13:10

Andreas Kaufmann

581
3
8

Yup, this typo seems correct – Minh-Long Luu Feb 12 '23 at 13:44
You're right, this is a typo! I triple checked my code and there I didn't make that mistake. There I'm returning "total_loss" as intended. I edited my question accordingly. – T.A. Anderson Feb 12 '23 at 16:30

score 1 · Answer 2 · answered Feb 13 '23 at 16:26

It seems like model.add_loss() solves the problem. But can anyone explain to me why? Is it only because now my custom loss is a tensor instead of a function? And why wouldn't it work with my joint_loss being a function?

The full code at the according place would look like this:

# create model and compile
autoencoder = keras.Model([input_img, input_label], [decoded, label_output])

autoencoder.add_loss(joint_loss(input_img, decoded, input_label, label_output, recon_weight, class_weight))


autoencoder.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=learning_rate))

The rest remains the same.

score 0 · Answer 3 · answered Feb 17 '23 at 10:09

There are so many problems with your loss function. What I mean is you are passing each input-example to the loss function which is going from the model.fit() function. So, your loss function will not be able to access the local variable of the Tensorflow Graph. The thing which you are doing here autoencoder.add_loss(joint_loss(input_img, decoded, input_label, label_output, recon_weight, class_weight)) is also not a good way to pass your loss function. The difference between model.compile(loss) and model.add_loss() function is The model.compile(loss) method is used to specify the primary loss function for the model during training. This is typically a supervised learning problem, where you have a set of inputs and corresponding labels, and you want to minimize the difference between the predicted outputs of the model and the true labels.

On the other hand, the model.add_loss() function is used to add additional loss terms to the model that are not directly related to the supervised learning task. This is useful in situations where you want to add regularization terms, such as L1 or L2 regularization, to the loss function. These additional loss terms are added to the primary loss function specified by compile and are used to penalize certain model parameters in order to reduce overfitting.

Well, I have changed your code and used the tf.Gradient() as tape to compute the loss function, because in your case we cannot pass the input-image in each iteration through the model.compile() method.

x_train = tf.random.normal((51,64,128,1))
x_test = tf.random.normal((13,64,128,1))

y_train = tf.random.uniform((51,1), 0,2, dtype=tf.int32)
y_test = tf.random.uniform((13,1), 0,2, dtype=tf.int32)

train_examples = tf.data.Dataset.from_tensor_slices((x_train,y_train))
test_examples = tf.data.Dataset.from_tensor_slices((x_test,y_test))

train_examples = train_examples.batch(1)
test_examples = test_examples.batch(1)

encoding_dim=14 # number filters for reduced dimension
f1 = 16 # number filters
f2=f3 = 8 # number filters
f4=f5 = 64 # number filters
k1=k2 = 3 # kernel size
num_epochs = 25 # number of epochs
bat_size = 32 # batch_size
learning_rate = 0.0001 # learning rate
reconstr_weight = 0.5 # parameter for manipulating joint loss
class_weight = 0.5 # parameter for manipulating joint loss

def joint_loss(imgs_true, imgs_pred, y_true, y_pred, reconstruction_weight, classification_weight): 

  y_true = tf.cast(y_true , dtype=tf.float32)
  y_pred = tf.cast(y_pred , dtype=tf.float32)

  reconstruction_loss = tf.reduce_mean(tf.square(imgs_true - imgs_pred))
  classification_loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(y_true, y_pred))
  total_loss = (tf.math.scalar_mul(reconstruction_weight, reconstruction_loss) + 
            tf.math.scalar_mul(classification_weight, classification_loss))
  
  return total_loss



# define function of the autoencoder that's to be optimized; give back validation loss
tf.keras.backend.clear_session()
class AutoEncoder(tf.keras.Model):
  def __init__(self,encoding_dim, f1, f2, f3, f4, f5, k1, k2):
    super().__init__()

    self.conv2d_1 = keras.layers.Conv2D(f1, (k1, k2), activation='relu', padding='same')
    self.maxpool_1 = keras.layers.MaxPooling2D((2, 2), padding='same')
    self.conv2d_2 = keras.layers.Conv2D(f2, (k1, k2), activation='relu', padding='same')
    self.maxpool_2 = keras.layers.MaxPooling2D((2, 2), padding='same')
    self.conv2d_3 = keras.layers.Conv2D(f3, (k1, k2), activation='relu', padding='same')
    self.maxpool_3 = keras.layers.MaxPooling2D((2, 2), padding='same')

    self.bottelneck = keras.layers.Conv2D(encoding_dim, (k1, k2), activation='relu', padding='same')

    self.encoded_flattened = keras.layers.Flatten()
    self.encoded_flattened_dense1 = keras.layers.Dense(f4, activation='relu')
    self.encoded_flattened_dense2 = keras.layers.Dense(f5, activation='relu')
    self.label_output = keras.layers.Dense(1, activation='sigmoid')

    self.upsample_1 = keras.layers.UpSampling2D((2, 2))
    self.conv2d_4 = keras.layers.Conv2D(f3, (k1, k2), activation='relu', padding='same')
    self.upsample_2 = keras.layers.UpSampling2D((2, 2))
    self.conv2d_5 = keras.layers.Conv2D(f2, (k1, k2), activation='relu', padding='same')
    self.upsample_3 = keras.layers.UpSampling2D((2, 2))
    self.conv2d_6 = keras.layers.Conv2D(f1, (k1, k2), activation='relu', padding='same')
    self.decoded = keras.layers.Conv2D(1, (k1, k2), activation='sigmoid', padding='same')

    # explanation for parameters in "hyperparameter" below
  def call(self, input_img , input_label):
    # Encoding layers
    x = self.conv2d_1(input_img)
    x = self.maxpool_1(x)
    x = self.conv2d_2(x)
    x = self.maxpool_2(x)
    x = self.conv2d_3(x)
    x = self.maxpool_3(x)


    # Bottleneck
    encoded = self.bottelneck(x)


    # Define the classification branch
    encoded_flattened = self.encoded_flattened(encoded)
    encoded_flattened_dense1 = self.encoded_flattened_dense1(encoded_flattened)
    encoded_flattened_dense2 = self.encoded_flattened_dense2(encoded_flattened_dense1)
    label_output  = self.label_output(encoded_flattened_dense2)


    # Decoding layers
    x = self.upsample_1(encoded)
    x = self.conv2d_4(x)
    x = self.upsample_2(x)
    x = self.conv2d_5(x)
    x = self.upsample_3(x)
    x = self.conv2d_6(x)
    decoded = self.decoded(x)

    return [decoded, label_output]

autoencoder = AutoEncoder(encoding_dim, f1, f2, f3, f4, f5, k1, k2)
autoencoder(x_train[0:1,:,:] , y_train[:1,:])

Now, to train the model we have to use the custom-Gradient class.

optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)

#Metrics which are used to store the average loss and accuracy for per epoch
train_loss = tf.keras.metrics.Mean(name='train_loss')

@tf.function
def train_step(inp , tar_inp):
  with tf.GradientTape() as tape:
    decoded, label_output = autoencoder(inp, tar_inp)
    loss = joint_loss(inp, decoded, tar_inp, label_output, reconstr_weight, class_weight)

  gradients = tape.gradient(loss, autoencoder.trainable_variables) #Tape.gradient is used to compute the gradients
  optimizer.apply_gradients(zip(gradients, autoencoder.trainable_variables)) #This step is used to update the gradients

  train_loss(loss) #keep adding the loss

import time
for epoch in range(20):
  start = time.time()

  train_loss.reset_states()
 
  for (batch, [inp, tar]) in enumerate(train_examples):
    train_step(inp, tar)

    if batch % 50 == 0:
      print(f'Epoch {epoch + 1} Batch {batch} Loss {train_loss.result():.4f}')

  print(f'Epoch {epoch + 1} Loss {train_loss.result():.4f}')

  print(f'Time taken for 1 epoch: {time.time() - start:.2f} secs\n')

You can add the test-loss by yourself.

TypeError: Keras symbolic inputs/outputs do not implement `__len__`. When using custom loss function

3 Answers3

TypeError: Keras symbolic inputs/outputs do not implement `len`. When using custom loss function