tfa.metrics.F1Score custom metrics error : "Shapes must be equal rank"

Question

I am trying to add F1Score as a metrics for a seq to seq classification task. The shapes of y_true and y_pred are the same but my custom metrics class keeps printing the following error :

ValueError: Shapes must be equal rank, but are 1 and 2 for '{{node AssignAddVariableOp_2}} = AssignAddVariableOp[dtype=DT_FLOAT](AssignAddVariableOp_2/resource, Sum_2)' with input shapes: [], [1024,15].

Here is the metrics class :

class F1metrics(keras.metrics.Metric):
    def __init__(self, threshold = 0.5, num_classes = 15, average = 'macro', **kwargs):
        super().__init__(**kwargs)
        self.average = average
        self.num_classes = num_classes
        self.threshold = threshold
        self.metric = tfa.metrics.F1Score(average = self.average, num_classes = self.num_classes, threshold = self.threshold)
        self.result = None
        
    def update_state(self, y_true, y_pred, sample_weight=None) :
        y_true = tf.cast(y_true, dtype = tf.float32)
        metric = self.metric
        metric.update_state(y_true, y_pred)
        self.result = metric.result()
        
    def result(self):
        return tf.reduce_mean(self.result)

Here is the model :

keras.backend.clear_session()

model = keras.models.Sequential([
    keras.layers.Input(shape = (1024,)),
    keras.layers.Embedding(vocab_size, 512, input_length = 1024, mask_zero = True),
    keras.layers.Bidirectional(keras.layers.GRU(128, return_sequences = True)),
    keras.layers.Bidirectional(keras.layers.GRU(64, return_sequences = True)),
    keras.layers.Bidirectional(keras.layers.GRU(32, return_sequences = True)),
    keras.layers.TimeDistributed(keras.layers.Dense(15, activation = 'softmax'))
])

model.summary()

model.compile(loss = 'categorical_crossentropy',
              optimizer = keras.optimizers.Nadam(learning_rate = 1e-4),
              metrics = [F1metrics()])

model.fit(train_set, 
          epochs = 5, 
          validation_data = val_set)

model summary

Both train and val sets are tf.data.Datasets :

<PrefetchDataset shapes: ((None, 1024), (None, 1024, 15)), types: (tf.int64, tf.int32)>

The data is text (tokenize + pad tokens). And i am trying to do some text classification thanks to the IOB method. Thus labels are one hot encoded. Model works fine with regular keras metrics.

Generating random data for testing :

X_test = np.zeros((1000, 1024))
Y_test = np.zeros((1000, 1024, 15))
for i in range(1000):
    mask = np.random.randint(1, 1024)
    X_test[i] = np.concatenate([np.random.randint(1,10000,mask), np.zeros(1024-mask)])
    for j in range(1024):
        rand_label = np.random.randint(0,15)
        Y_test[i][j][rand_label] = 1

Could someone help ? Thanks a lot.

tfa.metrics.F1Score custom metrics error : "Shapes must be equal rank"

0 Answers0