4

I am trying to understand the appropriate use of AUC ROC in my keras model.

The specific model I am training is a multi-class, multi-lab

el classifier, where each sample can belong to multiple classes. the code is as follows (starting at one-hot encoding):


##one-hot encode 
categoryone=to_categorical(data['categoryone'],num_classes=12)
categorytwo=to_categorical(data['categorytwo'],num_classes=7)

trainingtarget = np.concatenate((categoryone,categorytwo), axis=1)

each sample can belong to to one class in categery one and one class in categorytwo: the model is built (4 layer model) and compiled and fit as follows:

metrics=[tf.keras.metrics.AUC(name='auc'), tf.keras.metrics.Precision(name='precision'),
      tf.keras.metrics.Recall(name='recall')]

finalmodel=tf.keras.models.Sequential()
finalmodel.add(tf.keras.layers.Dense(units=832, input_dim=20767, activation="relu"))
finalmodel.add(tf.keras.layers.Dense(units=976, activation="relu"))
finalmodel.add(tf.keras.layers.Dense(units=880, activation="relu"))
finalmodel.add(tf.keras.layers.Dense(units=944, activation="relu"))
finalmodel.add(tf.keras.layers.Dense(units=19, activation="sigmoid"))

finalmodel.compile(loss="binary_crossentropy", optimizer=optimizer, metrics=metrics)

#fit model
finalmodel.fit(normCountsscale1.values, 
              trainingtarget,
              epochs=200,
              validation_split=0.1,
              callbacks=[(early_stopping, checkpoint)],
              shuffle=True,
              batch_size=32, 
              verbose=1
              )

I am aware that AUC ROC and precision are useful for binary classifiers (either label 1 or label 2),

my question is, what is the appropriate metric to use for a multi-class multi-label network as above (binary_crossentropy for belonging to more than one class)?

Does using the AUC distinguish between the appropriate classes here? i.e, can it successfully measure separability in category one AND in category two? and is this sufficient to use as a metric along with the usual confusion matrix, loss accuracy curves etc?

thanks

0 Answers0