0

I created an autoencoder model and train it with image sequence dataset. The final result of testset is a reconstructed images. How can I plot ROC curve and calculate AUC?

Edited: This is my code:

    model = AutoEncoder()
    model.compile(loss='mse', optimizer='adam')
    model.fit(dm, dm, 
          batch_size=batchSize, 
          epochs=epochs, 
          shuffle=False,
          callbacks=callbacks_list, verbose=1
          )

    # load testset
    ....
    reconstructed_sequences = model.predict(sequences,batch_size=1)
    sequences_reconstruction_cost = np.array([np.linalg.norm(np.subtract(sequences[i],reconstructed_sequences[i])) for i in range(0,sz)])
    sa = (sequences_reconstruction_cost - np.min(sequences_reconstruction_cost)) / np.max(sequences_reconstruction_cost)
    sr = 1.0 - sa

where sr is a final result containing the score of my detection.

Alpha9
  • 101
  • 7
  • can you share some sample code on what you have done so far , so that it will be helpful – nithin Jan 03 '20 at 08:29
  • I have edited my post and add sample code. – Alpha9 Jan 03 '20 at 08:41
  • ROC and AUC are defined for binary classification, so how it is related to your autoencoder? – Dr. Snoopy Jan 03 '20 at 11:18
  • @MatiasValdenegro my work is anomaly detection which detect abnormal or normal event in a frame. the sr score decide that the frame is abnormal or normal. Most of the work I have review compute the AUC to compare to another work. – Alpha9 Jan 03 '20 at 13:35
  • This has already been answered, see the duplicate answer, where y_score is your sr – Dr. Snoopy Jan 03 '20 at 22:56

1 Answers1

0

you need to run a range of thresholds from min(sr) to max(sr) and for each value count the quadruple (True Negative (TN), True Positive (TP), False Negative (FN), False Positive (FP)) to obtain many contingency tables. Then calculate both Sensitivity = TP / (TP + FN) and Specificity = TN / (FP + TN) for each threshold. A Receiver Operating Characteristic (ROC) curve is the plot of 1-Specificity (X axis) versus Sensitivity (Y Axis). Should not be too hard to code in python.

bhamadicharef
  • 360
  • 1
  • 11