Nvidia Digits accuracy and loss plots data

Question

I trained my model in Nvidia Digits 5 and I would now like to extract the accuracy and loss plots that were generated during training for a report. Is this data saved somewhere so that it would possible to extract the data for these plots so that I could plot it in Python and perhaps ultimately modify the plots to compare different models etc?

Answer 1

The best solution I have found is to either look at the HTML file or to scan the text file caffe_output.log that is produced by Caffe. The text file is usually stored in /var/digits/jobs/insert_your_job_id/ but you can also just run on linux systems:

locate caffe_output.log

Answer 2

Go to your DIGITS job folder and locate your job's subfolder. Inside you'll find a file status.pickle, which is a pickled object containing all your job's information. You can load it in python like so:

import digits
import pickle
data = pickle.load(open('status.pickle','rb'))

This object is somewhat generic and may contain multiple tasks. For a typical classification task it will likely be just one, but you will still need to access it via data.tasks[0]. From there you can grab the plots:

data.tasks[0].combined_graph_data()

which returns a somewhat convoluted dict (unfortunately - since your network can produce many accuracy/loss outputs, as well as even custom ones). It contains everything you need though - I managed to plot accuracy with:

plt.plot( data.tasks[0].combined_graph_data()['columns'][2][1:] )

but it's likely that you'll have to write a bit of custom code. As always, dir() is your friend.