0

Problem: I am getting very low (10%) accuracy when testing MNIST classification with MatCaffe.

Background I trained the MNIST digits using the lenet_train_test and lenet_solver and observed ~99% test accuracy in caffe basic interface. I saved the caffmodel and wanted to verify the accuracy using MatCaffe interface. Therefore, I created a deploy.prototxt file (using lenet_train_test), with batch size 10000 (equal to the number of test images). I also saved the MNIST test images and labels in a single mat file. My Matlab code is given below:

%% Load Data and Labels
load('mnist_test_lmdb.mat')
I = infoData.imgData; % All images
true_labels = infoData.labelInfo; % All labels

% Model deployment file and weights
modelName = 'lenet_deploy.prototxt';
modelWeight = 'lenet_iter_10000.caffemodel';

% Set-up net
caffe.set_device(1);
caffe.set_mode_gpu();
net = caffe.Net(modelName, modelWeight, 'test');

% Preprocess images
scale= 0.00390625;
J = single(I) * scale;
J = reshape(J, size(J,1), size(J,2), 1, size(J,3));

% Run Network
net.forward({J});
probs = net.blobs('prob').get_data()';

% Get predicted labels        
[~, pred_labels] = max(probs, [], 2);
pred_labels = pred_labels' - 1; % class label begins from 0

accuracy = sum(pred_labels(:,1)==true_labels(:,1))/size(true_labels,1);

I would like to know if I am missing something here in the code?

Hasnat
  • 664
  • 1
  • 7
  • 21
  • Are you preprocessing the images in exact same way that was used during training? – Prophecies Oct 21 '16 at 18:53
  • I think yes. As per the [lenet_train_test](https://github.com/BVLC/caffe/blob/master/examples/mnist/lenet_train_test.prototxt#L11) file, I just multiply the image with `scale= 0.00390625` to convert it within [0 1] range. And these images are extracted from the same lmdb file and I verified that they are 8 bit images i.e., within the range [0 256]. – Hasnat Oct 22 '16 at 12:01

0 Answers0