0

I had hard time working on caffe with HDF5 on the image classification and regression tasks, for some reason, the training on HDF5 will always fail at the first beginning that the test and train loss could very soon drop to close to zero. after trying all the tricks such reducing the learning rate, adding RELU, dropout, nothing started to work, so I started to doubt that the HDF5 data I am feeding to caffe is wrong.

so currently I am working on the universal dataset (Oxford 102 category flower dataset and also it has public code ), firstly I started out by trying ImageData and LMDB layer for the classification, they all worked very well. at last i used HDF5 data layer for the finetuning, the training_prototxt doesn't change unless on the data layer which uses HDF5 instead. and again, at the start of the learning, the loss drops from 5 to 0.14 at iteration 60, 0.00146 at iteration 100, that seems to prove that HDF5 data is incorrect.

i have two image&label to HDF5 snippet on the github, all of them seem to generate the HDF5 dataset, but for some reason these dataset doesn't seem to be not working with caffe

I wonder anything wrong with this data, or anything that makes this example run in HDF5 or if you have some HDF5 examples for classification or regression, which can be helpful to me a lot.

one snippet is shown as 

def generateHDF5FromText2(label_num):
    print '\nplease wait...'

    HDF5_FILE = ['hdf5_train.h5', 'hdf5_test1.h5']
        #store the training and testing data path and labels
    LIST_FILE = ['train.txt','test.txt']
    for kk, list_file in enumerate(LIST_FILE):

    #reading the training.txt or testing.txt to extract the all the image path and labels, store into the array
    path_list = []
    label_list = []
    with open(list_file, buffering=1) as hosts_file:
        for line in hosts_file:
            line = line.rstrip()
            array = line.split(' ')
            lab = int(array[1])
            label_list.append(lab)
            path_list.append(array[0])  

        print len(path_list), len(label_list)

                # init the temp data and labels storage for HDF5
        datas = np.zeros((len(path_list),3,227,227),dtype='f4') 
        labels = np.zeros((len(path_list), 1),dtype="f4")

        for ii, _file in enumerate(path_list):
                    # feed the image and label data to the TEMP data
            img = caffe.io.load_image( _file )
            img = caffe.io.resize( img, (227, 227, 3) ) # resize to fixed size
            img = np.transpose( img , (2,0,1))
            datas[ii] = img
            labels[ii] = int(label_list[ii])

            # store the temp data and label into the HDF5
        with h5py.File("/data2/"+HDF5_FILE[kk], 'w') as f:
            f['data'] = datas
            f['label'] = labels
            f.close()
Shai
  • 111,146
  • 38
  • 238
  • 371
user824624
  • 7,077
  • 27
  • 106
  • 183
  • you are working on RGB images *without* any transformation (e.g., RGB->BGR, mean subtraction, etc.) it might be the case that your trained model was trained using some transformtations. – Shai Oct 19 '15 at 13:09
  • One way of debugging this is take `X` images, convert them to `lmdb` and store them to HDF5. build a network with two input layers: one that reads lmdb the other reads HDF5. Then add a `EucleadeanLoss` layer comparing the two input images. This should indicate if the two images are identical or not. – Shai Oct 19 '15 at 13:13
  • it might be. how would I do mean subtraction in Caffe HDF5 Layer, in LMDB layer I could configure 'transform_param> mean_file' in prototxt. – user824624 Oct 19 '15 at 13:17
  • you can subtract the mean when you create the HDF5 layer. – Shai Oct 19 '15 at 13:18
  • one more thing: what is the intensity range of your HDF5 images? is it [0..1] or [0..255]? you might need to scale the data to the proper range... – Shai Oct 19 '15 at 13:30
  • I have tried [0...1] and [0...255] both, and both doesn't seem to work so far. I see examples to scale intensity to [0...1],but also I hear that caffe works on [0...255] whereas python images works on [0...1]. It all confuses me. – user824624 Oct 19 '15 at 13:33
  • are you trying to train a model from scratch, or are you trying to fine-tune existing caffemodel weights? – Shai Oct 19 '15 at 13:35
  • right now, I am trying to finetune the oxford-flower dataset and model, it is finetuned from existing imagenet model. I have updated the link of dataset and model , and snippet on my post. – user824624 Oct 19 '15 at 13:41
  • you must subtract mean from images! – Shai Nov 18 '15 at 07:22

1 Answers1

2

One input transformation that seems to happen in the original net and is missing from your HDF5 creation in mean subtraction.
You should obtain mean file (looks like "imagenet_mean.binaryproto" in your example), read it into python and subtract it from each image.

BTW, the mean file can give you a clue as to the scale of the input image (if pixel values should be in [0..1] range or [0..255]).

You might find caffe.io useful converting binaryproto to numpy array.

Shai
  • 111,146
  • 38
  • 238
  • 371