16

I am working on an image segmentation project, and have been trying to adopt the image augmentation technique to increase the training set size. At first, I just try to use the horizontal flip to double the image size, but I found the performance was much worse than not using it. Are there any insight that can be shared. Thanks.

hafiz031
  • 2,236
  • 3
  • 26
  • 48
user288609
  • 12,465
  • 26
  • 85
  • 127

3 Answers3

17

So basically you need to answer yourself one important question: Is a flipped image a valid image in your domain?

  1. If not - then it may harm your training process simply because you are providing a network an invalid input which may learn your network spurious patterns in your data. It's not so rare that flips might harm your training - e.g. in logo recognition it's important to not change the orientation of your data in order to learn logos correctly.

  2. If yes - then there might be loads of different reason why your model started to behave worse. One of them might be that it has simply too small capacity and it's not able to learn all the patterns in your data. Second - that you have not enough examples - and when you add the flipped image it turned out that it in fact memoized loads of your traning cases. Another thing is that maybe you learnt it for a too small amount of time and setting the number of iterations to a bigger value might be a good idea.

    One thing is sure - your model is not generalizing well since your flipped data is valid.

Marcin Możejko
  • 39,542
  • 10
  • 109
  • 120
  • Thank you! With respect to my case, if I flip the training image, it still makes sense to me. As a result, I will assume the flipped image is still a valid image in my domain. I will see whether the thres scenarios you mentioned in the item 2 can give me any guidance. Besides, if the model is not generalizing, are there any suggestions to overcome it. I am using segnet type of framework for segmentation work. – user288609 Feb 10 '17 at 16:44
  • What kind of segnet do you use in this case? Could you provide the insights on type and amount of data you have? – Marcin Możejko Feb 10 '17 at 17:59
  • The data is like remote sensing images. I have 100 images with 600*800 size. I tried u-net and segnet. – user288609 Feb 12 '17 at 05:36
  • 1
    This might be the case - the amount of data you provided is really really small. – Marcin Możejko Feb 22 '17 at 19:28
  • So - has my answer helped you? – Marcin Możejko Jul 05 '17 at 12:42
3

When you say that the performance is worse with data augmentation, are you comparing both on the same dataset?

I just made the mistake of comparing the accuracy of my model trained with data augmentation on the augmented dataset with the model trained without data augmentation on the regular dataset.

I was surprised to see that the accuracy was worse with data augmentation. However, when I compared both accuracies on the dataset without augmentation, the model with data augmentation showed better performance than the other one.

It is important to keep in mind that augmented datasets can be harder to deal with for the model. Therefore, even if the accuracy isn't as high as before, it might be actually higher when evaluated on the regular dataset.

Wise Cloud
  • 411
  • 4
  • 5
2

Image augmentation is a great way to stretch your dataset, but as you've shown, it's not a magic bullet. Image augmentation works (to a degree) by varying image features that are irrelevant to the model's underlying mapping function (i.e. image brightness shouldn't correlate to presence of a dog), while still leaving the objects in the image recognizable.

I think the easiest improvement you could make would be to vary your augmentation techniques. Instead of just flipping images horizontally, try zooming, cropping, rotating, stretching, adjusting brightness, contrast, adding noise, etc. This will vary your original images more than just one mode of augmentation. This blog I wrote for work goes through different types of augmentation and what they do, and this library is how we prefer to implement image augmentation.

Different Augmentation Examples: Shear Shear Augmentation Noise: Noise Augmentation Color Space: Color Space Augmentation

You always run the risk of overfitting your model to your training dataset by relying too much on augmentation to increase its size, but varying your augmentation techniques will help you avoid overfitting quite as much. If you have the resources, nothing works like fresh new data, and if you want to get super fancy, you can look into generative adversarial networks, in which you can basically create new data from scratch.

B Cohen
  • 21
  • 1