0

In my latest project I am required to train a deep convolutional neural network to detect football players. The problem is that all my positive exapmles are extracted from 3-4 videos and the grass is almost the same colour in each one. I am looking for a background substraction method which will isolate the human figure from the green background. Then, I can change the colour of the grass to a different hue. Is it a better idea to perform an edge detection or a color segmentation algorith? Any ideas? Thanks

example image

Danai Tri
  • 119
  • 3
  • 13
  • Welcome to StackOverflow. Please read and follow the posting guidelines in the help documentation. [on topic](http://stackoverflow.com/help/on-topic) and [how to ask](http://stackoverflow.com/help/how-to-ask) apply here. StackOverflow is not a design, coding, research, or tutorial service. – Prune Jun 20 '17 at 17:59
  • In general, the answer to "which method is best" in such a broad application is "try them and see." Since you've shown no progress on integrating current methods (indeed, no code at all), and haven't supplied the data in question, there's not much we can do for you. Anything we could suggest would be no more than an under-informed guess. – Prune Jun 20 '17 at 18:02
  • Thank you for your answer. I should probably transfer this question to a more appropriate site. Stackoverflow has always helped me very much thats why It was my first option. I tried changing the green channel values (with no satisfying results) as well as performing a k-means clustering on my positive samples and then trying to use the average sample as a mask. I was just looking for new ideas and so I didnt provide any code. – Danai Tri Jun 21 '17 at 00:34
  • Perhaps ... but notice that two other people think your question is worth an answer. *I* think it should move to the stats group, but there are only 2 closure votes after 6 hours. Your choice. – Prune Jun 21 '17 at 00:36

2 Answers2

1

Problem

How can I address that the grass color (background) is all the same in training set for my CNN?

Solutions

Is it a problem?

One way you can test to see if it is a problem is to see if your trained model can handle color changes without having to be retrained. Feed in the test images with the colors reorderd (blue becomes red, red becomes green, green becomes blue) and see how well it does. Try de-colorizing the image so that all color layers are just black and white.

If your CNN preforms just as well (within the uncertainty in your accuracy measurements) then there is no problem.

Maybe color is the problem

Perhaps color is a problem. Try your whole CNN with just b/w instead of 3 colors. If trained using b/w is able to preform just as well, then the b/w model could be the bounds detector and your color CNN could be used for something else like team detection.

Style transfer

Use a style transfer NN to convert you training set images from having green grass to have the "style of" brown grass.

Specifically create a grass detector

Train a CNN to specifically detect grass in each NxN (11x11) patch. Use that to create a bit mask of the same shape as your input image which is a mask of "is this pixel part of grass or not". Multiply the output of that with the input image dot-wise to create a grassless image. Visually verify that it is indeed grasses. Apply the invers of the mask to a new background and add the grassless image and the new background.

Anton Codes
  • 3,663
  • 1
  • 19
  • 28
  • Thanks for answering, thats really helpful. I already applied my CNN to a set of test images with different hue and brightness than my training samples and it was not able to handle such changes. I believe it wont be able to perform well in color reordered samples but I am curious of testing this. I am not aware of style transfer neural networks but thanks for sharing. I actually thought of creating a graseless mask but I was trying to create this mask by clustering all samples and calculating the average mean image for each cluster. – Danai Tri Jun 21 '17 at 00:57
  • Are you suggesting to finetune the color sensitive CNN with 3-channel gray images (the gray matrix is duplicated 3 times) or completely change the architecture (the input will be height x width x 1 instead of height x width x 3) and train the network again from scratch? The second case will cut down the number of input features, isnt this going to have a bad effect for the generalization of the network? – Danai Tri Jun 22 '17 at 09:53
  • I also tried to reorder the color channels with no good results. The BGR image (the network was trained with BGR images) has a score of 14.5525 while the RGB image has a score of -19.2657. – Danai Tri Jun 22 '17 at 09:55
  • Whatever is easier for you to do a few experiments and test. It may perform worse, but after a year of working with and experimenting I'm still surprised. It may not do as well, but itay do well enough to be part is a collection of experts. – Anton Codes Jun 22 '17 at 12:39
  • you may find this paper on subtracting background using Neural Nets of interest https://arxiv.org/abs/1702.01731 – Anton Codes Jun 22 '17 at 13:11
1

A simple solution might be color segmentation i.e. select color range (of grass) and assign it a constant value OR you can change the pixel values in the green channel.

user8190410
  • 1,264
  • 2
  • 8
  • 15