How to remove shadows from a video that has static background?

Question

I am trying to detect moving object and remove shadow from a video that has a static background. I am using Mixture of Gaussians(MOG) method to detect moving objects. I am using opencv3 and python 3.5. How can I remove shadows from the video and foreground mask both? I have used erosion and dilation for reducing noise. But it doesn't remove the shadows.

import cv2
import numpy as np

cap = cv2.VideoCapture('TownCentreXVID.avi')
fgbg = cv2.createBackgroundSubtractorMOG2()

while(1):

    _, frame = cap.read()
    mask = fgbg.apply(frame)



    kernel = np.ones((5,5),np.uint8)
    opening = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)
    closing = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)

    window = cv2.namedWindow('Original', cv2.WINDOW_NORMAL| cv2.WINDOW_KEEPRATIO )
    window = cv2.namedWindow('Mask', cv2.WINDOW_NORMAL| cv2.WINDOW_KEEPRATIO)
    window = cv2.namedWindow('Opening', cv2.WINDOW_NORMAL| cv2.WINDOW_KEEPRATIO )
    #window = cv2.namedWindow('Closing', cv2.WINDOW_NORMAL| cv2.WINDOW_KEEPRATIO)

    cv2.imshow('Original',frame)
    cv2.imshow('Mask',thresh)
    cv2.imshow('Opening',opening)
    #cv2.imshow('Closing',closing)

    k = cv2.waitKey(5) & 0xFF
    if k == 27:
        break

cv2.destroyAllWindows()
cap.release()

Can you post the whole code?`fgbg1 =cv2.BackgroundSubtractorMOG2(setShadowValue=0)` I am using this line now. But `cv2.imshow` gives error saying Mat is neither numpy array nor scalar — Mayukh Das, May 07 '19 at 20:40
Just take the line I wrote and put it after `fgbg = cv2.createBackgroundSubtractorMOG2()` — Christoffer, May 08 '19 at 07:26
That doesn't remove shadow. It don't mark shadow. If I use `setDetectshadows(1)` it marks the shadow in grey color. — Mayukh Das, May 08 '19 at 18:47

score 0 · Answer 1 · answered Apr 25 '19 at 17:30

0

The backgroundsubtractor returns a mask where foreground object are white and shadows are gray.
You can use thresholding to create a new mask without shadow, or with only the shadow.
Use the mask without the shadows to get only the foreground.
Use the mask with only shadow to replace the shadow on the background (with a reference background image).

Result:

Code:

import cv2
import numpy as np
# load image / mask
mask = cv2.imread("mask.png",0)
#threshold mask
ret, foreground = cv2.threshold(mask, 200, 255, cv2.THRESH_BINARY)
ret, shadow = cv2.threshold(mask, 200, 255, cv2.THRESH_TOZERO_INV)
# stack images vertically
res = np.concatenate((mask,foreground,shadow),axis=0)
#show image
cv2.imshow("Result",res)
cv2.waitKey(0)
cv2.destroyAllWindows()

answered Apr 25 '19 at 17:30

J.D.

4,511
2
7
20

Thanks. Your code works correctly. Is there any way to do the shadow removal in the video too,not in the mask only? – Mayukh Das Apr 26 '19 at 20:46
When there are no foreground objects detected in a frame, then store a copy of that frame. You can use this image to replace the shadow area ([example](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_core/py_image_arithmetics/py_image_arithmetics.html#bitwise-operations) ). For this, use the separated-shadow as a mask. Note: to use the shadow-gray as a mask, you'll have you do an extra threshold, so the gray is converted to white. – J.D. Apr 26 '19 at 22:38
Thanks. I will try that. – Mayukh Das Apr 27 '19 at 18:18
how can I create ROI here? because shadow will not be in same places in different frames. – Mayukh Das Apr 28 '19 at 17:41
Take the entire frame. Take the mask with only the shadow (which should be the same size as the frame. Create its negative using `shadow_mask_inverted = cv2.bitwise_not(shadow_mask)` Do `img1 = cv2.bitwise_and(frame,frame,shadow_mask_inverted)`, `img2 = cv2.bitwise_and(frame_without_object,frame_without_object,shadow_mask)`, `result = cv2.add(img1,img2)` – J.D. Apr 28 '19 at 17:49
If you need more help, it is best to open a new question ;) – J.D. Apr 28 '19 at 17:50

How to remove shadows from a video that has static background?

1 Answers1