0

I'm using the below script to replace the green screen of the original video with a background image but the result is not what I expected, I've changed u_green array parameters and also l_green but it only gets worst. In the end, I want to make it transparent which mask should I use?

I'd appreciate any help to fix this.

Python script:

import cv2
import numpy as np


video = cv2.VideoCapture("green.mp4")
image = cv2.imread("bg.jpg")

while True:

    ret, frame = video.read()

    frame = cv2.resize(frame, (640, 480))
    image = cv2.resize(image, (640, 480))

    u_green = np.array([104, 153, 70])
    l_green = np.array([30, 30, 0])

    mask = cv2.inRange(frame, l_green, u_green)
    res = cv2.bitwise_and(frame, frame, mask=mask)

    f = frame - res
    f = np.where(f == 0, f, image)

    cv2.imshow("video", frame)
    cv2.imshow("mask", f)

    if cv2.waitKey(25) == 27:
        break

video.release()
cv2.destroyAllWindows()

result : enter image description here

Update Source video: Link

Mohammad_Hosseini
  • 2,481
  • 3
  • 30
  • 53

1 Answers1

3

I did my best to mask out the screen using the HSV color space. There's still some green outline, but I can't increase the color margin any more without chopping out bits of clothes.

Edit: wrapped the code inside of a video loop.

Edit 2: I added a VideoWriter to save the results and swapped over to using the saturation channel since it had better separation.

Output Video:

https://drive.google.com/file/d/1GrECFwFy7JQJT6kUGrfLtlXjcfBsr7fP/view?usp=sharing

import cv2
import numpy as np

# open up video
cap = cv2.VideoCapture("video.mp4");

# grab one frame
scale = 0.5;
_, frame = cap.read();
h,w = frame.shape[:2];
h = int(h*scale);
w = int(w*scale);

# videowriter 
res = (w, h);
fourcc = cv2.VideoWriter_fourcc(*'XVID');
out = cv2.VideoWriter('test_vid.avi',fourcc, 30.0, res);

# loop
done = False;
while not done:
    # get frame
    ret, img = cap.read();
    if not ret:
        done = True;
        continue;

    # resize
    img = cv2.resize(img, res);

    # change to hsv
    hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV);
    h,s,v = cv2.split(hsv);

    # get uniques
    unique_colors, counts = np.unique(s, return_counts=True);

    # sort through and grab the most abundant unique color
    big_color = None;
    biggest = -1;
    for a in range(len(unique_colors)):
        if counts[a] > biggest:
            biggest = counts[a];
            big_color = int(unique_colors[a]);

    # get the color mask
    margin = 50;
    mask = cv2.inRange(s, big_color - margin, big_color + margin);

    # smooth out the mask and invert
    kernel = np.ones((3,3), np.uint8);
    mask = cv2.dilate(mask, kernel, iterations = 1);
    mask = cv2.medianBlur(mask, 5);
    mask = cv2.bitwise_not(mask);

    # crop out the image
    crop = np.zeros_like(img);
    crop[mask == 255] = img[mask == 255];

    # show
    cv2.imshow("Mask", mask);
    cv2.imshow("Blank", crop);
    cv2.imshow("Image", img);
    done = cv2.waitKey(1) == ord('q');

    # save
    out.write(crop);

# close caps
cap.release();
out.release();
Ian Chu
  • 2,924
  • 9
  • 14
  • This is awesome, thanks. but how can i use it for videos? as you can see my question is about how it can be done on videos with green screen background – Mohammad_Hosseini Feb 25 '21 at 04:19
  • you pull images out of opencv's VideoCapture – Ian Chu Feb 25 '21 at 05:20
  • But how? I'm not experienced with open cv – Mohammad_Hosseini Feb 25 '21 at 05:22
  • I'm getting this error: line 36, in _, contours, _ = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE); ValueError: not enough values to unpack (expected 3, got 2) – Mohammad_Hosseini Feb 25 '21 at 06:34
  • Should i upload my source video for you? – Mohammad_Hosseini Feb 25 '21 at 06:47
  • 2
    Regarding the error with `findContours`: @IanChu obviously uses some OpenCV 3.x version, where the method signature has three return values. You might use some OpenCV 4.x version, where there are only two return values. Use `contours, _ = cv2.findContours(...)` instead. – HansHirse Feb 25 '21 at 08:05
  • 3
    @IanChu To prevent those follow-up questions, try to use something like `cnts = cv2.findContours(...)` followed by `cnts = cnts[0] if len(cnts) == 2 else cnts[1]` in your answers. Most of the people asking OpenCV questions tend to use quite new OpenCV versions, and that `findContours` incompatibility will confuse them. – HansHirse Feb 25 '21 at 08:10
  • @IanChu I've updated the code using the HansHirse tip. it runs without any error but it does not affect video! – Mohammad_Hosseini Feb 25 '21 at 10:09
  • 1
    Gah sorry! I forgot to note my opencv version, my bad. When you say it doesn't affect video, what do you mean? The cv2.imshow() should be showing you the cropped image. Do you mean that you want to save a new video with just the cropped images? – Ian Chu Feb 25 '21 at 14:20
  • It would help if you uploaded the video to your question description. Then everyone could see what you're working with. – Ian Chu Feb 25 '21 at 14:22
  • I mean the output video and the source video are the same. I've updated my question with the link to the source video. – Mohammad_Hosseini Feb 25 '21 at 15:49
  • @IanChu You made my day, thanks, but is there any way to keep the audio? because the output has no sound. – Mohammad_Hosseini Feb 26 '21 at 10:33
  • as far as I know opencv doesn't really handle audio. You'll have to find some other library and try to overlay the audio on top of the new video. – Ian Chu Feb 26 '21 at 15:19
  • @IanChu Do you know any library which I can use for this matter? – Mohammad_Hosseini Feb 27 '21 at 11:30
  • 1
    this isn't something I've tried to do before, so I don't know of any particular method. Here's what a quick google search brings up: https://stackoverflow.com/questions/46864915/python-add-audio-to-video-opencv – Ian Chu Feb 27 '21 at 15:15