2

I need to use Gaussian Mixture Models on an RGB image, and therefore the dataset is quite big. This needs to run on real time (from a webcam feed). I first coded this with Matlab and I was able to achieve a running time of 0.5 seconds for an image of 1729 × 866. The images for the final application will be smaller and therefore the timing will be faster.

However, I need to implement this with Python and OpenCV for the final application (I need it to run on an embedded board). I translated all my code and used sklearn.mixture.GMM to replace fitgmdist in Matlab. The line of code calculating the GMM model itself is performed in only 7.7e-05 seconds, but the one to fit the model takes 19 seconds. I have tried other types of covariance, such as 'diag' or 'spherical', and the time does reduce a little but the results are worse and the time is still not good enough, not even close.

I was wondering if there is any other library I can use, or if it would be worth it to translate the functions from Matlab to Python.

Here is my example:

import cv2
import numpy as np
import math
from sklearn.mixture import GMM

im = cv2.imread('Boat.jpg');

h, w, _ = im.shape;       # Height and width of the image

# Extract Blue, Green and Red
imB = im[:,:,0]; imG = im[:,:,1]; imR = im[:,:,2];

# Reshape Blue, Green and Red channels into single-row vectors
imB_V = np.reshape(imB, [1, h * w]);
imG_V = np.reshape(imG, [1, h * w]);
imR_V = np.reshape(imR, [1, h * w]);

# Combine the 3 single-row vectors into a 3-row matrix
im_V =  np.vstack((imR_V, imG_V, imB_V));

# Calculate the bimodal GMM
nmodes = 2;
GMModel = GMM(n_components = nmodes, covariance_type = 'full', verbose = 0, tol = 1e-3)
GMModel = GMModel.fit(np.transpose(im_V))

Thank you very much for your help

Zynk
  • 2,767
  • 3
  • 12
  • 11
  • are you using the GMM for background subtraction? – Miki May 18 '16 at 13:27
  • Yes, that's what I am doing – Zynk May 18 '16 at 13:35
  • Why you're not using OpenCV [backgroundSubtractorMog](http://docs.opencv.org/3.1.0/db/d5c/tutorial_py_bg_subtraction.html#gsc.tab=0) or backgroundSubtractorMog2? – Miki May 18 '16 at 13:36
  • Thanks, I will have a look. However, I want to use GMM because my background is changing and my camera is moving. – Zynk May 18 '16 at 13:41
  • 1
    Uh.... background subtraction with moving camera... good luck with that ;D – Miki May 18 '16 at 14:05
  • Thanks! I actually have reasons to believe it will work (I have tested with Matlab already), my main problem now is real time with Python and OpenCV! – Zynk May 18 '16 at 14:16
  • In a video application, consequtive images will be very similar. So you should use a GMM across frames, such that it only needs a couple of EM iterations on each frame. – Jon Nordby Aug 19 '22 at 14:26
  • And you should downscale the images before fitting. If full-resolution output is needed, then upscale the masks afterwards. – Jon Nordby Aug 19 '22 at 14:27

1 Answers1

1

You can try fit with the 'diagonal' or spherical covariance matrix instead of full. covariance_type='diag' or covariance_type='spherical'

I believe it will be much faster.

Michael D
  • 1,711
  • 4
  • 23
  • 38
  • Thank you, I tried this but didn't get as good results as with full covariance matrix, and this was significant for this application. – Zynk Aug 23 '16 at 12:15
  • maybe you can increase the number of components of GMM and it can compensate/ improve result – Michael D Dec 06 '17 at 09:34