0

I am using c++,OpenCV library and in my software, I have estimated the optical flow in a video. Now, I want to group some moving objects, e.g. moving cars. I have used a dense optical flow algorithm (Farneback).

My first thoughts so far are to use "k means" algorithm to do the clustering.

I have thought of using the results of the Farneback optical flow to compute the displacement of the frames in each direction as following :

E.g. :

Let Dx be the displacement in x direction( either positive or negative) and Dy the displacement in y direction(either positive or negative) .

Then i pass the array [Dx,Dy] as an input to k means with k=2 clusters. I hope this will give a rough background / foreground substraction .

However i am facing problems in computing the displacemet because the output of calcOpticalFlowFarneback is InputOutputArray flow . Should I access this array using a function like that for example ? : 

findDisplacements(const Mat& flow, int step) {
const Point2f& Dx,Dy;
const Point2f& fxy = flow.at<Point2f>(y, x);
Dx=Point(cvRound(x+fxy.x))-Point(x,y);
Dy=Point(cvRound(y+fxy.y))-Point(y,x);
}
george_t
  • 188
  • 1
  • 3
  • 19
  • What things you have tried so far? – Gupta Mar 08 '16 at 09:58
  • I have already used kmeans which works i guess and returns an array bestLabels. However i haven't managed to show the result as an image. I am working also on another code using background substraction which also works . Should i combine them ? Or use for example background substraction first and k means later ? – george_t Mar 08 '16 at 10:12
  • What dense optical flow algorithm did you use? – ChronoTrigger Mar 17 '16 at 20:36
  • What is the value of optical flow here? You try to differentiate objects by direction/speed? For that maybe yes, if you know the `k`. But nevertheless, you should know which motion vector the label corresponds to (i guess you formed them on scan line order). – mainactual Mar 17 '16 at 22:13
  • @mainactual i am aiming at differentiating objects by direction . The value of k depends on my selection i guess. Could you please explain further your last comment in the parenthesis about scan line order ? What does it mean ? – george_t Mar 18 '16 at 01:20
  • @ChronoTrigger i used Farneback's method . – george_t Mar 18 '16 at 01:20

2 Answers2

0

A small example where the motion vectors are used as feature to cluster:

cv::Mat img0 = cv::imread("test0.png");    // Image at time t-1
cv::Mat img1 = cv::imread("test1.png");    // Image at time t
cv::Mat flow;
//estimate Optical Flow
cv::calcOpticalFlowFarneback(img0, img1, flow, 0.5, 3, 21, 20, 5, 1.1);
std::vector<cv::Point2f> samples(flow.rows * flow.cols);
// arange sample vector
int n = 0;
for( int r = 0; r < flow.rows; r++) {
  for( int c = 0; c < flow.cols; c++){
    samples[n++] = flow.at<cv::Point2f>(r,c);
}}
cv::kmeans(samples, ...
Tobias Senst
  • 2,665
  • 17
  • 38
0

You can use the history of motion.

updateMotionHistory(silh, mhi, timestamp, MHI_DURATION);
calcMotionGradient(mhi, mask, orient, MAX_TIME_DELTA, MIN_TIME_DELTA, 3);
segmentMotion(mhi, segmask, regions, timestamp, MAX_TIME_DELTA);
Video Analysis Deep Learning
  • 338
  • 2
  • 12