0

I am trying to differentiate between camera motion and tool motion in a surgical video. I have tried optical flow using opencv farneback and pass the results to an ML model to learn but no success.a major issue is getting good keypoints in case of camera motion. Is there an alternate technique to distinguish between camera motion and tool/tissue movement? Note: camera motion happens only in 10% of the video

Khushi
  • 1
  • 1

1 Answers1

0

I wish I could add a comment (too new to be able to comment), as I don't have a good answer for you.

I think it really depends on the nature of the input image. Can you show some typical input images here?

What is your optical flow result look like? I thought you might get some reasonable results.

Have you tried some motion estimation method, to analyze if there is global movement across different frames, or there is only some local movements?

Guang
  • 355
  • 1
  • 8
  • Hi Guang, thanks a lot for the response. what type of motion estimation methods? is there a sample article I can follow? the input images are typical laparoscopic surgery images inside the body. Do you think passing the hsv result to a ML classifier should have done the job? I am not able to understand if model is the issue or the optical flow itself. – Khushi May 30 '21 at 07:33
  • found a sample laparoscopic surgery video on youtube. at 2:30 the actual surgery starts. https://www.youtube.com/watch?v=J9mbWHnjsok . you can see tool and camera motions happening there – Khushi May 30 '21 at 07:39
  • Hi, I watched the video, in this case I think it is quite challenging, since the whole view is moving due to tissue movement as well as camera movement. I am curious what is your optical flow result look like for your actual data. I don't think there is any simple solution for your problem, you probably need some in-depth understanding of your data and available methods, and combine several techniques to find a solution that fits your problem the best. Good luck! – Guang Jun 01 '21 at 19:36
  • yes, tissue movement is causing issues while calculating optical flow. Thanks for taking a look though – Khushi Jun 01 '21 at 21:03