0

I am looking for the fastest real-time object trackers for tracking small fast-moving stones that are free-falling vertically, given that there can be up to 50 objects in a single frame, and their shape is very similar.

I have trained a YoloV5 object detection model on stones and the inference speed is doing pretty good (120 FPS), but when I pass the .pt weights file to DeepSort algorithm for object tracking and test it on a normal speed video, it does not track my objects at all. However, I tried to Slow-Motion the video to * 0.25 speed and re-tested DeepSort and it worked, but was not able to associate stones and differentiate well between them (one ID is given to multiple objects).

Note: I am using the pre-trained weights on pedestrians of the deep part of DeepSort.

Is there any solution to:

1- Make the model work on the normal video without having to slow-motion the video?

2- Solve the problem of ID switching and ID repeating?

3- Should I re-train the deep part of DeepSort on my dataset of stones? or I can use the pre-trained weights?

Any help of any kind will be very appreciated :)

Yasser Jaber
  • 9
  • 1

2 Answers2

1

1- Make the model work on the normal video without having to slow-motion the video?

Most of the Github repos that implement DeepSort perform the tracking offline. This is, when the object detection + association process is done for a certain frame it takes the next, and so on, till it is done. So the FPS of your video shouldn't affect your tracking results as the only thing that changes in the video by slowing it down is the presentation timestamp (PTS) of each video frame.

2- Solve the problem of ID switching and ID repeating?

Most of the DeepSort implementations on Github (https://github.com/nwojke/deep_sort, https://github.com/ZQPei/deep_sort_pytorch) have not implemented Lambda as per Eq(5) in the paper. This implies that the position of the objects are not taken into consideration when performing the ID association. In you case this is a waste of information, specially as the stones are falling and their movement is easily predictable.

3- Should I re-train the deep part of DeepSort on my dataset of stones? or I can use the pre-trained weights?

Visually, your stones most likely look very similar. This means that training a custom ReID model on stones would have very little effect on you final tracking results. Hence, in your specific case, it is more important that the stones' position gets into consideration when performing the ID association, so we are back at the previous point.

Here is a repo that implements most of what you need (https://github.com/mikel-brostrom/Yolov5_DeepSort_Pytorch)

Mike B
  • 2,136
  • 2
  • 12
  • 31
0

Start with computer vision basics before powering your YOLOv5 model. Have you heard about the atmospheric turbulence model? You can read about it here or just check Chapter 5 (Image Restoration and Reconstruction) of Digital Image Processing 3rd Edition by Rafael Gonzalez.

Perhaps this paper help you understand more about moving objects: https://openaccess.thecvf.com/content/CVPR2021/html/Rozumnyi_DeFMO_Deblurring_and_Shape_Recovery_of_Fast_Moving_Objects_CVPR_2021_paper.html

Good luck and enjoy!

Illustrati
  • 311
  • 1
  • 9