0

Say I already have segmentation masks of some (biological) cells that I want to track; these cells are fairly similar to one another but do still vary slightly in shape and may also change shape as they are moving. Importantly, these cells do not divide (although they may overlap). My issue is most microscopy-focused tracking algorithms (e.g. trackpy, Ilastik) I've seen only use the cell's/object's centroid for tracking, which leads to mistakes when other cells are nearby. The mistakes are obvious to me since I can tell from the cells' shapes which one is which.

My question is, do people have recommendations for papers/packages that track multiple segmentation masks (preferably from cells but not necessarily) and also take into account the structure of the object (e.g. it's shape, brightness, etc.)? Preferably, these methods should be relatively robust to masks occasionally disappearing for a frame or two or being obscured.

Crazed
  • 87
  • 2
  • 8
  • you could try opencv's "multitracker" but that was a GSoC project and I don't think anyone's really using or maintaining that. last I checked, the entire tracking API left something to be desired. -- you could have better luck with some "DAsiamRPN" example from the dnn samples directory. -- all those trackers learn the appearance of their object... but if objects look generally similar (weren't trained to know the important ways in which cells differ), they can still have trouble with this task. – Christoph Rackwitz Jan 16 '23 at 14:24

1 Answers1

0

What you are looking for is a tracking algorithm that takes motion and appearance into consideration in the association process. The top two methods with these characteristics on paperswithcode, as of today, are:

But don't discard other advanced motion only solutions as they are usually good enough and less complex that than the combined ones.

DISCALIMER. I am the owner of this repo: https://github.com/mikel-brostrom/yolov8_tracking. There you will find such methods and hparam tracker tuning for your custom dataset in order to achieve optimal results

Mike B
  • 2,136
  • 2
  • 12
  • 31