2

I have some image processing that allows me to extract a binary image containing thick segments and i'm facing the issue that these segments may cross each other. Hence I need to find an efficient way to separate them, i'll have to implement this in C++ so anything OpenCV-based would help.
Here is a sample input image, both "blobs" would need to be split in 3 different segments.

input image

I have tried 2 ideas until now, I'm stuck with both of them and that's why I'm asking here if there are any "state of the art" solution to this apparently simple problem.

  1. My first idea was to compute the skeleton of the blobs, find intersection points. This part was easy. Then I planned to start with an end-point, traverse the segment until I reach an intersection and "cross the intersection". If you look at the bottom blob, the skeleton can produce weird "Y" shapes even when a "T" shape was expected, making it impossible to decide how to traverse an intersection.

  2. Second idea was to compute the distance transform, followed by the gradient direction, and then I still need a way to merge pixels with similar direction and a way to handle endpoints / intersections.

image transform original and transform overlapped

Julien M
  • 657
  • 3
  • 10
  • 40
  • You may get started with [Harris Corner Detection](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_feature2d/py_features_harris/py_features_harris.html) for finding the intersection points. Here is a MATLAB execution [result](https://i.stack.imgur.com/JqD5P.png). You can try cutting across two close intersection "key points", when the distance is close. For "T" shape it looks straight forward. For X shape, you can search for two lines that are almost parallel. – Rotem Dec 27 '19 at 22:29
  • Did you try Hough transform for extracting lines? – MeiH Dec 28 '19 at 07:51
  • @Rotem that would give me intersection just like I did with my skeleton, what would I do next to extract each segment individually? – Julien M Dec 28 '19 at 09:58
  • @Meisam No I've not tried the hough transform, how do you think it could help? It won't return separated lines as far as I know – Julien M Dec 28 '19 at 10:52
  • By setting Hough's thresholds suitably, it gives you an estimate of each line's equation separately (even they are intersected). By that estimation and some simple morphological operation you can segment them independent from each other. – MeiH Dec 28 '19 at 11:37
  • @Meisam would you apply it on the skeleton or the original image? My segments are not perfectly straight lines – Julien M Dec 28 '19 at 15:45
  • On the original one. Of course because of the curves in the lines, the transform may represent each of the desired lines with multiple lines. In this case (which you have the slopes of all lines) you can classify and merge the desired lines based on their slopes' differences. – MeiH Dec 28 '19 at 18:20
  • I guess you can improve your skeletons if you move the points to a local maximum of the distance transform of the curves (perpendicular to the skeleton direction?). – Micka Apr 18 '20 at 21:24
  • for houghLines I would recommend houghLinesP with a minimum length of 2x the maximum of the distanceTransform. This could be nice to find the junctions. – Micka Apr 18 '20 at 21:38

1 Answers1

2

It is a good idea to use a skeleton, as this will make the processing independent of the stroke width. Near a junction, the skeleton is perturbed so that the direction locally changes.

You can consider a skeleton and split at the junction points (connected to more than two independent curves). You will build a graph where the edges are arcs between the junction points. Make sure to preserve connexity.

Then you can estimate the direction of an arc by finding points that are one and two thicknesses away from the junction. You will pair the edges such that the (green) points form the best alignment. You can do this in a greedy way (best fit first), until you exhaust all possible matchings or until the angle is considered too large.

enter image description here

enter image description here

The crossings with a wide angle are challenging because they will form a skeleton with two distant junctions. When matching a curve to another, you might consider matchings with distant curves, provided there is a path in between, such that it fits inside a stripe as large as the strokes.

  • Thanks a lot for your advice. Would you have any tips regarding the structure to store and traverse each curve? Like a way to extract linked list from the skeleton created with OpenCV – Julien M Dec 29 '19 at 17:52
  • @JulienM: this is just a graph. –  Dec 30 '19 at 08:47