Fragmented line fitting

Question

I want to fit a line to line fragments, i.e. a small number (often less than 10) of line segments that approximately belong to the line. The line has a small slope. But there are outliers: segments (usually smaller) outside the line. The figure below shows a typical case. There is no horizontal overlap between the pieces.

I would prefer to avoid trying a fit on all subsets of segments and keeping the best. I also wouldn't rely on RANSAC as the sample is too small.

Any suggestion ?

Update:

I now plan to recast the problem as that of fitting a line on points, namely the infinities of points on the individual line segments, assuming a constant linear density. By rewriting the least squares equations in integral form, one sees that we can consider the segments as concentrated at their middle, with a weight equal to their length; there is also an extra term taking their slope into account. This gives a good grounding to the fitting on segments.

Now I still have to incorporate outlier detection. Inspired by RANSAC, I can pick the longest segments and use them in isolation or in pairs to get candidate lines. For each line, evaluate the total error, and keep the line giving the smallest value. From there, some criterion (yet to be found) should allow rejecting the outliers and performing the final least-squares fit on the inliers.

segmentize each line at shortest line's 1/10th of a length; least squares best fit; remove the worst outlier *line*; repeat. for each iteration calculate [the sigma](https://en.wikipedia.org/wiki/Standard_deviation) overall, see sharp drop, stop. or, at each step, for each line calculate its Hausdorff distance from the best fit line, remove the farthest line, repeat; stop when the distance is smaller than some preset value // drops significantly in magnitude. — Will Ness, Apr 21 '16 at 15:25
@WillNess: Mh, if the initial fit is made with all segments, the least squares fit will perform poorly and can be much influenced by outliers. Then there is a risk that the inliers be rejected. Anyway, the idea of removing the outliers one by one is interesting. — , Apr 21 '16 at 15:30
Why? you said the outliers are short. segmenting all at same length beefs up the influence of the closer longer lines. — Will Ness, Apr 21 '16 at 15:31
@WillNess: that's the weakness of least squares: the errors are squared, giving undue high emphasis to the outliers. A single point can ruin an otherwise excellent fit. — , Apr 21 '16 at 15:33
@WillNess: http://demonstrations.wolfram.com/ComparingLeastSquaresFitAndLeastAbsoluteDeviationsFit/ (Check the bottom left snapshot). — , Apr 21 '16 at 15:40
but your case is the bottom middle. and besides, use the least absolute deviations then, it does seem a better fit. — Will Ness, Apr 21 '16 at 16:02
I would dream of using absolute deviations, but the fit is computationally much more expensive as the equations aren't linear. I should have stated that in the post. You may argue that my problem is tiny, but it repeats hundred times in a fraction of a second. — , Apr 21 '16 at 17:19
either way, the shorter outliers will be detected first (as in the bottom middle snapshot); then the best fit is recalculated, and becomes closer to the real one. -- another way is to *randomly* sample (say, 50?) pairs of points (after segmentation, so there are much more points on the good lines, still), throw away short pairs, find the pairs' direction, throw away the outliers, find average, using this direction as X axis find the average Y; discard lines that are the farthest from this Y; repeat until max-delta-Y << max-delta-X (among the pairs), then use this Y=const line as final fit. — Will Ness, Apr 21 '16 at 19:19
D.K. Prasad et. al propose in their [paper](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6166585), a **parameter-free** algorithm to approximate a curve into a set of line segments. fig. 1 of the same reference illustrates a summary of their method. — yuri, Sep 14 '22 at 11:09
@yuri: interesting, I had not investigated on the idea that the line is digital (which mine are). Thanks. — , Sep 15 '22 at 08:11

Tatarize · Answer 1 · 2016-04-17T22:38:37.540

1

I'd guess the slope is going to be around the average of the line fragment slopes times a factor equal to the length of the fragment (or square of the length of the fragment, depending on how that length of the outlying fragment compares). And then bestfit that line with that slope.

So take the line fragments, convert the slopes to angles (arctan2(y1-y0,x1-x0)) multiply that by the length add them all up, divide by (total length of all fragments). Do the same thing for the position (position of midpoint of the line fragment * length of fragment) / (total length of all fragments), then make sure the line with that slope intercepts the point with that value.

Update:

If we are not to consider much about the slopes we should rather just bestfit the line positionally with regard to the impact of the various segments which again we weight by their length.

Find the total length of the fragments. Iterate the fragments until you are 1/3rd of the way through the total length of the fragments. That is going to be your x of your first point. Then pick some arbitrarily small value and iterate through fragments again, sampling at your chosen rate. Then the impact of that sample is the given y multiplied by the linear distance of the x from the x 1/3rd of the way through the total fragments all normalized by the total sum of the linear distances across the all the fragments. Do the same for 2/3rds of the way. And draw a line between the two resultant points.

edited Apr 17 '16 at 22:38

answered Apr 17 '16 at 21:29

Tatarize

10,238
4
58
64

This reminds of the Theil-Sen estimator for the slope. It is more robust to take the median instead of the average, but that median should be somehow "weighted" by the segment lengths. – Apr 17 '16 at 21:37
1

Another consideration is that the global slope shouldn't depend on the individual slopes alone but also on the locations of the segments: assume that all segments are vertically centered on the same horizontal but all with the same nonzero slope: the fitted line should be horizontal anyway. – Apr 17 '16 at 21:49
Hm. The test case of horizontally fitted line segments all with a slope of 2. Would, on my suggestion slap a line with that slope through the centermost bit there. Though a lot of this seems rather throw stuff at the wall and figure out what works best. – Tatarize Apr 17 '16 at 22:16
Doing the entire graph might even be overkill. You should try just finding the average first half of the fragments weighted with the segment lengths and the second half of the fragments weighted with the segment lengths. Making sure to call the point exactly half way through the total length, a dividing point between two fragments. – Tatarize Apr 17 '16 at 22:43
that's incorrect. imagine a bunch of 1-length lines at 10-degrees slope angle each, start points at y=0, x={0,1,...,100}. the best fit line has slope 0. – Will Ness Apr 21 '16 at 15:22
Two comments up you'll see I came up with that same objection when Yves noted it. – Tatarize Apr 22 '16 at 06:33

score 1 · Answer 2 · edited May 23 '17 at 11:52

As you have asked, I have some suggestions. Complete and working answer would be a bit too much for me to arrive at. My suggestion contains two major parts. Taking them one by one:

Handling outliers:

One suggestion for getting rid of the outliers is to Cluster the line segments. Then on, don't worry about the lines that fall outside the cluster. But, how do you cluster the lines? Divide the entire 2D plane into y = 0 to a, y = a to 2a, y = 2a to 3a etc. Line segments which fall in the same y = i to j stripe would be the one you will use to generate your i and j values for the correct stripe.

There is however one issue: What if the line segments are not well divided horizontally? What if majority lines are inclined at 38 degrees instead of being close to 0? In that case, you may do a Principle Component Analysis. Sorry to link you to such an open-ended idea - your question kinda demands it.

Realign your lines so that they are majorly parallel to X-axis and then, as I mentioned above, find the stripe that contains most of the lines.

Approximating the best fitting line:

Now, after you have finalized the correct stripe, take all the line segments that fell in the stripe and densify them. Densification is the step of approximating the line segments as a collection of points. Since all of these line segments are between y = i and y = j, therefore, you may start with the line y = (i + j) / 2 as the best fit line. Then:

Find the distance of all the points from this line, keeping the distance as negative when the point is above the line and the distance as positive when the point is below the line.
Sum all the distances. Let's call this summed value are approximationError.
Your target is now to find that y value for which approximationError is 0.
Decrease y if majority points lied below the line, increase it if majority points lied above it.
You will finally arrive at a line like y = c.
Now, incline this line to the same angle by which you changed all your input line segments during the Principle Component Analysis step.
To get the line segment, cut this line by x-value of the two x-farthest points in the stripe.

I realize that this all may not be easy to visualize. Here is a link to the wikipedia image for PCA. Here is a link to another answer demonstrating line densification.

It indeed seems useful to start by estimating the slope. PCA could be an approach, but PCA on what ? Endpoints weighted by the segment length ? There is a risk that the PCA itself suffers from outliers. (In this context, PCA can be seen as a total least squares regression, i.e. the initial problem.) Is there a "robust PCA" ? — , Apr 18 '16 at 04:46
@YvesDaoust: For one, you could densify the lines and do PCA on points. This way PCA would be less likely to suffer on outliers because by definition outliers would just be points of few line segments in the entire set of points on all line segments. — displayName, Apr 18 '16 at 11:04
@YvesDaoust: Just curios if the densification of line worked for PCA...? — displayName, Apr 23 '16 at 16:28
@dislayName: I haven't tried anything yet. I plan to use standard LS, which is very similar to PCA, but to do it after tentative outlier rejection. — , Apr 23 '16 at 16:57

Fragmented line fitting

2 Answers2