According to this answer, cv::fitLine
with distType
set to CV_DIST_L2
is a standard least squares method. However, looking at the code it's seems to use a different formula:
origin = (mean(x), mean(y)); // this is as in standard ls
angle = 0.5 * atan2( 2 * cov(x, y), (var(x) - var(y)));
can someone explain me the last formula?
Edit: this is actually a total least squares. For more information read here.