1

I'm trying for a computer vision project to determine the projection transformation occurring in a football image. I detect the vanishing points, get 2 point matches, and calculate the projection from model field points to image points based on cross ratios. This works really well for almost all points, but for points (which lie behind the camera) the projection goes completely wrong. Do you know why and how I can fix this?

It's based on the article Fast 2D model-to-image registration using vanishing points for sports video analysis and I use this projection function given on the page 3. I tried calculating the result using different methods, too (namely based on intersections), but the result is the same:

enter image description here

There should be a bottom field line, but that one is projected to way out far to the right.

I also tried using decimal to see if it was a negative overflow error, but that wouldn't have made much sense to me, since the same result showed up on Wolfram Alpha with testing.

def Projection(vanpointH, vanpointV, pointmatch2, pointmatch1):
"""

:param vanpointH:
:param vanpointV:
:param pointmatch1:
:param pointmatch2:
:returns function that takes a single modelpoint as input:
"""
X1 = pointmatch1[1]
point1field = pointmatch1[0]
X2 = pointmatch2[1]
point2field = pointmatch2[0]
point1VP = linecalc.calcLineEquation([[point1field[0], point1field[1], vanpointH[0], vanpointH[1], 1]])
point1VP2 = linecalc.calcLineEquation([[point1field[0], point1field[1], vanpointV[0], vanpointV[1], 1]])
point2VP = linecalc.calcLineEquation([[point2field[0], point2field[1], vanpointV[0], vanpointV[1], 1]])
point2VP2 = linecalc.calcLineEquation([[point2field[0], point2field[1], vanpointH[0], vanpointH[1], 1]])
inters = linecalc.calcIntersections([point1VP, point2VP])[0]
inters2 = linecalc.calcIntersections([point1VP2, point2VP2])[0]

def lambdaFcnX(X, inters):
    # This fcn provides the solution of where the point to be projected is, according to the matching,
    # on the line connecting point1 and vanpointH. Based only on that the cross ratio is the same as in the model field
    return (((X[0] - X1[0]) * (inters[1] - point1field[1])) / ((X2[0] - X1[0]) * (inters[1] - vanpointH[1])))

def lambdaFcnX2(X, inters):
    # This fcn provides the solution of where the point to be projected is, according to the matching,
    # on the line connecting point2 and vanpointH, Based only on that the cross ratio is the same as in the model field
    return (((X[0] - X1[0]) * (point2field[1] - inters[1])) / ((X2[0] - X1[0]) * (point2field[1] - vanpointH[1])))

def lambdaFcnY(X, v1, v2):
    # return (((X[1] - X1[1]) * (np.subtract(v2,v1))) / ((X2[1] - X1[1]) * (np.subtract(v2, vanpointV))))
    return (((X[1] - X1[1]) * (v2[0] - v1[0])) / ((X2[1] - X1[1]) * (v2[0] - vanpointV[0])))

def projection(Point):
    lambdaPointx = lambdaFcnX(Point, inters)
    lambdaPointx2 = lambdaFcnX2(Point, inters2)

    v1 = (np.multiply(-(lambdaPointx / (1 - lambdaPointx)), vanpointH) + np.multiply((1 / (1 - lambdaPointx)),
                                                                                     point1field))

    v2 = (np.multiply(-(lambdaPointx2 / (1 - lambdaPointx2)), vanpointH) + np.multiply((1 / (1 - lambdaPointx2)),
                                                                                       inters2))

    lambdaPointy = lambdaFcnY(Point, v1, v2)

    point = np.multiply(-(lambdaPointy / (1 - lambdaPointy)), vanpointV) + np.multiply((1 / (1 - lambdaPointy)), v1)
    return point

return projection

match1 = ((650,390,1),(2478,615,1))
match2 = ((740,795,1),(2114,1284,1))

vanpoint1 = [-2.07526585e+03, -5.07454315e+02,  1.00000000e+00]
vanpoint2 = [ 5.53599881e+03, -2.08240612e+02,  1.00000000e+00]

model = Projection(vanpoint2,vanpoint1,match2,match1)
model((110,1597))

Suppose the vanishing points are

vanpoint1 = [-2.07526585e+03, -5.07454315e+02,  1.00000000e+00]
vanpoint2 = [ 5.53599881e+03, -2.08240612e+02,  1.00000000e+00]

and two matches are:

match1 = ((650,390,1),(2478,615,1))
match2 = ((740,795,1),(2114,1284,1))

These work for almost all points as seen in the picture. The left bottom point, however, is completely off and gets image coordinates [ 4.36108177e+04, -1.13418258e+04] This happens going down from (312,1597); for (312,1597) the result is [-2.34989787e+08, 6.87155603e+07] which is where it's supposed to be.

Why does it shift all the way to 4000? It would make sense perhaps if I calculated the camera matrix and then the point was behind the camera. But since what I do is actually similar to homography estimation (2D mapping) I cannot make geometric sense of this. However, my knowledge of this is definitely limited.

Edit: does this perhaps have to do with the topology of the projective plane and that it's non orientable (wraps around)? My knowledge of topology is not what it should be...

Jacco.Oosterhuis
  • 43
  • 5

1 Answers1

1

Okay, figured it out. This might not make too much sense to others, but it does for me (and if anyone ever has the same problem...)

Geometrically, I realized the following when using an equivalent approach, where v1 and v2 are calculated based on the different vanishing points and I project based on the intersection of the lines connecting points with the vanishing points. Here at some point, these lines become parallel, and after that the intersection actually lies completely on the other side. And that makes sense; it just took me a while to realize it does.

In the code above, the last cross ratio, called lambdapointy, goes to 1 and after that above. Here the same thing happens, but it was easiest to visualize based on the intersections.

Also know how to solve it; this is just in case anyone else tries such code.

Jacco.Oosterhuis
  • 43
  • 5