I'm trying to use OpenCV's solvePnPRansac method to locate the camera position around a 3D model loaded in Blender. However, manually putting in points gives me an incorrect t/r vector and cv2.projectPoints gets the image coordinates wrong, so I suspect something about the input is messed up.
I manually selected 10 points on the 3D model in Blender's arbitrary units, and used those as the world coordinates. I found the corresponding locations of those points in the 960x540 rendered image, and those are my input 2D coordinates (by pixels).
The camera's focal length and sensor size are set to 40mm focal and 40x18mm sensor, and since this is mm I manually measured both as approximated their lengths in pixels as about 916 pixels and 412 pixels - however, both these sets of values give me coordinates that are completely off.
The output of projectPoints for me are coordinates that range from -13,000 to 20,000. I think my input values are incorrect but how do I fix this?
size = (960, 540)
image_points = np.array([
(315, 279), # 1
(440, 273), # 2
(407, 386), # 3
(369, 372), # 4
(440, 317), # 5
(457, 373), # 6
(439, 385), # 7
(369, 337), # 8
(407, 358), # 9
(313, 291), # 10
], dtype="double")
model_points = np.array([
(0.77644, -0.63806, 2.55822), # stripe of adidas 1/3 down
(0.75437, 0.49247, 2.75569), # first curve up on the logo
(0.82970, 0.17763, 1.58873), # lower right part of hole in a
(0.82900, -0.16946, 1.68983), # 1/3 down left side of i
(0.84466, 0.52011, 2.37269), # bottom of gold circle left of F
(0.67476, 0.59525, 1.68853), # left side of thick top 's'
(0.74288, 0.46100, 1.58926), # inwards (left) of right below e joint
(0.82560, -0.14537, 2.07217), # middle of f top bar, left of upwards
(0.83161, 0.17382, 1.88820), # middle line of a near bottom of Y
(0.78115, -0.66043, 2.46363) # 1/4 up 2nd d circle in adidas
], dtype = "double")
f = 916.8
sx = 916.8
sy = 412.56
width, height = (size[1], size[0])
camera_matrix = np.array([
[width*f/sx, 0, width/2],
[0, height*f/sy, height/2],
[0, 0, 1]
], dtype = "double")
dist_coeffs = np.zeros((4,1))
s,t,r, i = cv2.solvePnPRansac(model_points, image_points, camera_matrix, dist_coeffs)
img, x = cv2.projectPoints(model_points, r, t, camera_matrix, dist_coeffs)
print(img)
