OpenCV: Remap in reverse order

Question

I want to project an image from spherical to cubemap. From what I understood studying maths, I need to create a theta, phi distribution for each pixel and then convert it into cartesian system to get a normalized pixel map.

I used the following code to do so

theta = 0
phi = np.pi/2
squareLength = 2048

# theta phi distribution for X-positive face
t = np.linspace(theta + np.pi/4, theta - np.pi/4, squareLength)
p = np.linspace(phi + np.pi/4, phi - np.pi/4, squareLength)
x, y = np.meshgrid(t, p)

# converting into cartesion sytem for X-positive face (where r is the distance from sphere center to cube plane and X is constantly 0.5 in cartesian system)
X = np.zeros_like(y)
X[:,:] = 0.5
r = X / (np.cos(x) * np.sin(y))
Y = r * np.sin(x) * np.sin(y)
Z = r * np.cos(y)
XYZ = np.stack((X, Y, Z), axis=2)

# shifting pixels from the negative side
XYZ = XYZ + [0, 0.5, 0.5]

# since i want to project on X-positive face my map should be
x_map = -XYZ[:, :, 1] * squareLength
y_map = XYZ[:,:, 2] * squareLength

The above map created should give me my desired result with cv2.remap() but it's not. Then I tried looping through pixels and implement my own remap without interpolation or extrapolation. With some hit and trial, I deduced the following formula which gives me the correct result

for i in range(2048):
    for j in range(2048):
        try:
            image[int(y_map[i,j]), int(x_map[i,j])] = im[i, j]
        except:
            pass

which is reverse of actual cv2 remapping which says dst(x,y)=src(mapx(x,y),mapy(x,y))

I do not understand if did the math all wrong or is there a way to covert x_map and y_map to correct forms so that cv2.remap() gives the desired result.

INPUT IMAGE

DESIRED RESULT (this one is without interpolation using loops)

CURRENT RESULT (using cv2.remap())

Answer 1

I'm quite new in opencv and I didn't work with so difficult math algorithms before but I tried to do this. I rewrote your code a bit and here it is:

import numpy as np
import cv2


src = cv2.imread("data/pink_sq.png")


def make_map():
    theta = 0
    phi = np.pi / 2
    squareLength = 4000

    # theta phi distribution for X-positive face
    t = np.linspace((theta - np.pi / 4), (theta + np.pi / 4), squareLength)
    p = np.linspace((phi + np.pi / 4), (phi - np.pi / 4), squareLength)
    x, y = np.meshgrid(t, p)

    x_res = np.zeros_like(y)
    x_res[:, :] = 0.5
    r = x_res * (np.cos(x))
    r /= np.amax(r)
    y_res = r * x
    z_res = r * np.cos(y)
    xyz = np.stack((x_res, y_res, z_res), axis=2)

    # shifting pixels from the negative side
    xyz = xyz + [0, 0.5, 0.5]

    # since i want to project on X-positive face my map should be
    x_map = xyz[:, :, 1] * squareLength
    y_map = xyz[:, :, 2] * squareLength

    map_x = y_map.astype("float32")
    map_y = x_map.astype("float32")
    return map_x, map_y


map_x, map_y = make_map()
dst = cv2.remap(src, map_y, map_x, cv2.INTER_LINEAR)

cv2.imwrite("res.png", dst)

I don't understand the math in this code at all but I rewrote it a bit and I should say that it works quite good. Here is the result image: And yes, there is a bit difference between my result picture and yours but I hope it is ok :) If I'm not right somewhere of course downvote this answer because I'm not sure that it is correct one.

Answer 2

I'm almost certain the issue has to do with the orientation of the reference frame in space. Maybe if you clarify the Math a bit we can help.