I want to compute a normal map N(i,j) from a depth map D(i,j), where (i,j) describes 2D pixel location. A basic case of this problem is discussed here: Calculate surface normals from depth image using neighboring pixels cross product
In my case, however, D(i,j) is projective depth: A 3D object (given by a set of 3D points) is projected onto a 2D image using an intrinsic camera matrix [K|0] (see below for a definition) by multiplying each 3D point in homogeneous coordinates (i.e., a 4D vector [x,y,z,1].T) with [K|0]. In this case, D(i,j) corresponds to the norm of the 3D point whose projection hit pixel (i,j).
The intrinsic camera matrix is given by
[K|0] = |fx 0 ox 0|
| 0 fy oy 0|
| 0 0 1 0|
where fx/fy are focal lengths along the x/y axes and ox/oy are camera offsets along the x/y axes.
Given this setup, what are the normals N(i,j)?
