0

I have a depth map encoded in 24 bits (labeled "Original"). With the code below:

carla_img = cv.imread('carla_deep.png', flags=cv.IMREAD_COLOR)
carla_img = carla_img[:, :, :3]
carla_img = carla_img[:,:,::-1]
gray_depth = ((carla_img[:,:,0] + carla_img[:,:,1] * 256.0 + carla_img[:,:,2] * 256.0 * 256.0)/((256.0 * 256.0 * 256.0) - 1))
gray_depth = gray_depth * 1000

I am able to convert it as in the "Converted" image. As shown here: https://carla.readthedocs.io/en/latest/ref_sensors/ enter image description here

How can I reverse this process (Without using any larger external libraries and using at most openCV)? In Python I create a depth map with the help of OpenCV. I wanted to save the obtained depth map in the form of Carla (24-bit).

This is how I create depth map:

imgL = cv.imread('leftImg.png',0)
imgR = cv.imread('rightImg.png',0)
stereo = cv.StereoBM_create(numDisparities=128, blockSize=17)
disparity = stereo.compute(imgL,imgR)

CameraFOV = 120
Focus_length = width /(2 * math.tan(CameraFOV * math.pi / 360))
camerasBaseline = 0.3
depthMap = (camerasBaseline * Focus_length) / disparity

How can I save the obtained depth map in the same form as in the picture marked "Original"?

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
Pawlit16
  • 123
  • 2
  • 14
  • Disparities maps aren't depth maps. I want to save the depth map as 24 bits. So write the distance data on the R (8-bit), G (8-bit), B (8-bit) channel. To make it come out such an image as marked "Original". Normal depth maps are 8 bit (grayscale only). In the post I am asking about the reverse process I presented from 24-bit to 8-bit. – Pawlit16 Jan 22 '22 at 17:44
  • Ok sorry, it's easier, but I dont know python enough for those channel reordering stuff. Looks like depthmap/1000 and the channel1=depthmap*(256*256*256-1); channel2= ... – Micka Jan 22 '22 at 18:04
  • Depth maps are grayscale. It is only in color in your original because someone has added a color map to code each grayscale with a difference color. You need to know what color map was used to create your original. – fmw42 Jan 22 '22 at 18:42
  • 1
    @fmw42 no that's not a colormap, that is literally 24 bits of depth, reinterpreted as RGB. I looked at the channels. – Christoph Rackwitz Jan 22 '22 at 18:43
  • so the problem is how to do the calculation backwards... that's not too hard. you are only missing the "modulo" operator. one could also do some reinterpretation tricks and spare the math, but that'll assume a certain endianness. – Christoph Rackwitz Jan 22 '22 at 18:44
  • they stick 24 bits of one value into 24 bits, that happens to be three 8-bit values. this post seems to contain screenshots. the images are resampled. -- this is a trivial task so I'm disinclined to answer it. if OP managed to write the code in the question, they should be able to reason backwards too. – Christoph Rackwitz Jan 22 '22 at 18:49
  • OK. I see from https://carla.readthedocs.io/en/latest/ref_sensors/ that the 3 channels are used for higher bit depths to give more resolution – fmw42 Jan 22 '22 at 18:49
  • Is the converted grayscale image 24-bits or 8-bits? If the latter, then the resolution has been lost. If the former, then one should be able to get back to the RGB 8-bit each image. But that would be a question for the CARLA developers or forum – fmw42 Jan 22 '22 at 18:53

1 Answers1

1

Docs say:

normalized = (R + G * 256 + B * 256 * 256) / (256 * 256 * 256 - 1)
in_meters = 1000 * normalized

So if you have a depth map in_meters, you do the reverse, by rearranging the equations.

You need to make sure your depth map (from block matching) is in units of meters. Your calculations there look sensible, assuming your cameras have a baseline of 0.3 meters.

First variant

take the calculation apart, using division and modulo operations.

Various .astype are required to turn floats into integers, and wider integers into narrow integers (assumption for pictures).

normalized = in_meters / 1000
BGR = (normalized * (2**24-1)).astype(np.uint32)
BG,R = np.divmod(BGR, 2**8)
B,G = np.divmod(BG, 2**8)
carla_img = np.dstack([B,G,R]).astype(np.uint8) # BGR order

Second variant

One could also do this with a view, reinterpreting the uint32 data as four uint8 values. This assumes a little endian system, which is a fair assumption but one needs to be aware of it.

...
reinterpreted = BGR.view(np.uint8) # lowest byte first, i.e. order is RGBx
reinterpreted.shape = BGR.shape + (4,) # np.view refuses to add a dimension

carla_img = reinterpreted[:,:,(2,1,0)] # select BGR
# this may require a .copy() to get data without holes (OpenCV may want this)

Disclaimer

I could not test the code in this answer because you haven't provided usable data.

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36