2

I have images from a SICK Trispector depth laser scanner. The image format is PNG. SICK calls it Trispector 2.5D PNG. The images contain both reflection data and depth data according to SICK's documentation. But SICK will not provide information on how to use this data without using their or partners' software. Essentially, what I need is the depth data. Reflection data might be a nice to have but is not necessary. The resulting image I get is monochrome. It seems to have the reflection data in the top part of the image and overlapping height data in the bottom. The scanned object is a crate of beer bottles with bottle caps. You can see an example here:

Scan image

I have tried opening the image in many different image viewers and looked for information on 2.5D, but it does not appear to be relevant for this. In Matlab image preview I get one side of the height data individually, but I don't know how to use this information. See the following image from Matlab preview:

Matlab preview of a scan image

Does anyone know how to extrapolate the height data from an image like this? Maybe someone's worked with SICK's SOPAS or SICK scanners before, and understand this "2.5D PNG" format that SICK calls it. An OpenCV solution would be nice.

Edit: As @DanMašek comments, the problem is that of separating two images of different bitdepth from a single PNG. He provides further insight in the problem and a great OpenCV solution for separating the intensity and depth images as 8- and 16-bit, respectively:

Correctly separated intensity and depth images using @DanMašek's approach

Lars Lort
  • 23
  • 5
  • I would tell them that you are looking for a new scanner vendor that actually lets you understand the data the scanner provides. – Jonathon Reinhart Mar 25 '20 at 12:14
  • @JonathonReinhart I agree, but I believe they market this scanner mostly towards quite simple applications that do not need to be heavily integrated with other processes. The support guy is nice and helpful, but they do not "interfere with how customers handle the image". The university bought the scanner and they can be quite careless with what products they spend their money on. – Lars Lort Mar 25 '20 at 12:22
  • This quetion is probably off topic for Stack Overflow, but I feel your pain. I will note that the image you provided is definitely strange. It looks different in my browser than it does on your Matlab screenshot, and in my phone's image gallery. I would try looking at it in different image viewer software like GIMP or a tool that dumps out information about PNG layers, etc. to try and understand what they're doing. – Jonathon Reinhart Mar 25 '20 at 12:25
  • 1
    From [support portal](https://supportportal.sick.com/faq/trispector-faq/): "This is normal. The .png image of the TriSpector is split into two parts, the 8 bit intensity image on top and the 16 bit heightmap on bottom. Since the .png image is 8 bit, the heightmap needs two pixels in the png image for each real pixel. Thus it becomes twice as large and looks corrupted. If you want to look at the image, load it using a TriSpector emulator." | There's a bit more, but let me finish my lunch first :) – Dan Mašek Mar 25 '20 at 13:02
  • 1
    Here's a simple Python script demonstrating how to separate the two components from the PNG: https://pastebin.com/Nw7zmPhy – Dan Mašek Mar 25 '20 at 13:48
  • 1
    Voting to reopen this question -- IMHO the problem statement is complete, it's just about a peculiarity of a somewhat obscure bit of hardware. I've got an illustrated answer prepared (based on official information from SICK) along with proof of concept in Python (see above -- don't have Matlab available). – Dan Mašek Mar 25 '20 at 14:55
  • 1
    @DanMašek I agree, you have provided a great solution and formulated that my problem was more related to separating two bit-maps of size 8 and 16 from a single PNG. If I can mark your question as an answer I will. I am obviously new here, so let me know what I can do. Thanks! – Lars Lort Mar 25 '20 at 16:31
  • Thanks Lars. The reopen request came through, so there's the answer. Glad I could help -- haven't used this sensor before, but I've been working with the LMS and Ranger devices for well over a decade now.| If you don't have access to the support site yet, it might be worth getting it - there is some additional reference available that you don't find on the public site, as well as forums frequented by SICK employees (although i haven't managed to extract much useful info from them so far, TBH). – Dan Mašek Mar 26 '20 at 01:09

1 Answers1

2

Note: This information is based on the SICK TriSpector FAQ located on the SICK support forums (not publicly accessible, but you can request access).

The PNG images generated by SICK TriSpector store a concatenation of two pixel buffers:

  • An 8-bit intensity image
  • A 16-bit (little-endian) heightmap image

The resulting PNG image has the same width as each component, and thrice the height (since the PNG is 8-bit, and we have 3 bytes in total for each pixel position).

Let's consider a simple example, where the components have 3 rows and 4 columns. The data stored in the PNG will have the following structure:

Layout of the source PNG and way to split it into the two components

The first step, as illustrated above, is to split the image into the two components. The PNG contains 9 rows, a third of that is 3 rows -- hence rows 0-2 contain the intensity, and the rest is the heightmap. The intensity image is directly usable, heightmap needs some further processing.

If we're on a little-endian architecture and are not concerned about portability, we can take advantage of the in-memory layout and just reinterpret the pixel buffer as 16 bit unsigned integers (in Python numpy.ndarray.view, in C++ create a new Mat wrapping the buffer).

The more flexible, albeit slower method is to combine the parts manually. Reshape the array to have the correct number of rows, then split it up based on odd or even column number (Skip indexing in Python). Convert each sub-array into 16bit unsigned integers, and finally combine them according to the formula LSB + 256 * HSB.

Illustration of splitting the height-map into sub-components

Example script in Python:

import cv2
import numpy as np

img = cv2.imread('combined.png', cv2.IMREAD_GRAYSCALE)
height, width = img.shape

# Determine the actual height of the component images
real_height = height/3

# Extract the top (intensity) part
intensity_img = img[:real_height,...]

# Extract the botton (heightmap) part
# Since this has two values per pixel, also reshape it to have the correct number of rows
heightmap_raw = img[real_height:,...].reshape(real_height,-1)

# The heightmap is 16 bit, two subsequent 8 bit pixels need to be combined
# ABCD -> A+256*B, C+256*D

# Quick but non-portable (needs little-endian architecture)
heightmap_img = heightmap_raw.view(np.uint16)

# Slower but portable
heightmap_a = heightmap_raw[...,::2].astype(np.uint16)
heightmap_b = heightmap_raw[...,1::2].astype(np.uint16)
heightmap_img = heightmap_a + 256 * heightmap_b

# Note: intensity is np.uint8, heightmap is np.uint16

cv2.imwrite('intensity.png', intensity_img)
cv2.imwrite('heightmap.png', heightmap_img)

Extracted intensity image:

Example extracted intensity image

Extracted height-map image (note that the original data was downscaled by factor of 256 while saving this for illustration, loosing a lot of the detail):

Example downscaled heightmap image

Dan Mašek
  • 17,852
  • 6
  • 57
  • 85