6

I would like to write a data converter tool. I need analyze the bitstream in a file to display the 2D cross-sections of a 3D volume.

The dataset I am trying to view can be found here: https://figshare.com/articles/SSOCT_test_dataset_for_OCTproZ/12356705.

It's the file titled: burned_wood_with_tape_1664x512x256_12bit.raw (832 MB)

Would extremely appreciate some direction. Willing to award a bounty if I could get some software to display the dataset as images using a data conversion.

As I'm totally new to this concept, I don't have code to show for this problem. However, here's a little something I tried using inspiration from other questions on SO:

import rawpy
import imageio

path = "Datasets/burned_wood_with_tape_1664x512x256_12bit.raw"
for item in path:
    item_path = path + item
    raw = rawpy.imread(item_path)
    rgb = raw.postprocess()
    rawpy.imshow(rgb)
Raiyan Chowdhury
  • 281
  • 2
  • 20
  • I don't seem to be yielding a response from viewers. Please feel free to ask any questions about the problem if I can make it any more clear. The goal is to view a dataset of 3-dimensional volumes as 2-dimensional images. – Raiyan Chowdhury Nov 04 '20 at 19:54
  • 1
    I don't think is a GPU issue performance per se, could be you are issuing commands your gpu can't support or you are not rendering the data correctly, you are probably getting alpha based data and you are rendering it as colored one. As example try to debug the output content of "raw" vs "rgb" that should give you some clues, most likely "rgb" will be all zeroes – LemonCool Nov 05 '20 at 19:03
  • @LemonCool I'm quite certain it's a GPU issue. I contacted the developers of the software and they said only an NVIDIA graphics card is supported. – Raiyan Chowdhury Nov 05 '20 at 20:30
  • 1
    well then you got your answer, not a performance issue but a compatibly issue, you are using unsupported GPU apis, as best practice should always check for support on initialization – LemonCool Nov 06 '20 at 16:45
  • 1
    @LemonCool I understand the software is not compatible with my GPU. I'm trying to view the cross-sections of the 3D volume manually by analyzing the bit-stream of the file; not using the software. The earlier part of the question is just to provide some motivation (i.e. An inexpensive approach to this open-source software). – Raiyan Chowdhury Nov 06 '20 at 20:56
  • @RaiyanChowdhury Is it possible in your software to disable GPU usage? Just to run on CPU all the image rendering? Also if not possible without GPU can you tell if stored format inside files is difficult or not? Is it possible to visualize these images from file somehow using just CPU? – Arty Nov 07 '20 at 17:06
  • @Arty So the software requires a CUDA-compatible graphics card such as NVIDIA. From what I've heard from the developers this is unlikely to work, as the source code includes CUDA software. – Raiyan Chowdhury Nov 07 '20 at 22:12
  • @RaiyanChowdhury Do you know what is that `.raw` file contains? Meaning what format of images is inside? Because when I just tried to plot 3D image out of this raw file I got something non-meaniningful. Do you know any details how this format is stored inside? And how is this data to be visualized? E.g. where is `x` and `y` and `z` values in this data for 3D plot? – Arty Nov 08 '20 at 08:48
  • @RaiyanChowdhury As I see by description in the RAW file there are something like `A-scan samples`, also `many A-scans per B-scan`, also many `B-scans per volume` and `volumes per buffer`. Definitely it is not X/Y/Z data, so it can't be displayed right away from file as 3D image. It needs to be processed by some mathematical formula to be converted into XYZ. Do you know any details about this all? – Arty Nov 08 '20 at 08:51
  • 1
    @RaiyanChowdhury I see there are many mathematical transformations/steps [described here](https://github.com/spectralcode/OCTproZ/blob/master/processing.md) that are used to convert input RAW data into XYZ image. Definitely it will not be that easy to implement all these steps from scratch in python code. Also it will be very slow I think without GPU. – Arty Nov 08 '20 at 08:53
  • @RaiyanChowdhury Jut for curiosity I decided to do a 3D plot of input data, I [got this image](https://i.stack.imgur.com/lJI5i.png). Basically what I did - I thought RAW data contains just one 3D image inside file in ZYX planes order. I readed in this data, and plotted it into 3D (with some stepping, not to draw each pixel but each 32-th pixel, to do sparse plot). And got image linked before. Definitely this image doesn't contain anything meaniningful like 3D image, just random data. So for sure it needs to be transformed somehow before 3D-drawing. – Arty Nov 08 '20 at 09:00
  • @RaiyanChowdhury If you want to play with 3D visualization that I got you may [take my code here](https://cutt.ly/AgKc1eW) - just copy this script to your local machine and run, it needs `burned_wood_with_tape_1664x512x256_12bit.raw` to be in same directory, also one time you need to install next modules by command `python -m pip install numpy matplotlib`. Plot that was drawn is draggable, you may drag it with mouse and it will rotate all XYZ directions. Also in my code change `stepz, stepy, stepx` variables to make more dense or sparse dots on plot. – Arty Nov 08 '20 at 09:12
  • @RaiyanChowdhury I just got something more meaningful, [I got this image](https://i.stack.imgur.com/I8Dm1.png). Basically what I did I tried to draw data of sizeX 1664 and sizeY 512 (these are numbers from RAW file name) and value (two bytes unsigned int) is considered to be Z coordinate. So I've drawn 3D surface like in linked image. Full [code is here](https://cutt.ly/sgKbS1v). Try running it on your local machine. Plot is draggable by mouse, so you can rotate it in all XYZ directions. Now looks like some meaningful data. Is it at least non-closely something that you want? – Arty Nov 08 '20 at 09:41
  • @RaiyanChowdhury I see [this description image](https://ndownloader.figshare.com/files/22773491/preview/22773491/preview.jpg) that shows how data is organized, basically in my previous comment I drawn just one 2D layer shown in this image. – Arty Nov 08 '20 at 09:48
  • @RaiyanChowdhury I created [animated image](https://i.stack.imgur.com/zZrSY.gif) (gif) out of first 36 frames of rendering 3D data from RAW file. Almost same code as in previous comments but animated version. Looks like some meaningful data, don't know what it means, but looks quite OK. Your comments are needed! Looks like some promising result. – Arty Nov 08 '20 at 10:09
  • @Arty I was looking for 2D cross-sectional images with this question but this is a pretty cool 3D representation. This is going in the right direction but we want to eventually be able to discern the data as a volume or a stack of images. Each value in the .raw file represents the intensity of a pixel, and what I'm trying to view the *planes* of the 3D volume shown in the question. – Raiyan Chowdhury Nov 08 '20 at 15:24
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/224290/discussion-between-arty-and-raiyan-chowdhury). – Arty Nov 08 '20 at 16:02
  • @RaiyanChowdhury I moved our long-comments discussion [to this chat](https://chat.stackoverflow.com/rooms/224290/discussion-between-arty-and-raiyan-chowdhury) please go there to continue! – Arty Nov 08 '20 at 16:03
  • @RaiyanChowdhury Don't know if you're getting notifications about message in [this our chat](https://chat.stackoverflow.com/rooms/224290/discussion-between-arty-and-raiyan-chowdhury) but I was writing messages for you today (new resulting images!). If you're not getting notifications then please go there from time to time to check current progress of work and my replies! – Arty Nov 09 '20 at 06:45

2 Answers2

1

I don't think it's a valid RAW file at all.
If you try this code:

import rawpy
import imageio

path = 'Datasets/burned_wood_with_tape_1664x512x256_12bit.raw'
raw = rawpy.imread(path)
rgb = raw.postprocess()

You will get a following error:

----> 5 raw = rawpy.imread(path)
      6 rgb = raw.postprocess()

~\Anaconda3\envs\py37tf2gpu\lib\site-packages\rawpy\__init__.py in imread(pathOrFile)
     18         d.open_buffer(pathOrFile)
     19     else:
---> 20         d.open_file(pathOrFile)
     21     return d

rawpy\_rawpy.pyx in rawpy._rawpy.RawPy.open_file()

rawpy\_rawpy.pyx in rawpy._rawpy.RawPy.handle_error()

LibRawFileUnsupportedError: b'Unsupported file format or not RAW file'
Karol Żak
  • 2,158
  • 20
  • 24
1

Down below I implemented next visualization.

Example RAW file burned_wood_with_tape_1664x512x256_12bit.raw consists of 1664 samples per A-scan, 512 A-scans per B-scan, 16 B-scans per buffer, 16 buffers per volume, and 2 volumes in this file, each sample is encoded as 2-bytes unsigned integer in little endian order, only 12 higher bits are used, lower 4 bits contain zeros. Samples are centered approximately around 2^15, to be precise data has these stats min 0 max 47648 mean 32757 standard deviation 454.5.

I draw gray images of size 1664 x 512, there are total 16 * 16 * 2 = 512 such images (frames) in a file. I draw animated frames on screen using matplotlib library, also rendering these animation into GIF file. One example of rendered GIF at reduced quality is located after code.

To render/draw images of different resulting resolution you need to change code line with plt.rcParams['figure.figsize'], this fig size contains (widht_in_inches, height_in_inches), by default DPI (dots per inch) equals to 100, meaning that if you want to have resulting GIF of resolution 720x265 then you need to set this figure size to (7.2, 2.65). Also resulting GIF contains animation of a bit smaller resolution because axes and padding is included into resulting figure size.

My next code needs pip modules to be installed one time by command python -m pip install numpy matplotlib.

Try it online!

# Needs: python -m pip install numpy matplotlib
def oct_show(file, *, begin = 0, end = None):
    import os, numpy as np, matplotlib, matplotlib.pyplot as plt, matplotlib.animation
    plt.rcParams['figure.figsize'] = (7.2, 2.65) # (4.8, 1.75) (7.2, 2.65) (9.6, 3.5)
    sizeX, sizeY, cnt, bits = 1664, 512, 16 * 16 * 2, 12
    stepX, stepY = 16, 8
    fps = 5
    try:
        fsize, opened_here = None, False
        if type(file) is str:
            fsize = os.path.getsize(file)
            file, opened_here = open(file, 'rb'), True
        by = (bits + 7) // 8
        if end is None and fsize is not None:
            end = fsize // (sizeX * sizeY * by)
        imgs = []
        file.seek(begin * sizeY * sizeX * by)
        a = file.read((end - begin) * sizeY * sizeX * by)
        a = np.frombuffer(a, dtype = np.uint16)
        a = a.reshape(end - begin, sizeY, sizeX)
        amin, amax, amean, stdd = np.amin(a), np.amax(a), np.mean(a), np.std(a)
        print('min', amin, 'max', amax, 'mean', round(amean, 1), 'std_dev', round(stdd, 3))
        a = (a.astype(np.float32) - amean) / stdd
        a = np.maximum(0.1, np.minimum(a * 128 + 128.5, 255.1)).astype(np.uint8)
        a = a[:, :, :, None].repeat(3, axis = -1)
        
        fig, ax = plt.subplots()
        plt.subplots_adjust(left = 0.08, right = 0.99, bottom = 0.06, top = 0.97)
        for i in range(a.shape[0]):
            title = ax.text(
                0.5, 1.02, f'Frame {i}',
                size = plt.rcParams['axes.titlesize'],
                ha = 'center', transform = ax.transAxes,
            )
            imgs.append([ax.imshow(a[i], interpolation = 'antialiased'), title])
        ani = matplotlib.animation.ArtistAnimation(plt.gcf(), imgs, interval = 1000 // fps)
        print('Saving animated frames to GIF...', flush = True)
        ani.save(file.name + '.gif', writer = 'imagemagick', fps = fps)
        print('Showing animated frames on screen...', flush = True)
        plt.show()
    finally:
        if opened_here:
            file.close()

oct_show('burned_wood_with_tape_1664x512x256_12bit.raw')

Example output GIF:

gif

Arty
  • 14,883
  • 6
  • 36
  • 69