Opening a compressed Dicom (MONOCHROME2) file in Python

Question

I tried to open a Dicom file which is a compressed version (JPEG 2000 Image Compression).

I used different free software tools as well as Python libraries (pydicom, VTK, OpenCV) to open it, but they were unsuccessful
In pydicom, decompress() adapts the transfer syntax of the data set, but not the Photometric Interpretation
It seems GDCM can also handle compressed versions, but there is no "one-click" solution to install GDCM on Windows [Solved!]

I would appreciate any suggestions on this.

Dicom Tags:

(0008, 0016) SOP Class UID                       UI: 
(0008, 0018) SOP Instance UID                    UI: 1.2.276.0.75.2.2.42.114374073191699.20160725100733359.3844234830
(0008, 0020) Study Date                          DA: ''
(0008, 0021) Series Date                         DA: ''
(0008, 0022) Acquisition Date                    DA: ''
(0008, 002a) Acquisition DateTime                DT: ''
(0008, 0030) Study Time                          TM: '100304'
(0008, 0031) Series Time                         TM: '100733'
(0008, 0032) Acquisition Time                    TM: '100733'
(0008, 0050) Accession Number                    SH: ''
(0008, 0060) Modality                            CS: ''
(0008, 0070) Manufacturer                        LO: ''
(0008, 0080) Institution Name                    LO: ''
(0008, 1010) Station Name                        SH: ''
(0008, 1030) Study Description                   LO: ''
(0008, 103e) Series Description                  LO: 'Model Acquistion'
(0008, 1070) Operators' Name                     PN: ''
(0008, 1072)  Operator Identification Sequence   1 item(s) ---- 
   (0008, 0080) Institution Name                    LO: ''
   ---------
(0008, 1090) Manufacturer's Model Name           LO: '4000'
(0010, 0010) Patient's Name                      PN: ''
(0010, 0020) Patient ID                          LO: ''
(0010, 0021) Issuer of Patient ID                LO: ''
(0010, 0030) Patient's Birth Date                DA: ''
(0010, 0040) Patient's Sex                       CS: ''
(0010, 4000) Patient Comments                    LT: ''
(0018, 0088) Spacing Between Slices              DS: "0.0"
(0018, 1000) Device Serial Number                LO: '11307'
(0018, 1020) Software Versions                   LO: '6.5.0.772'
(0018, 1030) Protocol Name                       LO: ''
(0020, 000d) Study Instance UID                  UI: 1.2.276.0.75.2.2.42.114374073191699.20160725100304296.3844877810
(0020, 000e) Series Instance UID                 UI: 1.2.276.0.75.2.2.42.114374073191699.20160725100733328.3844960960
(0020, 0010) Study ID                            SH: '2016072510030428'
(0020, 0011) Series Number                       IS: "-2147483648"
(0020, 0012) Acquisition Number                  IS: "0"
(0020, 0013) Instance Number                     IS: "1"
(0020, 0052) Frame of Reference UID              UI: 1.2.276.0.75.2.2.42.114374073191699.20160725100733328.3844960960
(0020, 0060) Laterality                          CS: 'OS'
(0020, 0200) Synchronization Frame of Reference  UI: 1.2.276.0.75.2.2.42.114374073191699.20140417175345187.1980128920
(0020, 4000) Image Comments                      LT: ''
(0028, 0002) Samples per Pixel                   US: 1
(0028, 0004) Photometric Interpretation          CS: 'MONOCHROME2'
(0028, 0008) Number of Frames                    IS: "2"
(0028, 0010) Rows                                US: 1024
(0028, 0011) Columns                             US: 1024
(0028, 0030) Pixel Spacing                       DS: '0.005865103,0.001955034'
(0028, 0100) Bits Allocated                      US: 8
(0028, 0101) Bits Stored                         US: 8
(0028, 0102) High Bit                            US: 7
(0028, 0103) Pixel Representation                US: 1
(0028, 1050) Window Center                       DS: "29.0"
(0028, 1051) Window Width                        DS: "210.0"
(0028, 2110) Lossy Image Compression             CS: '01'
(0028, 2112) Lossy Image Compression Ratio       DS: "10.0"
(0032, 1060) Requested Procedure Description     LO: ''
(0040, 0008)  Scheduled Protocol Code Sequence   1 item(s) ---- 
   (0008, 0100) Code Value                          SH: 'SD-E1'
   (0008, 0102) Coding Scheme Designator            SH: '99CZM'
   (0008, 0103) Coding Scheme Version               SH: '1.0'
   (0008, 0104) Code Meaning                        LO: 'ALL SCANS'
   (0008, 010d) Context Group Extension Creator UID UI: CZM
   (0061, 0111) Private tag data                    UL: 1
   (0061, 0113) Private tag data                    LO: '1'
   (0061, 0115) Private tag data                    LO: ''
   (0061, 0117) Private tag data                    LO: 'SD-E1.xml'
   (0061, 0119) Private tag data                    LO: 'False'
   (0061, 011b) Private tag data                    LO: ''
   (0061, 011d) Private tag data                    LO: 'True'
   ---------
(0040, 0244) Performed Procedure Step Start Date DA: ''
(0040, 0245) Performed Procedure Step Start Time TM: '100733'
(0040, 0260)  Performed Protocol Code Sequence   1 item(s) ---- 
   (0008, 0100) Code Value                          SH: 'SD-S2'
   (0008, 0102) Coding Scheme Designator            SH: ''
   (0008, 0103) Coding Scheme Version               SH: '1.0'
   (0008, 0104) Code Meaning                        LO: 'Macular Cube 512x128'
   (0008, 010d) Context Group Extension Creator UID UI: CZM
   (0061, 0111) Private tag data                    UL: 2
   (0061, 0113) Private tag data                    LO: '1'
   (0061, 0115) Private tag data                    LO: ''
   (0061, 0117) Private tag data                    LO: 'SD-S2.xml'
   (0061, 0119) Private tag data                    LO: 'False'
   (0061, 011b) Private tag data                    LO: 'No Hires in center.No parameters can be modified.'
   (0061, 011d) Private tag data                    LO: 'False'
   ---------
(0040, 1001) Requested Procedure ID              SH: ''
(0057, 0001) Private Creator                     UI: 1.2.276.0.75.2.2.42.7
(0057, 1003) Private tag data                    UL: 1
(0057, 1015) Private tag data                    LO: 'CZMI'
(0057, 1021) Private tag data                    LO: ''
(0057, 1023) Private tag data                    LO: '116525681374'
(0059, 1000) Private tag data                    LO: 'DATAFILES/E039/3RT8Q85TM1Y7VT7X31WNLNBV99GA43BOJ27IOLS4X2ZU.EX.DCM'
(0059, 1005) Private tag data                    SL: 0
(0059, 3500) Private tag data                    SL: 1
(0063, 1000) Private tag data                    FL: 0.0
(0063, 1005) Private tag data                    FL: 6.0
(0063, 1010) Private tag data                    FL: 2.0
(0063, 1015) Private tag data                    FL: -64.0
(0063, 1020) Private tag data                    UL: 141
(0063, 1025) Private tag data                    FL: 0.0
(0063, 1026) Private tag data                    FL: 0.0
(0063, 1030) Private tag data                    FL: 1.0
(0063, 1032) Private tag data                    FL: 1.0
(0063, 1035) Private tag data                    SL: 113
(0063, 1047) Private tag data                    FL: 297.0
(0063, 1048) Private tag data                    FL: 872.0
(0063, 1049) Private tag data                    FL: 1222.0
(0071, 1070) Private tag data                    FL: -292.0
(0071, 1095) Private tag data                    FL: 0.0
(0071, 1100) Private tag data                    FL: 0.0
(0071, 1105) Private tag data                    SL: 0
(0073, 1085) Private tag data                    FL: 1.0
(0073, 1090) Private tag data                    SL: 0
(0073, 1095) Private tag data                    SL: 0
(0073, 1100) Private tag data                    SL: 0
(0073, 1105) Private tag data                    FL: 0.0
(0073, 1110) Private tag data                    FL: 0.0
(0073, 1125) Private tag data                    SL: Array of 128 elements
(0073, 1135) Private tag data                    FL: 0.6000000238418579
(0073, 1200) Private tag data                    SL: Array of 128 elements
(0075, 1015)  Private tag data   0 item(s) ---- 
(0075, 1020) Private tag data                    SL: 0
(0075, 1021) Private tag data                    SL: 0
(0075, 1035) Private tag data                    FL: 3.5299999713897705
(0075, 1065) Private tag data                    FL: 0.0
(0075, 1070) Private tag data                    FL: 0.0
(0075, 1075) Private tag data                    FL: 0.0
(0075, 1080) Private tag data                    FL: 0.0
(0075, 1085) Private tag data                    SL: 0
(0075, 1210) Private tag data                    UL: 0
(0075, 1215) Private tag data                    FL: -inf
(0075, 1220) Private tag data                    FL: -inf
(7fe0, 0010) Pixel Data                          OB: Array of 211106 elements ```

I saw many codes that a combination of *Slice Thickness*, *Pixel Spacing* and *Pixel Array* is used to plot a Dicom file. In this file, there is no *Slice Thickness*. — Amin, Feb 03 '20 at 17:57
Just to understand that: the DICOM dump above is the decompressed file? It does not show the SOP Class UID, so I'm not sure what it actually is, and it is not compressed. A JPEG2000 compressed file cannot be MONOCHROME2, so maybe you can elaborate a bit more what you did, and what kind of data you have. — MrBean Bremen, Feb 04 '20 at 10:31
Based on the value of Transfer Syntax: `dcmread("XXXX.dcm").file_meta.TransferSyntaxUID` =1.2.840.10008.1.2.4.91, JPEG 2000 Image Compression was used ([DICOM Transfer Syntaxes](https://www.dicomlibrary.com/dicom/transfer-syntax/)). [Table 8.2.4-1](http://dicom.nema.org/medical/dicom/2017e/output/chtml/part05/sect_8.2.4.html) shows this compressed type can be MONOCHROME2. — Amin, Feb 04 '20 at 17:20
Ah, you are right - my memory has failed me... I'm not sure what the problem is, but if you have the possibility to post an anonymized data set, you could open an issue in [pydicom](https://github.com/pydicom/pydicom) - the chances are higher to get a meaningful response there. — MrBean Bremen, Feb 04 '20 at 18:12
I think you are trying to open Zeiss OCT DCM, this format is a internal and private one that not open to outside users. You may need ask Zeiss engineers help to transform it into nomral dicoms with their Forum PACS. — chuanfeng Lv, Feb 23 '20 at 02:09

score 2 · Answer 1 · edited Apr 06 '22 at 14:15

I've determined how to unscramble these obfuscated CZM DICOM datasets. Essentially CZM transposes three regions of the JPEG 2000 data and then overwrites those by XORing every 7th byte with 0x5A, which is pretty nasty.

The following function should reverse this, producing a normal JPEG 2000 data stream that can then be written to file or opened with Pillow:

import math


def unscramble_czm(frame: bytes) -> bytearray:
    """Return an unscrambled image frame.

    Parameters
    ----------
    frame : bytes
        The scrambled CZM JPEG 2000 data frame as found in the DICOM dataset.

    Returns
    -------
    bytearray
        The unscrambled JPEG 2000 data.
    """
    # Fix the 0x5A XORing
    frame = bytearray(frame)
    for ii in range(0, len(frame), 7):
        frame[ii] = frame[ii] ^ 0x5A

    # Offset to the start of the JP2 header - empirically determined
    jp2_offset = math.floor(len(frame) / 5 * 3)

    # Double check that our empirically determined jp2_offset is correct
    offset = frame.find(b"\x00\x00\x00\x0C")
    if offset == -1:
        raise ValueError("No JP2 header found in the scrambled pixel data")

    try:
        assert jp2_offset == offset
    except AssertionError:
        raise ValueError(
            f"JP2 header found at offset {offset} rather than the expected "
            f"{jp2_offset}"
        )

    d = bytearray()
    d.extend(frame[jp2_offset:jp2_offset + 253])
    d.extend(frame[993:1016])
    d.extend(frame[276:763])
    d.extend(frame[23:276])
    d.extend(frame[1016:jp2_offset])
    d.extend(frame[:23])
    d.extend(frame[763:993])
    d.extend(frame[jp2_offset + 253:])

    assert len(d) == len(frame)

    return d

Opening a compressed Dicom (MONOCHROME2) file in Python

1 Answers1