zipfile and shutil not keeping the file exif data

Question

I have a .zip archive and within it a single .dng image file with metadata about exposure time, etc. When unpacking this achive using either shutil.unpack_archive or ZipFile.extractall(), the .dng is extracted but the metadata is gone.

if file_ext == '.zip':
    with tempfile.TemporaryDirectory() as local_dir_unzip:
        with zipfile.ZipFile(local_file.name, 'r') as zip_file:
            zip_file.extractall(local_dir_unzip)
        unpacked_files = os.listdir(local_dir_unzip)
        dng_file_name = [f for f in unpacked_files if f.endswith('.dng')][0]

This file has no metadata. But if I unpack this archive using macOS's archive utility the extracted file has all the metadata.

EDIT: Using exiftool I was able to see that in fact the unpacked file did have all the EXIF information, but for some reason it is inaccessible to both the OS's get info function and to the exifread python package:

[ExifTool]      ExifToolVersion                 : 12.30
[ExifTool]      Now                             : 2022:04:05 19:59:38+03:00
[ExifTool]      NewGUID                         : 20220405-1959-3800-71AE-CDAFBE855D39
[ExifTool]      FileSequence                    : 0
[ExifTool]      ProcessingTime                  : 0.509 s
[File]          FileName                        : 20220404170458.dng
[File]          BaseName                        : 20220404170458
[File]          Directory                       : .
[File]          FilePath                        : /private/var/folders/ng/jsgvdn7s4w3gffffmfwmx9s80000gn/T/tmpbta5zv01/20220404170458.dng
[File]          FileSize                        : 10 MiB
[File]          FileModifyDate                  : 2022:04:05 19:58:56+03:00
[File]          FileAccessDate                  : 2022:04:05 19:58:56+03:00
[File]          FileInodeChangeDate             : 2022:04:05 19:58:56+03:00
[File]          FilePermissions                 : -rw-r--r--
[File]          FileAttributes                  : Regular; (none)
[File]          FileDeviceNumber                : 16777221
[File]          FileInodeNumber                 : 28568573
[File]          FileHardLinks                   : 1
[File]          FileUserID                      : yarden
[File]          FileGroupID                     : staff
[File]          FileDeviceID                    : 0.0
[File]          FileBlockSize                   : 4096
[File]          FileBlockCount                  : 20968
[File]          FileCreateDate                  : 2022:04:05 19:58:56+03:00
[File]          MDItemFSContentChangeDate       : 2022:04:05 19:58:56+03:00
[File]          MDItemFSCreationDate            : 2022:04:05 19:58:56+03:00
[File]          MDItemFSCreatorCode             :
[File]          MDItemFSFinderFlags             : 0
[File]          MDItemFSHasCustomIcon           : 0
[File]          MDItemFSInvisible               : 0
[File]          MDItemFSIsExtensionHidden       : 0
[File]          MDItemFSIsStationery            : 0
[File]          MDItemFSLabel                   : 0 (none)
[File]          MDItemFSName                    : 20220404170458.dng
[File]          MDItemFSNodeCount               : 10733045
[File]          MDItemFSOwnerGroupID            : 20
[File]          MDItemFSOwnerUserID             : 501
[File]          MDItemFSSize                    : 10733045
[File]          MDItemFSTypeCode                :
[File]          FileType                        : DNG
[File]          FileTypeExtension               : dng
[File]          MIMEType                        : image/x-adobe-dng
[File]          ExifByteOrder                   : Big-endian (Motorola, MM)
[EXIF]          SubfileType                     : Full-resolution image
[EXIF]          ImageWidth                      : 4224
[EXIF]          ImageHeight                     : 3024
[EXIF]          BitsPerSample                   : 16
[EXIF]          Compression                     : JPEG
[EXIF]          PhotometricInterpretation       : Color Filter Array
[EXIF]          Make                            : Apple
[EXIF]          Model                           : iPhone 13 mini
[EXIF]          Orientation                     : Horizontal (normal)
[EXIF]          SamplesPerPixel                 : 1
[EXIF]          PlanarConfiguration             : Chunky
[EXIF]          Software                        : 15.2.1
[EXIF]          ModifyDate                      : 2022:04:04 17:04:58
[EXIF]          TileWidth                       : 264
[EXIF]          TileLength                      : 378
[EXIF]          TileOffsets                     : (Binary data 1017 bytes, use -b option to extract)
[EXIF]          TileByteCounts                  : (Binary data 770 bytes, use -b option to extract)
[EXIF]          CFARepeatPatternDim             : 2 2
[EXIF]          CFAPattern2                     : 0 1 1 2
[EXIF]          ExposureTime                    : 1/120
[EXIF]          FNumber                         : 2.4
[EXIF]          ExposureProgram                 : Manual
[EXIF]          ISO                             : 100
[EXIF]          ExifVersion                     : 0232
[EXIF]          DateTimeOriginal                : 2022:04:04 17:04:58
[EXIF]          CreateDate                      : 2022:04:04 17:04:58
[EXIF]          OffsetTime                      : +03:00
[EXIF]          OffsetTimeOriginal              : +03:00
[EXIF]          OffsetTimeDigitized             : +03:00
[EXIF]          ShutterSpeedValue               : 1/120
[EXIF]          ApertureValue                   : 2.4
[EXIF]          BrightnessValue                 : 3.266772747
[EXIF]          ExposureCompensation            : 0
[EXIF]          MeteringMode                    : Multi-segment
[EXIF]          Flash                           : Off, Did not fire
[EXIF]          FocalLength                     : 1.5 mm
[EXIF]          SubSecTimeOriginal              : 470
[EXIF]          SubSecTimeDigitized             : 470
[EXIF]          ExifImageWidth                  : 4224
[EXIF]          ExifImageHeight                 : 3024
[EXIF]          SensingMethod                   : One-chip color area
[EXIF]          SceneType                       : Directly photographed
[EXIF]          ExposureMode                    : Manual
[EXIF]          WhiteBalance                    : Auto
[EXIF]          DigitalZoomRatio                : 1.031377899
[EXIF]          FocalLengthIn35mmFormat         : 14 mm
[EXIF]          LensInfo                        : 1.539999962mm f/2.4
[EXIF]          LensMake                        : Apple
[EXIF]          LensModel                       : iPhone 13 mini back camera 1.54mm f/2.4
[EXIF]          DNGVersion                      : 1.6.0.0
[EXIF]          DNGBackwardVersion              : 1.3.0.0
[EXIF]          UniqueCameraModel               : iPhone14,4 back ultra wide camera
[EXIF]          CFAPlaneColor                   : Red,Green,Blue
[EXIF]          BlackLevelRepeatDim             : 1 1
[EXIF]          BlackLevel                      : 528
[EXIF]          WhiteLevel                      : 4095
[EXIF]          DefaultCropOrigin               : 0 0
[EXIF]          DefaultCropSize                 : 4032 3024
[EXIF]          ColorMatrix1                    : 1.200746655 -0.5329620838 -0.2371858805 -0.4524382055 1.501772404 -0.004542022012 -0.03874496371 0.1614724994 0.6470876932
[EXIF]          ColorMatrix2                    : 0.8832943439 -0.3023501933 -0.1193092167 -0.4258989692 1.299629331 0.101080738 -0.1047417223 0.2313982993 0.4722827077
[EXIF]          AsShotNeutral                   : 0.5384514332 1 0.5940536857
[EXIF]          BaselineExposure                : 0.01105523016
[EXIF]          CalibrationIlluminant1          : Standard Light A
[EXIF]          CalibrationIlluminant2          : D65
[EXIF]          ActiveArea                      : 0 96 3024 4128
[EXIF]          NoiseReductionApplied           : undef
[EXIF]          OpcodeList3                     : [opcode 14], FixVignetteRadial
[EXIF]          NoiseProfile                    : 0.00019817037529 2.21563225e-06
[MakerNotes]    RunTimeFlags                    : Valid
[MakerNotes]    RunTimeValue                    : 352844527447041
[MakerNotes]    RunTimeEpoch                    : 0
[MakerNotes]    RunTimeScale                    : 1000000000
[MakerNotes]    AccelerationVector              : -0.9574939019 0.02597124688 0.3083640337
[Composite]     RunTimeSincePowerUp             : 4 days 2:00:45
[Composite]     Aperture                        : 2.4
[Composite]     CFAPattern                      : [Red,Green][Green,Blue]
[Composite]     ImageSize                       : 4224x3024
[Composite]     Megapixels                      : 12.8
[Composite]     ScaleFactor35efl                : 9.1
[Composite]     ShutterSpeed                    : 1/120
[Composite]     SubSecCreateDate                : 2022:04:04 17:04:58.470+03:00
[Composite]     SubSecDateTimeOriginal          : 2022:04:04 17:04:58.470+03:00
[Composite]     SubSecModifyDate                : 2022:04:04 17:04:58+03:00
[Composite]     CircleOfConfusion               : 0.003 mm
[Composite]     FOV                             : 104.3 deg
[Composite]     FocalLength35efl                : 1.5 mm (35 mm equivalent: 14.0 mm)
[Composite]     HyperfocalDistance              : 0.30 m
[Composite]     LightValue                      : 9.4
[Composite]     LensID                          : iPhone 13 mini back camera 1.54mm f/2.4

Does this answer your question? [zipfile - python: how to get meta-data of files inside compressed folder](https://stackoverflow.com/questions/36967551/zipfile-python-how-to-get-meta-data-of-files-inside-compressed-folder) — SM1312, Apr 05 '22 at 08:35
No, my problem is lack of exif data on this file after unpacking, while that post is in regards to reading file metadata (basic data, not exif) before unpacking. — Yarden, Apr 05 '22 at 08:43
How did you determine that there is no metadata in the unzipped DNG please? Also, when you unzip with Python and then with the built-in **macOS** `archive` utility, does the DNG come out at exactly the same size in both cases? And finally, when you extract with the **macOS** tool, did you try `ls -la` to see if there are additional hidden files or `@` signs indicating `xattr`-style data? — Mark Setchell, Apr 05 '22 at 10:22
Thank you for your comments. I've determined there is no metadata both by opening the OS's ```get info``` for the file and by using the ```exifread``` library in python. Both of these methods could identify the exif data in the file extracted using the OS utility. The DNG come out at exactly the same size though (to the byte). Using ```ls -la``` I see these hidden files: Using the OS utility: drwxr-xr-x, drwx------@ Using ```zipfile```: drwx------, drwx------@ — Yarden, Apr 05 '22 at 10:48
So, if you try `md5sum PythonExtractedDNG` and `md5sum ArchiveExtractedDNG` do you get the same? — Mark Setchell, Apr 05 '22 at 13:54
And what happens if you run `xattr ArchiveExtractedDNG` and `xattr PythonExtractedDNG` ? — Mark Setchell, Apr 05 '22 at 13:55
for ```xattr ArchiveExtractedDNG``` I'm getting ```com.apple.quarantine```. And for ```xattr PythonExtractedDNG``` I'm getting nothing. And I can confirm I'm getting the same results for md5. — Yarden, Apr 05 '22 at 14:11
The data you are viewing is almost certainly not EXIF data, as unzipping will not remove embedded metadata, but more likely Mac file system specific XAttr and MDItem data. Try running [exiftool](https://exiftool.org/) on the file with this command: `exiftool -G0 -a -s -api RequestAll=2 file.jpg` If the results show your metadata in tags starting with `XAttr` or `MDItem`, then the metadata is part of the file system, not embedded in the file. — StarGeek, Apr 05 '22 at 16:01
Thank you for this. Using ```exiftool```, I was able to see that both files have EXIF data. But for some reason, the EXIF data of the file unzipped with shutil is inaccessible to the OS's ```get info``` function or to the ```exifread``` package. — Yarden, Apr 05 '22 at 17:03
Maybe you could click [edit] and add some representative output from StarGeek's `exiftool` command and also update your other findings so that they are all easy to see in the question, rather than scrabbling around in the comments. Thank you. — Mark Setchell, Apr 05 '22 at 17:06

score 1 · Answer 1 · answered Apr 05 '22 at 17:23

1

Turns out, I was reading the .zip with exifread instead of the unpacked DNG file within. My bad. I still can't explain why the OSs get info has a hard time with the metadata of the unpacked file, but exifread works as expected. Thanks for all the help that got me to this point.

answered Apr 05 '22 at 17:23

Yarden

11
2

It may be that the info is obtained using `mdls UNZIPPEDFILE.DNG` and the macOS tool might automatically trigger the update of the metadata database by the `mdservices` daemon whereas you may have to wait for the file to be indexed when unpacked by Python. I am guessing here and may well be wrong. – Mark Setchell Apr 05 '22 at 17:32

zipfile and shutil not keeping the file exif data

1 Answers1