2

Given a .npz file from np.savez or np.savez_compressed, when it got loaded by np.load, is there any way to check whether the file was compressed or not?

I tried to look though docs and GitHub. It didn't tell me anything but how the file is compressed.

mkrieger1
  • 19,194
  • 5
  • 54
  • 65
ACitronella
  • 43
  • 1
  • 4

1 Answers1

2

np.load returns a NpzFile object.

  • It has a zip attribute which is a ZipFile object.

  • Which has an infolist() method which returns a list of ZipInfo objects.

  • Which has a compress_type attribute which contains the compression method used.

If np.savez was used, the compression type is ZIP_STORED, if np.savez_compressed was used, the compression type is ZIP_DEFLATED (relevant source code).

So to wrap it up:

import numpy
import zipfile

def is_compressed(npz_file):
    zip_infos = npz_file.zip.infolist()
    if len(zip_infos) == 0:
        raise RuntimeError("Did not find ZipInfos unexpectedly")
    compress_type = zip_infos[0].compress_type
    if compress_type == zipfile.ZIP_STORED:
        return False
    elif compress_type == zipfile.ZIP_DEFLATED:
        return True
    else:
        raise ValueError("Unexpected compression type")

# Example
a = numpy.array([1, 2, 3])
numpy.savez("uncompressed.npz", a)
numpy.savez_compressed("compressed.npz", a)
u = numpy.load("uncompressed.npz")
c = numpy.load("compressed.npz")
print(is_compressed(u))  # False
print(is_compressed(c))  # True
mkrieger1
  • 19,194
  • 5
  • 54
  • 65