Not in GZIP format java while reading the zip dynamically

Question

my requirement is to unzip a log file dynamically and write it in the console. But I am getting an exception

java.util.zip.ZipException: Not in GZIP format
    at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:166) ~[?:?]
    at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:80) ~[?:?]
    at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:92) ~[?:?]

here is my code


 private static void readLogFile(File f) {
        try(BufferedReader reader = new BufferedReader(new InputStreamReader(new GZIPInputStream(new FileInputStream(f)), "UTF-8"))) {
            String  nextLine;
            while ((nextLine = reader.readLine()) != null) {
                System.out.println(nextLine);
            }
        } catch (FileNotFoundException e) {
            logger.error("file doesn't exist." );
        } catch (IOException e) {
            logger.error("Error reading the file", e);
        }
    }

The size of my file is around 50 MB The size of my .gz file is 2.4 MB

I have already tried unzipping the file in the terminal and it works so the file is not corrupt as well

can someone help me with what is going wrong here?

So which command did you use to unzip the compressed file on console? — Gyro Gearless, Jun 13 '23 at 10:11
IIRC, `gzip -d` does this same as `gunzip`, which does _"gunzip can currently decompress files created by gzip, zip, compress, compress -H or pack. The detection of the input format is automatic."_, so maybe your file isn't actually gzip compressed, but using a different format. — Mark Rotteveel, Jun 13 '23 at 10:17
But if i check the validity of the file $ gzip -v -t application.2.log.gz . I get this reponse - application.2.log.gz: OK — Watermelon, Jun 13 '23 at 10:30
If `-d` decompress various formats, why do you assume that `-t` wouldn't be able to test those various formats? What are the first two bytes of the file? If they aren't 0x1f 0x8b, then it isn't gzip. — Mark Rotteveel, Jun 13 '23 at 10:38
*If they aren't 0x1f 0x8b, then it isn't gzip* And setting those two leading bytes to zero on a valid gzipped file will give you exactly that exception — g00se, Jun 13 '23 at 11:37

Michael Gantman · Answer 1 · 2023-06-13T11:44:04.540

0

Your code looks correct I think that your file may be compressed in format that is not GZIP and that format is supported gzip command line utility but not by GZIPInputStream class. If this is the case one of the workarounds would be is to invoke command line utility from Java using ProcessBuilder class. This is definitely not ideal as your immediately becomes operating system dependent, but it will work. Other options would be to request that your log files provided to you be compressed in GZIP (but that depends on who is your log files supplier and if they can and willing to co-operate with you) or look for 3d party java library that supports the format in which your files are compressed

edited Jun 13 '23 at 11:44

answered Jun 13 '23 at 11:05

Michael Gantman

7,315
2
19
36

The gzipped file has nothing to do with the charset parameter. That's for the `Reader` decoding *the text* that is the content of the gzipped file – g00se Jun 13 '23 at 11:34
@g00se you are correct, just noticed that it was not GZIPInputStream constructor parameter. Just modified my answer to remove the charset detail – Michael Gantman Jun 13 '23 at 11:43
The logs are generated by me and i zip them in the correct format. Maybe my approach of zipping the logs is wrong – Watermelon Jun 13 '23 at 11:58
Write a small java program that zips some text file and create a zip. See if you can unzip it with your java code and if so see how your 2 zips differ – Michael Gantman Jun 13 '23 at 12:16
I will try that. Thank You :) – Watermelon Jun 13 '23 at 12:39

score 0 · Answer 2 · answered Jun 13 '23 at 11:26

Ok, so it's not GZip but it could be ZIP right?

        String fileZip = "src/main/resources/unzipTest/compressed.zip";
        File destDir = new File("src/main/resources/unzipTest");

        byte[] buffer = new byte[1024];
        ZipInputStream zis = new ZipInputStream(new FileInputStream(fileZip));
        ZipEntry zipEntry = zis.getNextEntry();
        
        while (zipEntry != null) {
            File newFile = newFile(destDir, zipEntry);
            if (zipEntry.isDirectory()) {
                if (!newFile.isDirectory() && !newFile.mkdirs()) {
                    throw new IOException("Failed to create directory " + newFile);
                }
            } else {
                // fix for Windows-created archives
                File parent = newFile.getParentFile();
                if (!parent.isDirectory() && !parent.mkdirs()) {
                    throw new IOException("Failed to create directory " + parent);
                }

                // write file content
                FileOutputStream fos = new FileOutputStream(newFile);
                int len;
                while ((len = zis.read(buffer)) > 0) {
                    fos.write(buffer, 0, len);
                }
                fos.close();
            }
            zipEntry = zis.getNextEntry();
        }

        zis.closeEntry();
        zis.close();

as well as this:

public static File newFile(File destinationDir, ZipEntry zipEntry) throws IOException {
    File destFile = new File(destinationDir, zipEntry.getName());

    String destDirPath = destinationDir.getCanonicalPath();
    String destFilePath = destFile.getCanonicalPath();

    if (!destFilePath.startsWith(destDirPath + File.separator)) {
        throw new IOException("Entry is outside of the target dir: " + zipEntry.getName());
    }

    return destFile;
}

give that a shot.

Not in GZIP format java while reading the zip dynamically

2 Answers2