Corrupt zip when uploading via Axios POST

Question

I'm 'publishing' a zip file to a server app via a REST endpoint. If I POST via Postman or the app's frontend, I get a published zip file which is valid. I can turn around download it, and open it, etc.

If I attempt to do the same thing with my code and Axios, the server app's attempt to unzip and use the content I've uploaded fails. If I DL the archive, it is corrupt. The fact that the same archive works via Postman & the app's UE tells me this is either a PEBKAC or potentially an issue with Axios itself. Here is the code I'm using to POST to the endpoint. Note that at the end, I'm actually writing the data I POST to a local file on my machine as a zip so I can verify I'm not doing anything dumb whenI read the file via fs. The local copy of the file I created is fine and dandy.

Note how I'm actually hard-coding content-length, as well. I'm testing with a single file and I've verified the length is correct via fs.statSync AND that it matches the Content-Length I see when I upload via Postman & the App's UE.

var uploadFile = (data, fileInfo) => new Promise(resolveUpload => {

    console.log("Starting Upload API call for:", fileInfo.description);

    axios.post(aepServer + '/api/v1/files',
            data, {
                jar: cookieJar,
                withCredentials: true,
                headers: {
                    'Content-Type': 'application/octet-stream',
                    'path': fileInfo.path,
                    'description': fileInfo.description,
                    'Content-Length': 354198,
                    'Accept-Encoding': 'gzip, deflate, br',
                    'Accept': '*/*',
                    'Connection': 'keep-alive'

                },
                // DANGER: allow self-signed certs workaround which I must remove 
                httpsAgent: new https.Agent({
                    rejectUnauthorized: false,
                })
            }).then((response) => {
            fileResponse = response.data;
            console.log('\n', chalk.bgMagenta('FILE UPLOADED: '), response);
            fs.writeFileSync('c:\\data\\newfile.zip', data, 'binary');
            resolveUpload(fileResponse);
        })
        .catch((err) => {
            console.log("AXIOS ERROR: ", err);
        })
});

Does anything look wrong here? While looking at the response object, I do see something that has me scratching my head:

headers: {
  Accept: '*/*',
  'Content-Type': 'application/octet-stream',
  path: '/Workspaces/Public%20Apps/UFOs.yxzp',
  description: 'UFO Sitings in the US, 1995 to present',
  'Content-Length': 532362,
  'Accept-Encoding': 'gzip, deflate, br',
  Connection: 'keep-alive',
  Cookie: 'ayxSession=s%3Ac39f55a3-b219-43a5-9f8a-785e1222c81c.QR4KI8uXaQlL9axqkO8AkyabPVt3i37nGbz%2FJef0eqU',
  'User-Agent': 'axios/0.19.2'
},

Look at the content length: 532362 bytes. It seems like the ~354k value I hard-coded in the headers is being ignored somehow. Might this be my problem? [BTW, if I use the same code to upload a csv or txt file, all is well - this seems related to compressed files only]

EDIT: Welp, it looks like Axios does override that property and there's nothing I can do about it: Axios set Content-Length manually, nodeJS. Now the question is if setting this incorrectly would munge the file and WHY the value is wrong. When I do a data.length, I get the 354198 value.

score 1 · Answer 1 · answered Apr 27 '20 at 23:59

My answer was here:

Reading binary data in node.js

In essence, I was reading the zip file as 'binary" when I shouldn't have passed anything in at all.

var content = fs.readFileSync(aFile.readFrom);
//NOT
//var content = fs.readFileSync(aFile.readFrom, 'binary');

score 0 · Answer 2 · answered Apr 27 '20 at 19:07

0

can you try with 'multipart/form-data'

axios.post('upload_file', formData, {
  headers: {
    'Content-Type': 'multipart/form-data'
  }
})

alternatively get the CURL request using postman and use the header information supplied by postman

answered Apr 27 '20 at 19:07

Derviş Kayımbaşıoğlu

28,492
4
50
72

No change using form-data. the cURL request from postman actually passes less information: curl --location --request POST 'https://localhost/api/v1/files' \ --header 'Content-Type: application/octet-stream' \ --header 'descrption: One of the best files, ever' \ --header 'path: /Workspaces/Public%20Apps/UFO2s.yxzp' \ --header 'Content-Type: application/octet-stream' \ --header 'Cookie: ayxSession=s%3Add6ba65f-509b-49d5-b286-d854b7a2e1d8.KD8O8uxj%2FWqWHdT6XB4BPLLmSSlTAqX23l8wBv18Fd4' \ --data-binary '@/C:/demo-seeder/Files To Upload/Workspaces/Public Apps/UFOs.yxzp'. – Russell Christopher Apr 27 '20 at 20:24
The --data-binary parameter made me wonder if the problem was the way I was fs.reading the file, but that seems to be OK. – Russell Christopher Apr 27 '20 at 20:25

Corrupt zip when uploading via Axios POST

2 Answers2