0

I have a zip file with the size 1 GB on S3 bucket. After downloading, I can't seem to unzip it. It always says

End-of-central-directory signature not found.  Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.

Later, I download it again, using s3cmd this time. It says

WARNING: MD5 signatures do not match: computed=384c9a702c2730a6b46d21606137265d, received="b42099447c7a1a390d8e7e06a988804b-18"

Is there any s3 limitation I need to know or this is a bug?

Oscar Yuandinata
  • 1,025
  • 1
  • 13
  • 31

2 Answers2

1

This question seems dead, but I'll ask it for anyone landing here:

Amazon S3's multipart uploads (those suitable for big files) produce ETag values which no longer matches file's MD5, so if you're using this as checksum (as it seems looking at your received MD5) it won't work.

Best you can do for validation is ensuring ContentMD5 header is added to every part's header on your multipart upload ensuring file does not get corrupted during upload, and adding your own MD5 metadata field for checking data after download.

ergoithz
  • 41
  • 4
0

Thanks @ergoithz for reminding me that I had this question :)

The problem is already fixed, with AWS SDK for nodejs being the problem. Apparently it cannot upload large files using stream data fs.createReadStream(), so I switched to using Knox where it worked perfectly

Oscar Yuandinata
  • 1,025
  • 1
  • 13
  • 31