17

I have an S3 bucket with several zipped CSV files (utilization logs.) I'd like to query this data with Athena, but the output is completely garbled.

It appears Athena is trying to parse the zip files without decompressing them first. Is it possible to force Hive to recognize my files as compressed data?

Kirk Broadhurst
  • 27,836
  • 16
  • 104
  • 169
MattY
  • 527
  • 2
  • 4
  • 9

1 Answers1

34

For Athena compression is supported, but the supported formats are

  • Snappy (.snappy)
  • Zlib (.bz2)
  • GZIP (.gz)

Those formats are detected by their filename suffix. If the suffix doesn't match, the reader does not decode the content. I tested it with a test.csv.gz file and it worked right away. So try changing the compression from zip to gzip and it should work.

Scott
  • 16,711
  • 14
  • 75
  • 120
jens walter
  • 13,269
  • 2
  • 56
  • 54
  • 1
    Thanks Jens. I was afraid this might be the answer. I can't control the format in this bucket, I'll need to schedule a job to programmatically unzip and store them elsewhere. – MattY Dec 20 '16 at 20:33