4

I'm moving the data from Mysql to S3 using data pipeline and it creates empty file for couple of days. I believe, it is making my athena query fails with

"HIVE_CURSOR_ERROR: Unexpected end of input stream".

Below is my script

CREATE EXTERNAL TABLE `test`(
  `col0` bigint, 
  `col1` bigint, 
  `col2` string, 
  `col3` string 
  )
ROW FORMAT DELIMITED 
  FIELDS TERMINATED BY ',' 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  's3://dummy/'

Could you please let me know if there is any option to skip zero bytes S3 file?

Vamsi Prabhala
  • 48,685
  • 4
  • 36
  • 58

0 Answers0