1

I have a file a.csv.zst and i ve created textfile table.

When i select on hive i see first 100 rows without an error but when i select count(*) it gives an error unknown frame descriptor

I can decompress my file on linux with zstd -dc a.csv.zstd

How can i read it on hive?

It works on mr but not with tez

I didnt give an option to show compression level to hive table as i know the default value is 3 and i m not sure the file zstd compression level

CompEng
  • 7,161
  • 16
  • 68
  • 122
  • Is https://issues.apache.org/jira/browse/HDFS-14099 resolved in your Hadoop version? – mazaneicha Jun 08 '23 at 18:51
  • In hadoop 3.3.4 and hive 3.1.3 not resolved – CompEng Jun 09 '23 at 05:41
  • is there any comment ? – CompEng Jun 14 '23 at 07:55
  • HDFS-14099 shown as resolved in "Fix Version/s: 3.4.0, 3.2.3, 3.3.2" If you are on 3.3.4 it means that something else is going on. Posting a full stacktrace would probably help to understand what. – mazaneicha Jun 14 '23 at 12:28
  • it says : Error while processing statement: Failed: Execution Error , return code 2 from org.apache.hadoop.hive.ql.exec.tez.teztask. Vertex failed. ....... java.lang.internalerror: unknown frame descriptor at org.apache.hadoop.io.compress.zstd.ZStandardDecompressor.inflateBytesDirect......... – CompEng Jun 14 '23 at 12:46
  • I meant the complete stacktrace as found in a tez container log. – mazaneicha Jun 14 '23 at 13:17
  • @mazaneicha when i try it with mr it works but not with tez – CompEng Aug 19 '23 at 05:37

0 Answers0