0

I am facing issue in Hive while defining the escape character in OpenCvSerde properties. As per the code below :

CREATE EXTERNAL TABLE IF NOT EXISTS databasename.tablename (
    ColumnName1 datatype,
    ColumnName2 datatype,
    ColumnName3 datatype
) ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde'
WITH SERDEPROPERTIES (
   "separatorChar" = "|",
   "quoteChar"     = "\"",
   "escapeChar"    = "\r\n"
) Location '<HDFS LOCATION>' tblproperties ("skip.header.line.count"="1");

The csv file is treating '\r\n' as same as '\n'. Which is making a difference in count of number of records.

Please suggest the solution so that I can use \r\n as escapeChar in OpenCSVserde properties, or there is any other possibility to make changes in serde properties. P.S Using org.apache.hadoop.hive.serde2.OpenCSVSerde is mandatory for the purpose. I did go through link : character slash is not being read by hive on using OpenCSVSerde

But it didn't serve my purpose.

Thanks

Sachin Vardhan
  • 743
  • 1
  • 8
  • 19

1 Answers1

0

can you do an octaldump and see the delimiter? Can you also specify the "escapeChar" as "\u001" and try?

user3327034
  • 395
  • 3
  • 13