Remove specific column from Hive Partitioned external table

Question

I would like to drop one of the columns from a huge hive partitioned table. Wondering how can I do this. Here is what I have tried as below.

DDL :

CREATE TABLE xyz.test_change (a int, b int, c int);

ALTER command :

 ALTER TABLE xyz.test_change REPLACE COLUMNS (a int, b int);

Error :

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Replacing columns cannot drop columns for table xyz.test_change. SerDe may be incompatible

org.apache.hadoop.hive.ql.metadata.HiveException: Replacing columns cannot drop columns for table xyz.test_change. SerDe may be incompatible

Answer 1

REPLACE COLUMNS supports replacing columns only with a native SerDe (DynamicSerDe, MetadataTypedColumnsetSerDe, LazySimpleSerDe and ColumnarSerDe).

Make sure, that table has one of the native Serde as defined above.

You can refer more details here : https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Add/ReplaceColumns