1

I have insert overwrite query in hive, After query is executed data is dumped into trash folder.

Is there any option or property by which it can avoided?

sagar karira
  • 11
  • 1
  • 3

1 Answers1

1

In Hive 1.2.0 there is a PURGE option for DROP operation: https://issues.apache.org/jira/browse/HIVE-9118 and https://issues.apache.org/jira/browse/HIVE-7100

Unfortunately this does not work for external tables and for insert overwrite statement.

But still you can drop files before INSERT OVERWRITE, I know this is not always acceptable solution, using rm command with -skipTrash option:

hadoop fs -rm -r -f -skipTrash  hdfs://your_table_path/*

If you are on DEV environment, you may want to disable TRASH feature at all. This can be done by setting fs.trash.interval=0 in core-site.xml

leftjoin
  • 36,950
  • 8
  • 57
  • 116