2

When I am using DROP TABLE IF EXISTS <Table Name> in hive, it is not freeing the memory. The files are created as 0000_n.bz2 and they are still on disk.

I have two questions here:

1) Will these files keep on growing for each and every insert? 2) Is there any DROP equivalent to remove the files as well on the disk?

Ani Menon
  • 27,209
  • 16
  • 105
  • 126
Naga Vemprala
  • 718
  • 3
  • 16
  • 38

3 Answers3

8

There are two types of tables in hive. Hive managed table: If you drop a hive managed table the data in HDFS are automatically deleted.

External Table: If you drop an external table, hive doesnt delete the underlying data.

I believe yours is an external table.

sunil
  • 1,259
  • 1
  • 14
  • 27
8

Couple of things you can do:

  • Check if the table is an external table and in that case you need to drop the files manually on HDFS as dropping tables won't drop the files: hadoop fs -rm /HDFS_location/filename

  • Secondly check if you are in the right database. You need to issue use database command before dropping the tables. The database should be same as the one in which tables were created.

user3406675
  • 148
  • 9
0

Drop table if exists table_name purge;

This command will also remove data files from trash folder and cannot be recovered after table drop

Varsha Reddy Guntuka
  • 1