overwrite hdfs directory Sqoop import

Question

Is it possible to overwrite HDFS directory automatically instead of overwriting it every time manually while Sqoop import? (Do we have any option like "--overwrite" like we have for hive import "--hive-overwrite")

Dev · Accepted Answer · 2016-10-19T12:42:01.280

16

Use --delete-target-dir

It will delete <HDFS-target-dir> provided in command before writing data to this directory.

edited Oct 19 '16 at 12:42

answered Oct 19 '16 at 11:30

Dev

13,492
19
81
174

Hello, How we can deal this with sqoop with hcatalog integration. I know that we have a way to truncate the table before sqoop import. – Tharunkumar Reddy Oct 10 '19 at 06:30

score 6 · Answer 2 · edited May 14 '17 at 03:21

Use this: --delete-target-dir

This will work for overwriting the hdfs directory using sqoop syntax:

$ sqoop import --connect jdbc:mysql://localhost/dbname --username username -P --table tablename --delete-target-dir --target-dir '/targetdirectorypath' -m 1

E.g:

$ sqoop import --connect jdbc:mysql://localhost/abc --username root -P --table empsqooptargetdel --delete-target-dir --target-dir '/tmp/sqooptargetdirdelete' -m 1

This command will refresh the corresponding hdfs directory or hive table data with updated/fresh data, every time this command is run.

overwrite hdfs directory Sqoop import

2 Answers2

Linked