2

In my hdfs already exists the directory "mydatabase" and I am trying to update the data that is in mysql for hdfs with sqoop

However, in the import-all-tables command there is no overwrite option in the hdfs directory, only in hive

I'm trying to do the following command:

sqoop import-all-tables 
--connect jdbc:mysql://10.20.20.43/mydatabase 
--username blizzard 
--password blizzard 
--warehouse-dir /user/hdfs/mydatabase -m 1
--hive-database mydatabase
--hive-import 
--hive-overwrite
--null-string '\\N' 
--null-non-string '\\N' 
--verbose

But I have an error:

17/05/03 19:48:13 ERROR tool.ImportAllTablesTool: 
Encountered IOException running import job:
org.apache.hadoop.mapred.FileAlreadyExistsException: 
Output directory hdfs://10.20.20.43:8020/user/hdfs/mydatabase/table1
already exists

I want to import all the data, always, because I do not know which one could have been changed or not.

What is the best way to do this?

Thanks!

Edit:

The --delete-target-dir command is only for import and not for import-all-tables according to the documentation https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html

sola.carol
  • 47
  • 10
  • 1
    Possible duplicate of [overwrite hdfs directory Sqoop import](http://stackoverflow.com/questions/40130163/overwrite-hdfs-directory-sqoop-import) – Andrew May 03 '17 at 19:28
  • 1
    Is not duplicated, if you look in the documentation, the command (--delete-target-dir) of the post that you said is only for the command import and not for the command import-all-tables which is my doubt https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html @Andrew – sola.carol May 04 '17 at 16:34

1 Answers1

0

You can delete all data to rewrite them again. Try it before:

$hdfs dfs -rm -r /user/hduser/*
Quockhanh Pham
  • 384
  • 2
  • 11