Error : sqoop to add records in hdfs

Question

My scenario : I will get daily 100 records in hdfs through sqoop at particular time. But, yesterday i got only 50 records for that particular time today i need to get 50+100 records in hdfs through sqoop for that particular time. Please help me. Thanks in advance.

so whats the error in that? your question doesn't state that.... — Nishu Tayal, May 04 '16 at 10:45
how to add new records yesterday and today records in sqoop example — user2707767, May 04 '16 at 10:52

score 0 · Answer 1 · answered May 04 '16 at 11:08

To handle such scenario, you need to add a where condition on time. No matters, what the record count is.

You can use something like this in sqoop import command using --query parameter:

sqoop import \
  --connect jdbc:mysql://localhost:3306/sqoop \
  --username sqoop \
  --password sqoop \
  --query 'SELECT * from records
                  WHERE recordTime BETWEEN ('<datetime>' AND NOW())  \
  --target-dir /user/hadoop/records

You need to modify the where condition as per your table schema.

Please refer Sqoop Documentation for more details.

score 0 · Answer 2 · answered May 04 '16 at 12:14

sqoop import --connect jdbc:mysql://localhost:3306/your_mysql_databasename --username root -P --query 'SELECT * from records WHERE recordTime BETWEEN ('' AND NOW()) --target-dir /where you want to store data

and make when sqoop ask for password enter your mysql password eg.(my pwd is root)

Error : sqoop to add records in hdfs

2 Answers2