I would like to import data from MySQL table into HDFS. I have everything configured and I am able to create simple job in sqoop-shell that is copying data. However I would like to copy each time only new records, but I am not sure how to achieve this. When I create job there is a parameter named "check column" and I have columns like ID or eventTimestamp that seem to be suitable there. However, in such case I should enter also "last value" then. Do I have to manage this last value by myself and each time create new job with new "last value"? Why in such case create a job if it is used only once and then has to be recreated? Is it not possible for Sqoop to manage this, by storing each time new "last value" and import only new records? Moreover, why I have this error message when I enter anything as "last value": "Size of input exceeds allowance for this input field. Maximal allowed size is -1"?
Asked
Active
Viewed 351 times
1 Answers
0
Concerning the problem with "last value" I have added a bug: https://issues.apache.org/jira/browse/SQOOP-2640 It has fixed status now, so in release 1.99.7 it should be fine.
Cob
- 171
- 10