5

  • I have file like this:

    1^%~binod^*~1^%~ritesh^*~1^%~shisir^*~1^%~budhdha^*~1^%~romika^*~1^%~rubeena^*~

  • Where --input-fields-terminated-by '^%~' --input-lines-terminated-by '^*~'.

  • I tried to export via command:

    sqoop export --connect jdbc:mysql://localhost/sqoop --username root -P --table makalu --export-dir sqoop/makalu --input-fields-terminated-by '^%~' --input-lines-terminated-by '^*~'

  • But it writes a single line of:

id|name
1|%~binod

in MySQL.

  • Now my question is what would be the command to solve this scenario.
RaiBnod
  • 2,141
  • 2
  • 19
  • 25

2 Answers2

4

arguments --input-fields-terminated-by and --input-lines-terminated-by accepts single character

when you execute your export command in terminal you can see below warnings 

02/03 18:55:21 WARN sqoop.SqoopOptions: Character argument ^%~ has multiple characters; only the first will be used.
15/02/03 18:55:21 WARN sqoop.SqoopOptions: Character argument ^*~ has multiple characters; only the first will be used.

Update:

One thing that I have observed is if lines are terminated other than \n or new line, then its only inserting one record into the database when you export. I tried couple of combination with field and line termination but only in case of line is terminated with \n data is exported into RDBMS table

Even tried with --input-lines-terminated-by and --lines-terminated-by arguments but no luck.

I have tested this on

Cloudera CDH-5.0.0-1
Sqoop version: 1.4.4-cdh5.0.0

Only thing that I'm able to find is this.

Prasad Khode
  • 6,602
  • 11
  • 44
  • 59
  • Even in my case of input: 1%binod^1%ritesh^1%shisir^1%budhdha^1%romika^1%rubeena^ And command: sqoop export --connect jdbc:mysql://localhost/sqoop --username root -P --table makalu --export-dir sqoop/makalu --input-fields-terminated-by '%' --input-lines-terminated-by '^' Shows error: Caused by: java.lang.RuntimeException: Can't parse input data: 'null' and outputs: id|name 1|binod In MySQL. Now what would be the solution to export all those data. – RaiBnod Feb 03 '15 at 15:45
0

The argument --input-fields-terminated-by 'any delimiter in single quotes' works. Also do not use --direct in this case as then it overrides the delimiter. Obviously you would need a table created in mysql whose default delimited is ','

crb
  • 1