sqoop job not running with parameters

Question

I am trying to run a sqoop job .I am using sqoop version Sqoop 1.4.6-cdh5.8.0 and it is not working for this version

It is working fine with Sqoop 1.4.5-cdh5.4.0.

 sqoop job --create E8 -- import  --connect jdbc:mysql://localhost/test -- username root --password cloudera --table NAME --hive-import -m1

 sqoop job --exec E8 -- --table dummy1

Is there any syntax issue.If anyone can help with this.

       Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo 
       imports  will fail.
       Please set $ACCUMULO_HOME to the root of your Accumulo installation.
       16/12/23 04:48:10 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-
       cdh5.8.0
       Enter password: 
       16/12/23 04:48:19 INFO manager.MySQLManager: Preparing to use a 
       MySQL  streaming resultset.
       16/12/23 04:48:19 INFO tool.CodeGenTool: Beginning code generation
       16/12/23 04:48:20 INFO manager.SqlManager:
       Executing SQL statement: SELECT t.* FROM `NAME` AS t LIMIT 1
       16/12/23 04:48:20 ERROR manager.SqlManager: Error executing 
       statement: 
       com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table    
      'test.NAME' doesn't exist

Are you sure that table exist in Mysql in test database _(with same case)_? — Dev, Dec 23 '16 at 13:33
yes table dummy1 exists and it is not replcaing parameter but taking the NAME in select query — coder25, Dec 23 '16 at 13:36
AS per docs, it should replace this. Syntax seems fine to me. Can you try removing `--table NAME` from create job command and all this only in exec? — Dev, Dec 23 '16 at 13:47

score 3 · Accepted Answer · answered Dec 28 '16 at 17:02

Assuming you already did the basic checks (like manually placing the parameter in the job, and executing it) I would say the syntax looks to be correct.

When looking at the doc it is mentioned that one can override properties. Unfortunately, they only show an example that adds a property, and don't show overriding one.

A search led me to this open issue which leads me to believe that there is a bug that prevents you from overriding parameters properly.

Unfortunately I don't see a solution for this, some things that might help to work around the problem:

Parameterize on a different level
Play with the syntax (does it help if it is the first/last override element, what if you try to override AND add a user, what if you try to override the query parameter instead of the table parameter ...)

@coder25 Overriding may not work. What you could do as a workaround of course, is rather than creating the job and calling it with a parameter, is just running the job directly with all parameters. -- I am not up to judging the patch, but looking at the site I believe that patch has not been tested and implemented yet. — Dennis Jaheruddin, Dec 30 '16 at 08:49

score 2 · Answer 2 · answered Dec 31 '16 at 20:44

This seems to be a bug in sqoop-1.4.6-cdh5.8.0 and sqoop-1.4.6-cdh5.9.0

This, however, as you mentioned is working correctly with 1.4.5 version.

The below solution worked for me:

1) Download 'sqoop-1.4.5-cdh5.4.0.jar' from http://repo.spring.io/libs-release/org/apache/sqoop/sqoop/1.4.5-cdh5.4.0/

2) Replace 'sqoop-1.4.6-cdh5.8.0.jar' with 'sqoop-1.4.5-cdh5.4.0.jar' and modify the symbolic link 'sqoop.jar' to point to 'sqoop-1.4.5-cdh5.4.0.jar'

3) Though I don't support downgrading, but still this works as charm.

sqoop job not running with parameters

2 Answers2