-1

I want to import data from MySQL to remote Hive using sqoop. I have installed Sqoop on a middleware machine. When i run this command:

sqoop import --driver com.mysql.jdbc.Driver --connect jdbc:mysql://192.168.2.146:3306/fir --username root -P -m 1 --table beard_size_list --connect jdbc:hive2://192.168.2.141:10000/efir --username oracle -P -m 1 --hive-table lnd_beard_size_list --hive-import;

Is this command correct can i import data from remote MySQL to remote Hive?

When i ran this command it keeps on trying to connect to resource manager:

17/11/01 10:54:05 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.6.1.0-129
Enter password: 
17/11/01 10:54:10 INFO tool.BaseSqoopTool: Using Hive-specific delimiters 
for output. You can override
17/11/01 10:54:10 INFO tool.BaseSqoopTool: delimiters with --fields-
terminated-by, etc.
17/11/01 10:54:10 WARN sqoop.ConnFactory: Parameter --driver is set to an 
explicit driver however appropriate connection manager is not being set (via 
--connection-manager). Sqoop is going to fall back to 
org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which 
connection manager should be used next time.
17/11/01 10:54:10 INFO manager.SqlManager: Using default fetchSize of 1000
17/11/01 10:54:10 INFO tool.CodeGenTool: Beginning code generation
17/11/01 10:54:11 INFO manager.SqlManager: Executing SQL statement: SELECT 
t.* FROM beard_size_list AS t WHERE 1=0
17/11/01 10:54:11 INFO manager.SqlManager: Executing SQL statement: SELECT 
t.* FROM beard_size_list AS t WHERE 1=0
17/11/01 10:54:11 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is 
/usr/hdp/2.6.1.0-129/hadoop-mapreduce
Note: /tmp/sqoop-
oracle/compile/d93080265a09913fbfe9e06e92d314a3/beard_size_list.java uses or 
overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/11/01 10:54:15 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-
oracle/compile/d93080265a09913fbfe9e06e92d314a3/beard_size_list.jar
17/11/01 10:54:15 INFO mapreduce.ImportJobBase: Beginning import of 
beard_size_list
17/11/01 10:54:15 INFO Configuration.deprecation: mapred.jar is deprecated. 
Instead, use mapreduce.job.jar
17/11/01 10:54:15 INFO manager.SqlManager: Executing SQL statement: SELECT 
t.* FROM beard_size_list AS t WHERE 1=0
17/11/01 10:54:17 INFO Configuration.deprecation: mapred.map.tasks is 
deprecated. Instead, use mapreduce.job.maps
17/11/01 10:54:17 INFO client.RMProxy: Connecting to ResourceManager at 
hortonworksn2.com/192.168.2.191:8050
17/11/01 10:54:17 INFO client.AHSProxy: Connecting to Application History 
server at hortonworksn2.com/192.168.2.191:10200
17/11/01 10:54:19 INFO ipc.Client: Retrying connect to server: 
hortonworksn2.com/192.168.2.191:8050. Already tried 0 time(s); retry policy 
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 
MILLISECONDS)
17/11/01 10:54:20 INFO ipc.Client: Retrying connect to server: 
hortonworksn2.com/192.168.2.191:8050. Already tried 1 time(s); retry policy 
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 
MILLISECONDS)
17/11/01 10:54:21 INFO ipc.Client: Retrying connect to server: 
hortonworksn2.com/192.168.2.191:8050. Already tried 2 time(s); retry policy 
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 
MILLISECONDS)
17/11/01 10:54:22 INFO ipc.Client: Retrying connect to server: 
hortonworksn2.com/192.168.2.191:8050. Already tried 3 time(s); retry policy 
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 
MILLISECONDS)
17/11/01 10:54:23 INFO ipc.Client: Retrying connect to server: 
hortonworksn2.com/192.168.2.191:8050. Already tried 4 time(s); retry policy 
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 
MILLISECONDS)

The port it is trying to connect is 8050 but the actual port is 8033. How can i fix this?

James Z
  • 12,209
  • 10
  • 24
  • 44
daniyal.bashir
  • 88
  • 4
  • 14

3 Answers3

1

try this below command :

sqoop import --driver com.mysql.jdbc.Driver --connect jdbc:mysql://192.168.2.146:3306/fir --username root -P -m 1 --table beard_size_list ;

Ratul Arora
  • 31
  • 4
0

Please check the below property is set to yarn-site.xml correctly

<name>yarn.resourcemanager.address</name>
<value>192.168.2.191:8033</value>
maxmithun
  • 1,089
  • 9
  • 18
0

Why you have added -connect statement twice in your code? Try with below code:

sqoop import --driver com.mysql.jdbc.Driver --connect jdbc:mysql://192.168.2.146:3306/fir --username root -P -m 1 --split-by beard_size_list_table_primary_key --table beard_size_list --target-dir /user/data/raw/beard_size_list --fields-terminated-by "," --hive-import --create-hive-table --hive-table dbschema.beard_size_list

Note:

create-hive-table – Determines if set job will fail if a Hive table already exists. It will work in this case other wise you have create hive external table and set the target-dir path

Souvik
  • 377
  • 4
  • 16
  • my source and target are on remote machines and i am running sqoop from my middleware machine. It was just a try i am not sure if this command will work or not. – daniyal.bashir Nov 01 '17 at 10:10