0

I am trying to use sqoop to import all tables in a database from SQL relational database into hive. Before doing that, I would like to know how many records are in each of the tables in my source database. This way, I can have how estimations of the impact to the source database and I want such impact to be small.

Is there a simple way to do that? I tried to use a query such as select count(*) from dbname.tbname. It was not working. Here is what my sqoop looks like:

sqoop import -Dmapred.job.queue.name=$myqueue   
    --connect "jdbc:sqlserver://$svr;database=$db;username=$user;password=$pass" 
    --query "select count(*) from $tbname "  
    --target-dir $detdatadir/$filerecord -m3

Any help would be greatly appreciated.

Iskuskov Alexander
  • 4,077
  • 3
  • 23
  • 38
Max577
  • 5
  • 4

1 Answers1

2

sqoop command should be like this 

sqoop eval -Dmapred.job.queue.name=$myqueue   
--connect "jdbc:sqlserver://$svr;database=$db;username=$user;password=$pass" 
--query "select count(*) from $tbname"
LUZO
  • 1,019
  • 4
  • 19
  • 42
  • Thanks LUZO for your help! I have imported all the tables from an SQL databases without checking the size of each tables. But I think your idea is good that we can use sqoop eval instead of sqoop import. – Max577 May 25 '18 at 14:08
  • Thanks all for the comments. Vinayak also suggested to use sqoop eval. – Max577 May 25 '18 at 14:10
  • you can accept the answer if it works fine ... thanks – LUZO May 25 '18 at 15:47