2

I want to import the data from DB2 databese to the hadoop(HDFS,Hive).One way is to do it by sqoop, can we do the same with some other way?Pls share the other approach of doing this..thanks

Pankaj Chouhan
  • 31
  • 1
  • 5
  • Are u able to do jdbc connection to DB2. I am not much familiar with DB2? – subZero May 29 '14 at 13:05
  • I have not done that but yes it is possible by downloading the DB2 Connector provided by IBM... – Pankaj Chouhan May 29 '14 at 13:39
  • Then this might help you http://archanaschangale.wordpress.com/tag/dbinputformat/ – subZero May 29 '14 at 14:04
  • Be careful about that "connector". There is a JDBC driver called IBM Data server JDBC and that is what you need. There is also something called DB2 connect, but that for DRDA connection (to mainframes) – AngocA May 29 '14 at 22:19

3 Answers3

2

Sqoop is the best way to go. Anything else would require a serious amount of custom code. I've actually been on a project where we had a pretty esoteric reason we couldn't use Sqoop, and it ended up not being that trivial. You end up worrying about translating types, handling null values, encodings, escaping, retries, transactions, etc, etc.

Why reinvent the wheel? There are no other RDBMS <-> Hive connectors I know of because Sqoop does it well. Use Sqoop unless you have a very good, very specific reason not to.

Joe K
  • 18,204
  • 2
  • 36
  • 58
1

Try this Sqoop command.

sqoop import --driver com.ibm.db2.jcc.DB2Driver --connect jdbc:db2://db2.my.com:50000/databaseName --username database_name --password database_password --table table_name --split-by tbl_primarykey --target-dir sqoopimports
Venu A Positive
  • 2,992
  • 2
  • 28
  • 31
0

Use the DB2 export utility to export data from a database to a file and then FTP flat files to Hadoop, and load into Hive.

Simple Export operation requires target file, a file format, and a source file. db2 export to "target" of "fileformat" select * from "soruce"

Shiva
  • 1