I want to create one big table contains all the data from all table in database then export this table into csv file then import this file into Hbase ? My issue is first step which is how to create bigtable from all database tables? i will be thankful for any help . Thanks in advance .
Asked
Active
Viewed 867 times
0
-
2Just join them all, of couse that assumes that they all can be joined sensibly... – Tony Hopkinson Apr 18 '13 at 15:53
-
i have thousands table – Samy Louize Hanna Apr 18 '13 at 15:58
-
Well unless they all have the same structure, you need to take a step back then. Think about how you'd start afresh and then look at how yiou'd get your existing data in there. – Tony Hopkinson Apr 19 '13 at 13:14
3 Answers
1
You could use sqoop to transfer all the tables and data to HBase if your desire is to dump all the data to HBase or HDFS. It may not be efficient or feasible to convert all data to one big table. Just noting that, export of CLOB/BLOB data can't be performed to CVS file.
Niranjan Sarvi
- 899
- 7
- 9
-
good point. you could use "sqoop import-all-tables" tool to do that. but you need to keep these things in mind : (1)Each table must have a single-column primary key. (2)You must intend to import all columns of each table. (3)You must not intend to use non-default splitting column, nor impose any conditions via a WHERE clause. – Tariq Apr 19 '13 at 12:23
1
Take a look at sqoop, may be it can help.
We use it to store data from RDBMS into Hive tables. As for hive, it can automatically create tables with appropriate column names and types.
It also requires split_by option for custom queries, which specifies how to split data into chunks.
Ivan Klass
- 6,407
- 3
- 30
- 28
0
Why would you do that? Write a MapReduce job and inject the output into a the HBase table.
Tariq
- 34,076
- 8
- 57
- 79