1

I need to load data from sybase(production database) to HDFS. By using sqoop it is taking very long time and frequently hit the production database. So, I am thinking to create data files from sybase dump and after that copy the data files to hdfs. Is there any tool(open source) is available to create required data files(flat files) from sybase dump.

Thanks,

user1321939
  • 319
  • 2
  • 6
  • 18

1 Answers1

1

The iq_bcp command line utility is designed to do this on a per table basis. You just need to generate a list of tables, and you can iterate through the list.

iq_bcp [ [ database_name. ] owner. ] table_name { in | out } datafile

iq_bcp MyDB..MyTable out MyTable.csv -c -t#$#

-c specifies a character (plaintext) output -t allows you to customize the column delimiter. You will want to use a character or series of characters that do not appear in your extact e.g. if you have a text column that contains text with a comma, a csv will be tricky to import without additional work.

Sybase IQ: iq_bcp

Mike Gardner
  • 6,611
  • 5
  • 24
  • 34
  • Thanks a lot Michael. is it possible to directly apply this tool(with command) to sybase dump without hitting the production database. – user1321939 Dec 30 '15 at 09:46
  • No. It has to be run against a running server. Your alternative would be to restore the dumps to a backup server, and run the extracts from there. – Mike Gardner Dec 30 '15 at 13:36