1

I am loading a table of 200,000,000 rows daily. The fastload does this in 2-3 hours. However, I have other table to upload, so this creates an issue for me because those tables are even bigger.

Is there any other faster way ?

Elvis
  • 255
  • 3
  • 11

1 Answers1

2

Use TPT Load instead of FastLoad.

Did you investigate what's the bottleneck?

Is it maxing out on the load server reading the data or processing it? 100% network usage? The Teradata system?

Fastload is single-threaded, there's a high probability for maxing out on a single CPU. Then switching to TPT Load (which also runs the Fastload protocol) is an easy solution, you simply run multiple instances of an operator.

dnoeth
  • 59,503
  • 4
  • 39
  • 56
  • Thank you. According to the online articles `jdbc` does not support TPT, is that right ? – Elvis May 10 '15 at 22:53
  • I was making an online search for the TPT usage in `java` applications, but could not find any examples. I am currently using the `fastload` with java code. Do I need to have the same connection string : ` ... whatever TYPE=FASTLOAD` ? Do I only need to modify the Prepared insert statement ? – Elvis May 10 '15 at 23:21
  • You didn't specify that you're using JDBC-Fastload, IMHO this is always slower than native Fastload (and single-threaded, too). Is the data created by your Java application or read from another source? – dnoeth May 11 '15 at 05:58
  • The data is either read from DB2 database, or a different DSN of Teradata DB. So, yes, one can assume the data is created within the Java application – Elvis May 14 '15 at 04:44
  • @Elvis: Unless you do some complex processing in your application loading data from DB2 or Teradata is a common task for TPT... – dnoeth May 14 '15 at 07:26
  • so I can transport data with a Java application between Teradata DSNs and from DB2 to Teradata, right ? Is there any sample code . I was searching online everywhere, but could not find any. Most sample codes are for Fastload. Do I need only to change the Fastload query `INSERT . . .` to the `TPT` query `APPLY . . ` ? Thank You – Elvis May 14 '15 at 16:46
  • @Elvis: No, you can't run TPT directly from Java, but you could call the TPT API, this is what 3rd party ETL tools do. – dnoeth May 14 '15 at 17:09