0

I'm ingesting the content of a Teradata table that contains ~700,000,000 rows with JDBC FastExport.

My code (simplified) is the following:

Class.forName("com.teradata.jdbc.TeraDriver")
val url = "jdbc:teradata://xxx/database=xxx,CHARSET=UTF8,TMODE=TERA,TYPE=FASTEXPORT,RECONNECT_COUNT=50"
val connection = DriverManager.getConnection(url, "user", "pwd")
val stmt = connection.prepareStatement("SELECT * FROM MY_TABLE")
stmt.setFetchSize(100000)
val rs = stmt.executeQuery()
while (rs.next()) println(rs.getObject(1))

The driver I'm using come from "com.teradata.jdbc" % "terajdbc4" % "16.20.00.13".

The performance I'm getting is ~1,200,000 rows/min and I was wondering if I can expect more and if there's anything I can do to improve it.

Maxence Cramet
  • 552
  • 1
  • 5
  • 17
  • Depending on your actual bottleneck TPT Export might be faster, e.g. using multiple sessions/instances – dnoeth Jun 05 '20 at 12:32
  • Thx @dnoeth, that's our plan B. We've chosen JDBC because it was possible to use with Scala but we may review that. – Maxence Cramet Jun 05 '20 at 17:45
  • You can specify multiple sessions for FastExport in your connection string:`TYPE=FASTEXPORT,SESSIONS=`. Pretty hard to predict if upping your sessions will help performance, but easy enough to test. – Andrew Jun 05 '20 at 20:40
  • Thx @Andrew, I've run many tests with 5,100 and 1000 but actually it's slower than not specifying it – Maxence Cramet Jun 08 '20 at 13:24

0 Answers0