0

Usually, I use a JDBC connection with some ETL tool to move data from one database(i.e Teradata) to another database(i.e Greenplum).

However, both of these databases comes with inbuilt utilities which can load/export huge amounts of data very fast, far faster than JDBC!. But the downside as far as I am aware of is that it can do so only to/from a file.

So, if I want to use them I have to Follow a process like-

Teradata ---(Fast Export)---> File ---(Gploader)---> Greenplum

I am wondering if it is possible to skip the File part and Combine the two utilities.

Teradata ---(FastExport & Gploader)--> Greenplum.

That way I can transfer huge amounts of data very quickly!

Vijay Shekhawat
  • 3
  • 5
  • 1
    FastExport (and the newer & recommended TPT Export) can export to a Named Pipe, don't know if GPloader supports reading from a pipe – dnoeth Feb 09 '19 at 21:27
  • A named pipe gives you a single input channel. No need to use gpload. You can just pipe this into COPY. – A. Scherbaum Feb 09 '19 at 21:45

1 Answers1

0

Yes, you most certainly can. Greenplum supports all kinds of external tables. One solution is to use an External Table that executes a command. That command can be a Java program that connects to Teradata to get data and uses the FastExport option.

I wrote the tool "gplink" to do just this. It automates the creation of Greenplum External Tables for JDBC sources.

Github: https://github.com/pivotalguru/gplink

Teradata connection example: https://github.com/pivotalguru/gplink/blob/master/connections/teradata.properties

And my blog: http://www.pivotalguru.com/?page_id=982

Jon Roberts
  • 2,068
  • 1
  • 9
  • 11