-1

"What is the fastest way of loading data in vertica database ? If i have 100 GB of data which i need to laod in vertica table in a fast and efficient way."

Karam
  • 411
  • 4
  • 10

1 Answers1

2

I would say the fastest of all is the standard COPY command: COPY myschema.mytable FROM '/data/common/mytable.csv' ON ANY NODE Plus any other necessary parameters for column delimiter, string encloser, error handling, etc.

And, /data/common , here, would be a file location that is NFS-mounted to the same directory on all nodes of Vertica.

If /data/common/mytable.csv is not gzipped, you also allow for apportioned load: If you have, for example, 8 parse streams, each will grab 1/8th of the complete file volume (navigating to the nearest newline or other record delimiter) and just parse that part, all in parallel.

I can't imagine a faster way - hope this helps.

marcothesane
  • 6,192
  • 1
  • 11
  • 21