"What is the fastest way of loading data in vertica database ? If i have 100 GB of data which i need to laod in vertica table in a fast and efficient way."
Asked
Active
Viewed 140 times
1 Answers
2
I would say the fastest of all is the standard COPY command:
COPY myschema.mytable FROM '/data/common/mytable.csv' ON ANY NODE
Plus any other necessary parameters for column delimiter, string encloser, error handling, etc.
And, /data/common
, here, would be a file location that is NFS-mounted to the same directory on all nodes of Vertica.
If /data/common/mytable.csv
is not gzipped, you also allow for apportioned load: If you have, for example, 8 parse streams, each will grab 1/8th of the complete file volume (navigating to the nearest newline or other record delimiter) and just parse that part, all in parallel.
I can't imagine a faster way - hope this helps.

marcothesane
- 6,192
- 1
- 11
- 21