I use DSE Graph Loader reading input files from Hadoop Distributed File Systems.
I would like to insert the data into dse graph cluster(on multiple machines) in a distributed way.How can It be done?
Asked
Active
Viewed 74 times
1 Answers
0
The DSE Graph Loader is a command line utility which supports loading data from many sources including CSV, text, JSON, Gryo, HDFS and AWS S3 sources. It cannot be run as a Hadoop/Spark job.
To parallelize the injest with multiple threads, configure the parameter load_threads (default 1). Documentation can be found here: Configuring DSE Graph Loader
Brad Schoening
- 1,281
- 6
- 22
-
Can spark job manage graphloader running on 10 machines ? (in parallel) – user4808924 Nov 17 '16 at 11:29
-
No, Graph Loader is a command line utility. A single process with threads is the current state. It isn't a Hadoop/Spark job you could run. – Brad Schoening Nov 21 '16 at 17:43