Write JavaPairRdd to Csv

Question

JavaPairRdd has saveAsTextfile function, with which you can save data in a text format.

However what I need is to save the data as CSV file, so I can use it later with Neo4j.

My question is:

How to save the JavaPairRdd 's data in CSV format? Or is there a way to transform the rdd from :

Key   Value
Jack  [a,b,c]

to:

Key  value
 Jack  a
 Jack  b
 Jack  c

[This](https://stackoverflow.com/questions/45398795/saving-the-rdd-pair-in-particular-format-in-the-output-file) might be of interest. — Joakim Danielson, Jul 09 '18 at 15:21

score 1 · Answer 1 · answered Jul 11 '18 at 11:43

You should use the flatMapValues function on your JavaPairRdd: Pass each value in the key-value pair RDD through a flatMap function without changing the keys; this also retains the original RDD's partitioning.

Just by returning the value it will create a line per element in the input lists preserving the keys.

  // In Java
  JavaPairRDD<Object, List<String>> input = ...;
  JavaPairRDD<Object, String> output = input.flatMapValues((Function<List<String>, Iterable<String>>) Functions.identity());

can you answer [this question](https://stackoverflow.com/questions/51283041/transform-java-pair-rdd-to-rdd) please — A.HADDAD, Aug 02 '18 at 09:43

Write JavaPairRdd to Csv

1 Answers1

Linked