0

I have a JavaPairRDD of (String, Iterable[(String, String)]) and I want this to be converted into an JavaPairRDD of (String, RDD[String, String]), so that i can use the reduceByKey function to the internal JavaPairRDD.

In this example, I have a JavaPairRDD where key is a specific model number and value is an Iterable of tupled item numbers and the price of that item.

Here is an example of the JavaPairRDD:

("Model1", Iterable(("1234", "55.00"), ("5678", "52.50"), ("91011", "551.65"))
("Model2", Iterable(("6546", "55.00"), ("6798", "1255.05"), ("98797", "500.65"))

After applying the reduceByKey function I would like the result to look like this:

("Model1", JavaPairRDD(("1234", "55.00"), ("5678", "52.50"), ("91011", "551.65"))
("Model2", JavaPairRDD(("6546", "55.00"), ("6798", "1255.05"), ("98797", "500.65"))

A very similar thread converted this RDD using the following Scala code: How to convert an Iterable to an RDD

("To", List(("Tom",50),("Tod","30"),("Tom",70),("Tod","25"),("Tod",15))
("Ja", List(("Jack",50),("James","30"),("Jane",70),("James","25"),("Jasper",15))

rdd.flatMap{case(key, list) => list.map(item => ((key,item._1), item._2))}
   .reduceByKey(_+_)
   .map{case((key,name),hours) => (key, List((name, hours)))}
   .reduceByKey(_++_)

Is there a similar way to do this in Java?

Thanks for your help.

slash
  • 9
  • 1
  • I assume the problem is - you don't know scala? Basically you need to write the same code in Java. The same RDD transformations. – Vladislav Varslavans Jan 09 '20 at 06:30
  • @VladislavVarslavans I realize that's my problem. I do not know scala enough to translate these transformations to Java or if it's even possible to do it the same way. – slash Jan 09 '20 at 17:45

0 Answers0