1

I have a JavaPairRDD<String, String> results and I save it by calling:

results.saveAsTextFile("data")

Then I get files content like:

(www.abc.com,0.15712321 www.def.com,www.aaa.com,www.ccc.com)

Now, I want to create a table with three fields using Shark like:

url STRING  | rank DOUBLE | links STRING
www.abc.com | 0.15712321  | www.def.com,www.aaa.com,www.ccc.com

How can I do that?

Jacek Laskowski
  • 72,696
  • 27
  • 242
  • 420
MatrixZ
  • 46
  • 5

1 Answers1

0

The solution I get for now is to convert JavaPairRDD<String, String> to JavaRDD<String>

JavaRDD<String> ret = results.map(new Function<Tuple2<String, String>, String>() {
    @Override
    public String call(Tuple2<String, String> pair) {
        return pair._1() + " " + pair._2();
    }
});
ret.saveAsTextFile("data");

Then I get my files like:

www.abc.com 0.15712321 www.def.com,www.aaa.com,www.ccc.com

Now, it is easy to create a table.

MatrixZ
  • 46
  • 5