I want to store JavaRDD as a sequence file instead of textfile. But i don't see any Java API for that. Is there a way for this? Please let me know. Thanks!
Asked
Active
Viewed 3,889 times
2
-
Just use Scala :) You will make your life so much easier. – samthebest Jun 20 '14 at 13:08
1 Answers
4
Maybe JavaPairRDD has a method .saveAsSequenceFile if not I think you can use .saveAsHadoopFile and have SequenceFileOutputFormat.class as the format param. It seems to be explained a bit here: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SequenceFile-Java-API-Repeat-Key-Values-td353.html
samthebest
- 30,803
- 25
- 102
- 142
-
Thanks @samthebest for pointing this approach, but I tried and faced with an issue, here's my question: https://stackoverflow.com/questions/44187041/could-not-find-a-serializer-for-the-value-class-org-apache-hadoop-hbase-client?noredirect=1#comment78584021_44187041, do you have any ideas please? – Fisher Coder Aug 25 '17 at 21:02