I"m trying to convert term-frequency values into mahout vector representation, so that I can use LDA on the given vectors. I'm following the mahout wiki where the code snippest suggest how to convert exisitng vectors to Mahout Vectors.
https://cwiki.apache.org/MAHOUT/creating-vectors-from-text.html
Here's my code, I'm getting a NullPointerException in the place of creating the VectorWriter. The apache cwiki suggests using,
VectorWriter vectorWriter = SequenceFile.createWriter(filesystem, configuration, outfile, LongWritable.class, SparseVector.class);
But, I dont see the SequenceFile.createWriter in the org.apache.hadoop.io.SequenceFile;
This is the complete code segment.
fs = FileSystem.get(conf);
//I"m using SeqeunceFile.Writer because SequenceFile.createWriter is not available.
VectorWriter vectorWriter = (VectorWriter) new SequenceFile.Writer(fs, conf, path, LongWritable.class, RandomAccessSparseVector.class);
ArrayList<Vector> weights = new ArrayList<Vector>();
BufferedReader buffer = new BufferedReader(new FileReader("/home/hadoop/LDATest/LDAData/test"));
String line = null;
while((line = buffer.readLine()) != null)
{
String[] data = line.split(" "); // split the term,weight data
Vector weightVector = new RandomAccessSparseVector(1,1);
weightVector.setQuick(0, Double.parseDouble(data[1])); // add the weight
weights.add(weightVector);
}
vectorWriter.write(new VectorIterable(weights));
This the error,
Exception in thread "main" java.lang.NullPointerException at org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73) at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:910) at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:843) at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:831) at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:823) at kbsi.ideal.LDATest.iterableTest(LDATest.java:161) at kbsi.ideal.LDATest.main(LDATest.java:194)
I really appreciate your help on this. Thanks
