StreamInputFormat for mapreduce job

Question

I have an application that connects to a remote system and transfers data from it using sftp protocol. I want to use a mapreduce job to do the same. I would need a input format that reads from an input stream . I have been going through the docs for HStreamInputFormat and StreamInputFormat but my hadoop-2.0 doesnt seem to support these classes . How do I proceed ? Any links to tutorials or examples of reading from input streams using input formats ?

score 0 · Answer 1 · answered Dec 12 '12 at 05:34

0

If those SteamInputFormats doesn't support your need then you better off writing your own InputFormat with your customized needs. Please read this tutorial to learn how to write your own custom InputFormat and RecordReader.

answered Dec 12 '12 at 05:34

shazin

21,379
3
54
71

Thanks :) Cant write a InputFormat here because I don't know how to handle input streams as path to mapper and hadoop doen't have any base classes in place . – RadAl Dec 13 '12 at 06:12

StreamInputFormat for mapreduce job

1 Answers1