Is there a difference between webhdfs:// and hdfs:// in terms of underlying protocol?

Question

Right now I am ingesting logs via HttpFS, which is falling apart at the sheer volume. I cannot run JVM on my edge for performance reasons. So I was thinking of using the native protocol for ingest (I've heard it's either IPC or raw network socket magic)

webhdfs HttpFS underlying

hadoop fs -put localfile -fs webhdfs://datanode:14000/hadoopfile

hdfs ?? what's underlying

hadoop fs -put localfile hdfs://namenode/hadoopfile

score 0 · Answer 1 · answered Jul 02 '13 at 00:52

0

IMHO, there is actually no difference as the underlying protocol in both the cases is TCP/IP.

answered Jul 02 '13 at 00:52

Tariq

34,076
8
57
79

2

I think he means REST (webhdfs://) vs RPC (hdfs://) – Mike Park Jul 02 '13 at 04:21
Oh..Perhaps..I was actually caught up by 'underlying'. – Tariq Jul 02 '13 at 05:04
So there is a difference? One is HTTP, and the other binary? – Carl Sagan Jul 23 '13 at 19:35

Is there a difference between webhdfs:// and hdfs:// in terms of underlying protocol?

webhdfs HttpFS underlying

hdfs ?? what's underlying

1 Answers1