Stack Overflow
Stack Overflow

Questions tagged [webhdfs]

WebHDFS is a REST API that supports the complete FileSystem interface for HDFS (Hadoop Distributed File System)

WebHDFS is a REST API that supports the complete FileSystem interface for HDFS (Hadoop Distributed File System). This Api is used to establish a connection to the Hadoop Data Lake from a third-party tool such as SSIS: Using WebHDFS to connect Hadoop Data Lake to SSIS

268 questions
3
votes
1 answer

uploading a file to HDFS through node.js and hdfs module

I'm trying to upload a file to HDFS using node.js. I'm using the hdfs module but the file that ends up on my HDFS is blank (stupidfile.txt isn't) var WebHDFS = require('webhdfs'); var hdfs = WebHDFS.createClient(); var fs = require('fs') var…
Havnar
  • 2,558
  • 7
  • 33
  • 62
3
votes
1 answer

vertica copy unix epoch into TIMESTAMPTZ

I want to load data into vertica from HDFS, but time_stamp value in data is in Unix epoch format and I want to load it into vertica as TIMESTAMPTZ in Vertica. Dont know how to use cast function on the fly in this case. Can anyone help me with this…
roy
  • 6,344
  • 24
  • 92
  • 174
3
votes
1 answer

Graphs (line chart, pie chart) in web application with mapreduce programming

hello friends i am developing a web application where i need to write a map reduce code for processing a data from Hadoop Distributed file system and draw various graphs on the basis of reducer output so i am developing the application in Java in…
Sumeet
  • 43
  • 5
3
votes
1 answer

Retrieving tasktracker logs for a particular job programatically

Hi am working with OozieClient API. I need to retrieve the task tracker logs for a particular workflow job using the OozieClient API. If not with OozieClient API any other way using a program is also fine. As of now with the OozieClient i am able to…
dnivra
  • 749
  • 4
  • 12
  • 30
3
votes
2 answers

how to cache images from hadoop and also how to hide port number given in the url

http://testing:50070/webhdfs/v1/Test/asaw4zds_ssdf4_ht35-9a1a-4a7b-9n.jpg?op=OPEN I am fetching the above image through hadoop using webhdfs i want to cache this image in browser how to do that is there any kind of mechanism to cache images coming…
tina
  • 312
  • 1
  • 4
  • 18
2
votes
1 answer

High availability HDFS client python

In HDFSCLI docs it says that it can be configured to connect to multiple hosts by adding urls separated with semicolon ; (https://hdfscli.readthedocs.io/en/latest/quickstart.html#configuration). I use kerberos client, and this is my code - from…
Kallie
  • 147
  • 9
2
votes
2 answers

Hadoop Can't access datanode without using the IP

I have the following system: Windows host Linux guest with Docker (in Virtual Box) I have installed HDFS in Docker (Ubuntu, Virtual Box). I have used the bde2020 hadoop image from Docker Hub. This is my docker-compose: namenode: image:…
David Zamora
  • 383
  • 1
  • 4
  • 15
2
votes
0 answers

UnsupportedFileSystemException when trying to write to to hdfs via webhdfs in delta format

I want to write data in delta format to a hdfs which is accessible via webhdfs. Writing in parquet format works like charm but when I set the format to delta I get the following error 20/03/25 18:12:18 WARN DeltaLog: Failed to parse…
user2811630
  • 445
  • 3
  • 11
2
votes
1 answer

Is there a way to only display a file with webhdfs REST API?

Hello StackOverflow community. I have been running into a problem lately regarding webhdfs REST API. I have a servlet inside an application calling Apache Knox to access HDFS and HBase which prevents me from using configuration files and Hadoop base…
Philippe B.
  • 485
  • 4
  • 18
2
votes
4 answers

How to enable cors origin allow in Webhdfs - HDFS - Hadoop - Origin http://localhost:4200 is not allowed by Access-Control-Allow-Origin

When I try to access Webhdfs from my Angular 6 app I get the error shown below. It seems to me that I tried almost everything including changing settings in core-site.xml and hdfs-site.xml unfortunately without positive result. Obviously, most…
thedbogh
  • 614
  • 2
  • 10
  • 26
2
votes
2 answers

Hue UI is not accessible from a remote host

I'am trying to use Hue as a file browser for HDFS. So for that I have clone the hue repository and build the app with the following commands given in README.md of the hue repository. git clone https://github.com/cloudera/hue.git cd hue make…
Hansika Weerasena
  • 3,046
  • 1
  • 13
  • 22
2
votes
0 answers

how do you enable webhdfs?

I am trying to use webhdfs. I have place these lines to hdfs-site.xml file: dfs.webhdfs.enabled true I did a stop-all.sh and start-all.sh on hadoop cluster. I am trying to see if I can…
user1471980
  • 10,127
  • 48
  • 136
  • 235
2
votes
0 answers

logstash error with kerberos while inserting to webhdfs from kafka

We have logstash pipeline as below to read from kafka topic to webhdfs Input = kafka topic Output = webhdfs I have extracted from source (logstash-6.2.4.tar.gz) and did the testing Our hadoop cluster is kerberos enabled cluster I am getting…
Anushke Hewawaitharana
  • 51
  • 1
  • 5
2
votes
2 answers

Directory size in Azure Data Lake Store

Is there a simple API using which I can get the size of an ADLS directory? preferably something in C#, but it's not a must.
AK_
  • 7,981
  • 7
  • 46
  • 78
2
votes
2 answers

WEBHDFS REST API to copy/move files from windows server/local folder/desktop to HDFS

Using WEBHDFS REST API calls can i transfer or copy the files from Windows machine(i.e. windows server or windows local folder or desktop) to Hadoop-HDFS file system? If yes any sample command info? I have tried and i was able to do…
Sai
  • 97
  • 1
  • 2
  • 17
1 2
3
…
17 18