Stack Overflow
Stack Overflow

Questions tagged [hdp]

Hortonworks Data Platform (HDP) is a distribution of an open-source Apache Hadoop data platform containing a set of projects on top of the HDFS and YARN as core layer of Hadoop.

Hortonworks Data Platform (HDP) is a distribution of an open-source Apache Hadoop data platform containing a set of projects on top of the HDFS and YARN as core layer of Hadoop.

239 questions
2
votes
0 answers

Connecting sparkly to HDP-Sandbox Spark instance

I would like to connect R instance to Spark running on HDP-Sandbox deployed on Docker on one local machine. The error message indicates that --version call on spark-submit fails. R instance packageVersion("sparklyr") # [1] ‘1.0.1’ # Set old JAVA…
Konrad
  • 17,740
  • 16
  • 106
  • 167
2
votes
0 answers

Failed to CREATE_FILE in hdfs

I have a java program that try to write a file into hdfs, but i get this error message: Failed to CREATE_FILE /data/xxx because DFSClient_attempt_1552693040212_69961_m_000000_0_1265362509_1 is already the current lease holder.
Zied Hermi
  • 229
  • 1
  • 2
  • 11
2
votes
1 answer

Hierarchical Dirichlet Process - Inferring Truncation Level

I am making use of the HDP implementation by Gensim to infer the topics of a dataset, but I have a question regarding the truncation level. Is there a way to infer the most appropriate truncation level? I have noticed that the final number of…
midnight_raver
  • 33
  • 5
2
votes
1 answer

Is it possible to run Spark (2.3) jobs on hadoop3 clusters specifically HDP 3.1 and CDH6 (beta)

Also, CDH 6 is in beta stage and do they support spark 2.3 without any bells and whistles? is it possible to run the same old spark 2.x versions (2.3 specifically) on hadoop 3 enabled CDH or Hadoop clusters? I'm interested in knowing the backwards…
Suraj N
  • 78
  • 10
2
votes
1 answer

Ambari HDP throwing FileNotFoundException for mapreduce.tar.gz while submitting the mapreduce job

After installing a new Hadoop cluster using Ambari, I tried to submit a mapreduce job, but it failed throwing an error. Error: java.io.FileNotFoundException: File does not exist:…
Nagaraj Vittal
  • 881
  • 13
  • 26
1
vote
2 answers

Ambari UI not showing versions in cluster installation

I've successfully installed Ambari server/agent 2.7.5 on my Centos 7 machine. Now i am facing an issue while installing a cluster in the install wizard at the "Select version" step. I have no versions showing in my dropdown. What can be the…
Ran
  • 11
  • 2
1
vote
0 answers

Parquet writes boolean values as null

I am trying to write a dataframe in parquet format to hdfs. The dataframe contains all values before writing but when i write it to the disk, it converts the boolean values (true,false) to null. Here is the code: val finalDF =…
Khan Saab
  • 434
  • 1
  • 8
  • 20
1
vote
0 answers

HDFS + results from hdfs fsck / are diff from hdfs dfsadmin -report

we have hadoop cluster ( Ambari platform with HDP version - 2.6.4 ) and we performed verification step in order to understand if we have under replica blocks the first verification was with: su hdfs hdfs fsck / - --> its gives the results: …
jessica
  • 2,426
  • 24
  • 66
1
vote
0 answers

error when spark sql read parquet table with text partition

1、Background: I have a hive external table A, which was created in text format when it was created. The HDFS data of the partition is also text+gz. Table A is used by thousands of sql.files. All the 5-year historical partitions of Table A may be…
jiuxian77
  • 11
  • 3
1
vote
0 answers

ambari_agent restart cause ansible crash

we have big-data Hadoop cluster based on horton-works HDP version 2.6.4 and ambari 2.6.1 version all machines are with RHEL 7.2 version in our cluster we have more then 540 machines and on all machines we have ambari-agent that communicate with…
jessica
  • 2,426
  • 24
  • 66
1
vote
0 answers

What is the prefered solution for corrupted namenode metadata

we have HDP cluster , version 2.6.5 cluster include management of two name-node ( one is active and the secondary is standby ) and 65 datanode machines we have problem with the standby name-node that not started and from the namenode logs we can see…
jessica
  • 2,426
  • 24
  • 66
1
vote
1 answer

Apache Ambari not starting in hortonworks sandbox - probably 8080 is locked

I have HDP from hortonworks sandbox in a virtual box. My host OS is mac10.15.7. I am getting the hortonworks Sandbox home page at localhost:1080. However, when i try to go Ambari UI at localhost:8080 it throws error 404 as shown below I just check…
Jio
  • 578
  • 3
  • 8
  • 27
1
vote
2 answers

HiveQL query for data marked as table column names

I work in HDP 2.6.5 platformon using Hive (1.2.1000.2.6.5.0-292) on a simple database based on data from: https://grouplens.org/datasets/movielens/100k/ I have 4 tables named: genre, movies, ratings, users as below: CREATE TABLE genre(genre string,…
uhlik
  • 105
  • 9
1
vote
1 answer

Hive - the correct way to permanently change the date and type in the entire column

I would be grateful if someone could explain here step by step what the process of changing the date format and column type from string to date should look like in the table imported via Hive View to HDP 2.6.5. The data source is the well-known…
uhlik
  • 105
  • 9
1
vote
2 answers

How to Delete Zeppelin completely from host (HDP 3.0.1)

My Zeppelin installation has corrupted after some changes and I could not find the solution to the problem. So I tried to delete Zeppelin from Ambari UI -also I tried it by using REST API- but after delete operation, all configurations and…
r00t
  • 56
  • 4
1 2
3
…
15 16