Stack Overflow
Stack Overflow

Questions tagged [hdp]

Hortonworks Data Platform (HDP) is a distribution of an open-source Apache Hadoop data platform containing a set of projects on top of the HDFS and YARN as core layer of Hadoop.

Hortonworks Data Platform (HDP) is a distribution of an open-source Apache Hadoop data platform containing a set of projects on top of the HDFS and YARN as core layer of Hadoop.

239 questions
1
vote
0 answers

ambari + namenode HA and hdfs zkfc -formatZK

we have production HDP cluster , HDP version - 2.6.5 Ambari version - 2.6.1 We are performing now the change hostname configuration according to the document -…
Judy
  • 1,595
  • 6
  • 19
  • 41
1
vote
2 answers

while Installing `yum install python -pip` getting the below error in HDP 2.5

http://s3.amazonaws.com/dev.hortonworks.com/ambari/centos6/2.x/BUILDS/2.4.0.0-1225/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 Forbidden" Trying other mirror. To address this issue please refer to the…
Anurag Jangde
  • 31
  • 5
1
vote
0 answers

Spark-submit fails when using kafka structured streaming in pyspark 2.3.1

Spark-submit fails when using kafka structured streaming in pyspark 2.3.1 But the same code works in pyspark command, so l want to know how to solve it from pyspark.sql.types import * from pyspark.sql import SparkSession …
Lee. YunSu
  • 416
  • 6
  • 21
1
vote
2 answers

hue installation - make apps - failed

I am following this guide for installing HUE on HDP 3.1.4 , centos 7 https://gethue.com/configure-ambari-hdp-with-hue/ at building : sudo make apps ,it fails with : npm ERR! Linux 3.10.0-1127.13.1.el7.x86_64 npm ERR! argv "/usr/bin/node"…
Cosmin Chauciuc
  • 65
  • 5
1
vote
2 answers

Unable to read hive table content after writing the same through spark shell in hdp 3.0.1 using hive warehouse connector

I have written table in hive using hive warehouse connector. But I am unable to read the content of the same after writing. Below are the details of commands used: Commands to write the…
Divya Jain
  • 23
  • 7
1
vote
1 answer

Nifi Content Vs Attribute Modification Techniques

In Nifi we can design a flow in two ways : Content Based Modification (UpdateContent) - In this approach we are directly modifying the content of flowfiles . With this at each stage , the flowfile content will get persisted in flow file…
Aviral Kumar
  • 814
  • 1
  • 15
  • 40
1
vote
0 answers

execute flink 1.10 on a HDP 3.1 cluster to access hive tables

I want to use apache flink on a secure kerberized HDP 3.1 cluster, but am still stuck with the first steps. The latest release was downloaded and unzipped (https://flink.apache.org/downloads.html#apache-flink-1101) Now, I try to…
Georg Heiler
  • 16,916
  • 36
  • 162
  • 292
1
vote
0 answers

Spark + Hive : Number of partitions scanned exceeds limit (=4000)

We upgraded our Hadoop Platform (Spark; 2.3.0, Hive: 3.1), and I'm facing this exception when reading some Hive tables in Spark : "Number of partitions scanned on table 'my_table' exceeds limit (=4000)". Tables we are working on : table1 : external…
user2695912
  • 33
  • 1
  • 7
1
vote
1 answer

Ambari server can not register ambari agents

I want to install hadoop cluster with ambari server, but following error occurs Registering with the server... Registration with the server failed. Steps I have manually installed ambari agents on two slave hosts Then I have ran agent services on…
Grig Vardanyan
  • 11
  • 1
1
vote
1 answer

How to clear "Reserved Space for Replicas" without restart Hadoop services

How to find why "Reserved Space for Replicas" constantly increasing and how to limit space for this type of cache? We found that the "Reserved Space for Replicas" exceeds the Non DFS used space (last month). But we didn't find why :( We know how to…
zh_
  • 95
  • 8
1
vote
1 answer

hdfs + namenode + edit files increasing with huge size and how to limit the size of edit files

we have HDP cluster with 7 datanodes machines under /hadoop/hdfs/namenode/current/ we can see more then 1500 edit files each file is around 7M to 20M as the following 7.8M …
jessica
  • 2,426
  • 24
  • 66
1
vote
0 answers

Adding another Namenode to HDP HA Cluster

I have HDP3 High Availability enabled on the cluster. I have tested it and everything seems to be fine. Current Setup is nn1 (Active), nn2 (Passive) Now, I am trying to add another NameNode, which makes this setup from NN1, NN2 to NN1, NN2…
s_mj
  • 530
  • 11
  • 28
1
vote
1 answer

Normal write to apache-kafka but not able to read topic data in spark job

HDP 2.6.5 no kerb I am running kafka and spark in cluster. I am writing data to a particular topic in kafka and trying to run a python code for read and show data from kafka. However, reading freezes and does not throw an error. Starting…
Faringot
  • 29
  • 5
1
vote
1 answer

is it possible to Install Apache Bigtop Stack with Ambari

Can any one provide tutorial or steps to installing Apache BigTop stack using Ambari. I have tried to learn the following info https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+1.3.0+Release but what are the steps to install each of hadoop…
jessica
  • 2,426
  • 24
  • 66
1
vote
2 answers

ambari cluster + poor connection between ambari-agent to ambari server

we have ambari cluster with 872 data-nodes machines , when ambari version is 2.6.x we have for now some network problem , after long investigation we found that , ambari agent that runs on some machine not communicate well with the ambari…
jessica
  • 2,426
  • 24
  • 66
1 2 3
…
15 16