Stack Overflow
Stack Overflow

Questions tagged [hdp]

Hortonworks Data Platform (HDP) is a distribution of an open-source Apache Hadoop data platform containing a set of projects on top of the HDFS and YARN as core layer of Hadoop.

Hortonworks Data Platform (HDP) is a distribution of an open-source Apache Hadoop data platform containing a set of projects on top of the HDFS and YARN as core layer of Hadoop.

239 questions
3
votes
0 answers

How to manually / programatically add arbitrary data files to apache atlas?

Is there any way to add arbitrary data in HDFS to apache atlas? Having installed HDP 3.1 for evaluation, this appears to not be possible (eg. only data that is sqooped in, placed in a hive table, or some other narrow set of atlas-visible…
lampShadesDrifter
  • 3,925
  • 8
  • 40
  • 102
2
votes
1 answer

Name Node is going to Safe mode repeatedly

We are using HDP 3.0 in my project which is in 2 node. For every 20-30 days name node is going to safe mode with below errors. For every time I am manually executing below command to leave safe mode. hdfs dfsadmin -safemode leave below are the logs…
satish pujara
  • 219
  • 1
  • 11
2
votes
1 answer

Hive 3.1 cannot create external table that copy schema from internal/managed table

we try to make an external hive table which its schema is similar from existing internal/managed table and the data for ecternal will be inserted next step. Below the code to create external table with copy schema of managed table CREATE EXTERNAL…
m hanif f
  • 406
  • 1
  • 7
  • 20
2
votes
1 answer

HDP sanbox: yum install is throwing 403 error

I am trying to setup HDP sandbox and work on a big data project.But when trying to do yum install, I am getting below error http://s3.amazonaws.com/dev.hortonworks.com/ambari/centos6/2.x/BUILDS/2.4.0.0-1225/repodata/repomd.xml: [Errno 14] PYCURL…
merla
  • 489
  • 1
  • 5
  • 12
2
votes
0 answers

ambari yarn memory show more then 100% usage

we have ambari cluster with HDP version 2.6.5 from ambari dashboard we see the following we can see that YARN show 111% - this isn't logical value instead to show X from 100% what could be the problem here ?
Judy
  • 1,595
  • 6
  • 19
  • 41
2
votes
0 answers

Presto + Error opening Hive split when running presto query’s

we have Hadoop cluster based on HDP - version 2.6.4 and ambari , Note - Hadoop cluster include the hiveserver2 and hivemetastore we are running presto queries , for searching data on many partitions in HDFS filesystem the issue , is strange because…
jessica
  • 2,426
  • 24
  • 66
2
votes
1 answer

cdh quickstart 6.3.2 download link

Currently cloudera has stopped cloudera manager express and also unable to download cdh latest version. cdh 6.3.2 is last open source version. I looking for its VM for virtualbox download link. Anybody has cdh 6.3.2, please share it.
avikm
  • 511
  • 1
  • 7
  • 23
2
votes
1 answer

Kafka console consumer to read avro messages in HDP 3

Trying to consume kafka Avro messages from console consumer and not exactly sure how to deserialize the messages. sh /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --bootstrap-server localhost:6667 --topic test --consumer.config…
William R
  • 739
  • 2
  • 13
  • 34
2
votes
0 answers

Spark Hive Warehouse Connector Dependency issues

So I am trying to enhance my Spark application in Scala 2.11 to read data from HDInsight (HDP) using the Hive Warehouse Connector. The problem is that for whatever reason I am not able to import any version of the dependency required to perform…
bmcristi
  • 73
  • 7
2
votes
2 answers

spark HWC cannot write to an existing table

In HDP 3.1.0, HWC hive-warehouse-connector-assembly-1.0.0.3.1.0.0-78.jar, I cannot append (or overwrite) to an existing table depending on the database. I tested on one datase called DSN, it works and on another database called CLEAN_CRYPT it…
Jeannot77680
  • 49
  • 6
2
votes
1 answer

How to get document-topics using models.hdpmodel – Hierarchical Dirichlet Process in gensim

I just study gensim for topic modeling. when I use lda_model = gensim.models.ldamodel.LdaModel(...) the result lda_model has two functions: get_topics() and get_document_topics(). I can find the topic-word and document-topics by them. But, I want…
Feng Chen
  • 2,139
  • 4
  • 33
  • 62
2
votes
4 answers

How to write a table to hive from spark without using the warehouse connector in HDP 3.1

when trying to use spark 2.3 on HDP 3.1 to write to a Hive table without the warehouse connector directly into hives schema using: spark-shell --driver-memory 16g --master local[3] --conf spark.hadoop.metastore.catalog.default=hive val df =…
Georg Heiler
  • 16,916
  • 36
  • 162
  • 292
2
votes
0 answers

How to create delegated token in Metastore and HiveServer2?

I've got HDP3 kerberized cluster. The question is - how can I create delegation token for user that doesn't have keytab? With that user I want to retrieve information from Metastore and run SQL queries on Hive tables. Property…
Markiza
  • 444
  • 1
  • 5
  • 18
2
votes
0 answers

HortonWorks HDP 2.6.5 or 2.5.0 : "My User Web Root" displayed instead of AMBARI

I'm trying to follow the Udemy course on Hadoop, so I've no knowledge of Hadoop yet ;) I've downloaded HDP_2.6.5_virtualbox_180626.ova (and I've also tested with 2.5.0 version) on my macbook (10.14.5) double click it to open the latest version of…
Thomas
  • 1,231
  • 14
  • 25
2
votes
1 answer

What is the default Ranger admin username and password for hortonworks/sandbox-hdp:3.0.1

I use hortonworks/sandbox-hdp:3.0.1 and hortonworks/sandbox-proxy:1.0. as a docker containers. Tried to connect to ranger using admin/admin however without any success - The username or password you entered is incorrect. Tried to change ranger…
Sergey Konotop
  • 121
  • 1
  • 13
1
2
3
…
15 16