1

I am not able to get HDFS, Hive and Spark on Kylo UI under service section. enter image description here

Does anybody has any idea regarding this.

Alper t. Turker
  • 34,230
  • 9
  • 83
  • 115
Vish
  • 186
  • 3
  • 17

2 Answers2

1

Please look at the documentation for the service monitor plugins and their configuration files. There are two sections, one for CDH (Cloudera), and another for HDP (Hortonworks).

CDH

  1. Install the service monitor plugin jar at /opt/kylo/kylo-services/plugin and configure its property file at /opt/kylo/kylo-services/conf/. 

    cp /opt/kylo/setup/plugins/kylo-service-monitor-cloudera-service-<version>.jar /opt/kylo/kylo-services/plugin
chown kylo:kylo /opt/kylo/kylo-services/plugin/kylo-service-monitor-cloudera-service-<version>.jar

  2. Create service configuration file /opt/kylo/kylo-services/conf/cloudera.properties which belongs to user Kylo runs with. Populate these properties and change the values if needed as per your configuration for Cloudera Manager. Note the cloudera.services.status property which defines cluster services that Kylo will monitor.

    
clouderaRestClientConfig.username=cloudera
clouderaRestClientConfig.password=cloudera
clouderaRestClientConfig.serverUrl=127.0.0.1
clouderaRestClientConfig.port=7180
cloudera.services.status=HDFS/[DATANODE,NAMENODE,SECONDARYNAMENODE],HIVE/[HIVEMETASTORE,HIVESERVER2],YARN,SQOOP`

  3. Restart Kylo services

    service kylo-services restart

HDP

  1. Install the service monitor plugin jar at /opt/kylo/kylo-services/plugin and configure its property file at /opt/kylo/kylo-services/conf/. 

    cp /opt/kylo/setup/plugins/kylo-service-monitor-ambari-<version>.jar /opt/kylo/kylo-services/plugin
chown kylo:kylo /opt/kylo/kylo-services/plugin/kylo-service-monitor-ambari-<version>.jar

  2. Create service configuration file /opt/kylo/kylo-services/conf/ambari.properties which belongs to user Kylo runs with. Populate these properties and change the values if needed as per your configuration for Ambari. Note the ambari.services.status property which defines cluster services that Kylo will monitor.

    
ambariRestClientConfig.host=127.0.0.1
ambariRestClientConfig.port=8080
ambariRestClientConfig.username=admin
ambariRestClientConfig.password=admin
ambari.services.status=HDFS/[DATANODE,NAMENODE,SECONDARYNAMENODE],HIVE/[HIVEMETASTORE,HIVESERVER2],YARN,SQOOP

  3. Restart Kylo services

    service kylo-services restart
Jagrut Sharma
  • 4,574
  • 3
  • 14
  • 19
0

There are service plugins that need to be installed depending on whether you're on CDH or HDP.

http://kylo.readthedocs.io/en/latest/how-to-guides/ServiceMonitorPlugins.html

Cloudera Manager

Install the plugin JAR file:

cp /opt/kylo/setup/plugins/kylo-service-monitor-cloudera-service-<version>.jar /opt/kylo/kylo-services/plugin
chown kylo:kylo /opt/kylo/kylo-services/plugin/kylo-service-monitor-cloudera-service-<version>.jar

Create a /opt/kylo/kylo-services/conf/cloudera.properties configuration file:

clouderaRestClientConfig.username=cloudera
clouderaRestClientConfig.password=cloudera
clouderaRestClientConfig.serverUrl=127.0.0.1
clouderaRestClientConfig.port=7180
cloudera.services.status=HDFS/[DATANODE,NAMENODE,SECONDARYNAMENODE],HIVE/[HIVEMETASTORE,HIVESERVER2],YARN,SQOOP

Then restart Kylo:

sudo service kylo-services restart

Ambari

Install the plugin JAR file:

cp /opt/kylo/setup/plugins/kylo-service-monitor-ambari-<version>.jar /opt/kylo/kylo-services/plugin
chown kylo:kylo /opt/kylo/kylo-services/plugin/kylo-service-monitor-ambari-<version>.jar

Create a /opt/kylo/kylo-services/conf/ambari.properties configuration file:

ambariRestClientConfig.host=127.0.0.1
ambariRestClientConfig.port=8080
ambariRestClientConfig.username=admin
ambariRestClientConfig.password=admin
ambari.services.status=HDFS/[DATANODE,NAMENODE,SECONDARYNAMENODE],HIVE/[HIVEMETASTORE,HIVESERVER2],YARN,SQOOP

Then restart Kylo:

sudo service kylo-services restart
Greg Hart
  • 386
  • 1
  • 4
  • I am using apache distribution of hadoop not cloudera. How can I configure? – Vish Jan 17 '18 at 09:35
  • Kylo only has plugins for Ambari and Cloudera Manager. If you do not have either then you would need to create your own plugin to see the services in Kylo. – Greg Hart Feb 02 '18 at 23:41