0

We are testing CDH 5.6 in our company expecting to deploy it in production in a near future, but we're stuck with a problem with no answer yet.

Currently we're working with two clusters managed by Cloudera Manager: Central and Ingestion. Our purpose is to separate services such as HDFS, Hive, Spark, Hue in Central cluster and Flume, Sqoop, Kafka in Ingestion cluster.

The problem comes when we try to connect one cluster with the other one, we want to be able to configure Sqoop2-2 or Flume-2 services (Ingestion cluster) to talk directly with the HDFS service (located in Central cluster). But, as far as we know we can't do that in Cloudera Manager, cause the only service it shows in the configuration is located in Central cluster. For example, if we try to configure Hue service (Central cluster) with Sqoop2-2 service (Ingestion), we cannot choose Sqoop2-2 service in Hue Service Configuration:

Hue service configuration for Central Cluster, only shows Sqoop2 service, not Sqoop2-2

We would like to connect this ones:

Sqoop2-2 service in Ingestion cluster

Is there a way to configure cross services in multiple clusters via Cloudera Manager or we need to modify hdfs-site.xml manually to connect Sqoop2-2 service (Ingestion) with HDFS service (Central)?

Thanks in advance for your answers.

Best regards.

Kerbatia
  • 1
  • 3

1 Answers1

0

I don't believe there's a way to do this within cloudera manager. You could potentially get that done by using most of the data from /etc/hadoop/conf/hdfs-site.xml from the storage cluster to the ingestion cluster.

What is the reasoning for splitting these off anyway? Normally you would want your daemons sitting as close to the storage as possible for the best performance.

JasonS
  • 161
  • 2
  • 14