1

I'm trying to install Spark2 in my cloudera cluster (evaluation version) following the cloudera's instructions to install this component. I downloaded the CSD, installed it and using the parcel downloaded the component, distribute it but when I try to activate it I'm having this message:

CDH (5.8 and higher) parcel required for SPARK2 (2.2.0.cloudera1-1.cdh5.12.0.p0.142354) is not available.

This is the information of the cluster:

Version: Cloudera Enterprise Data Hub Edition Trial 5.12.1 (#6 built by jenkins on 20170818-0807 git: 9bdee611802535491d400e03c98ef694a2c77d0a)

Java VM Name: Java HotSpot(TM) 64-Bit Server VM

Java VM Vendor: Oracle Corporation

Java Version: 1.7.0_67

CSD

SPARK2_ON_YARN-2.2.0.cloudera1.jar

Parcel

http://archive.cloudera.com/spark2/parcels/2.2.0.cloudera1/

I'm thinking it could be because my CDH version (5.12.1) and the version of the last spark2 parcel (cdh5.12.0) but I don't find any other package for cdh5.12.1 and my next question is: for cdh5.13.0 which is the spark2 parcel?

Cœur
  • 37,241
  • 25
  • 195
  • 267
gleX
  • 595
  • 2
  • 8
  • 22

2 Answers2

2

The error message is misleading. The real issue is that your cluster is running on Java 1.7. Spark 2.2 is only supported on Java 1.8. Upgrade Java on your cluster and you should be able to install the Spark 2.2 parcel.

Wing Yew Poon
  • 21
  • 1
  • Hi, you are rigth, cloudera installed jdk 1.7 by default but I change the jvm version of all hosts to the jdk 1.8, so all applications are running over jdk 1.8 and just the cloudera manager is running with jdk 1.7 (last time I tried to change de version, the cluster just crash, this why I just change version for host and not fo cloudera manager) ... I'm not sure what java version its taken by default in an instalation, I suppose it take the java version defined in the host (in my case 1.8) – gleX Nov 02 '17 at 11:24
2

Finally solved. The problem was that I need to update de cdh core, after update, spark 2 just works fine.

gleX
  • 595
  • 2
  • 8
  • 22
  • I am also facing same issue, can you please provide the steps to fix it ? – Ajeet Nov 04 '17 at 07:44
  • The first thing you have to do, is to check the "parcels" modules to identify any available update for CHD 5.X, if there is an update for your cluster you must install it following the steps: Download, Distribute, Activated. With your CDH version updated, you can download, distribute and activate the spark2 from parcel. Thats all I did. – gleX Nov 08 '17 at 10:32
  • Thanks @gleX, I resolved it after installing CDH update. – Ajeet Nov 08 '17 at 11:19
  • Thanks, that was also exactly my problem. – gil.fernandes Nov 17 '17 at 17:20