0

We are trying to stand up a sandbox/evaluation instance of Zeppelin on a 4-node CentOS 6 cluster with Vagrant and having some issues with dependencies in the build process. Here is the high level script we’re running.

(Have tried running this as privileged account and as a user, with the same results.)

Recreate Steps

  • Install Hadoop 2.7.0 from Binary
  • Install Spark 1.4.0 from Binary
  • Install Maven 3.3.3 from Binary

  • Run the following:

    curl --silent --location  https://rpm.nodesource.com/setup | bash -
yum install -y nodejs
git config --global url."https://".insteadOf git://
git clone https://github.com/apache/incubator-zeppelin.git
cd incubator-zeppelin
npm update -g npm
npm install -g grunt-cli
npm install -g grunt 
npm install -g bower
mvn clean package -Pspark-1.4 -Dhadoop.version=2.7.0 -Phadoop-2.6 -DskipTests

Stack Trace

Here’s an example of the stack trace we are receiving when we get to the step for building zeppelin-web:

==> node1: Downloaded: https://repo.maven.apache.org/maven2/com/google/guava/guava/10.0.1/guava-10.0.1.jar (1467 KB at 154.5 KB/sec)
==> node1: [INFO] Installing node version v0.10.18
==> node1: [INFO] Creating temporary directory /home/vagrant/incubator-zeppelin/zeppelin-web/node_tmp
==> node1: [INFO] Downloading Node.js from http://nodejs.org/dist/v0.10.18/node-v0.10.18-linux-x64.tar.gz to /home/vagrant/incubator-zeppelin/zeppelin-web/node_tmp/node.tar.gz
==> node1: [INFO] No proxy was configured, downloading directly
==> node1: [INFO] Extracting Node.js files in node_tmp
==> node1: [INFO] Unpacking /home/vagrant/incubator-zeppelin/zeppelin-web/node_tmp/node.tar.gz into /home/vagrant/incubator-zeppelin/zeppelin-web/node_tmp
==> node1: [INFO] Moving node binary to /home/vagrant/incubator-zeppelin/zeppelin-web/node/node
==> node1: [INFO] Deleting temporary directory /home/vagrant/incubator-zeppelin/zeppelin-web/node_tmp
==> node1: [INFO] Installed node locally.
==> node1: 
==> node1: [INFO] Installing npm version 1.3.8
==> node1: [INFO] Downloading NPM from http://registry.npmjs.org/npm/-/npm-1.3.8.tgz to /home/vagrant/incubator-zeppelin/zeppelin-web/npm.tar.gz
==> node1: [INFO] No proxy was configured, downloading directly
==> node1: [INFO] Extracting NPM files in node/
==> node1: [INFO] Unpacking /home/vagrant/incubator-zeppelin/zeppelin-web/npm.tar.gz into /home/vagrant/incubator-zeppelin/zeppelin-web/node
==> node1: [INFO] Installed NPM locally.
==> node1: [INFO] 
==> node1: [INFO] --- frontend-maven-plugin:0.0.23:npm (npm install) @ zeppelin-web ---
==> node1: [INFO] Running 'npm install --color=false' in /home/vagrant/incubator-zeppelin/zeppelin-web
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-angular-templates
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-dom-munger
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-autoprefixer
==> node1: [ERROR] npm http GET https://registry.npmjs.org/bower
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-concurrent
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-contrib-connect
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-contrib-clean
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-contrib-concat
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-contrib-copy
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-contrib-cssmin
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-contrib-htmlmin
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-contrib-jshint
==> node1: [ERROR] npm http GET https://registry.npmjs.org/grunt-contrib-uglify

...

It goes on like that for a while, then fails the whole build.

TL;DR

Is there something missing in terms of prerequisites, or is there some trickery to getting it to build on CentOS 6 that’s not in the documentation? :) Also, this has been posted to the Zeppelin User Board, so feel free to answer there too/instead.

parrottsquawk
  • 83
  • 8

2 Answers2

2

Was able to reproduce error using above steps. After looking at this link: link

I was able to get a Successful Build using:

mvn clean install -DskipTests

Hope you find this helpful.

Vasilis Vagias
  • 416
  • 4
  • 6
  • Not to jump to conclusions but if you don't require the package goal can you try running – Vasilis Vagias Jul 07 '15 at 18:33
  • > mvn clean install -DskipTests I got a build failure when running package but SUCCESS with install – Vasilis Vagias Jul 07 '15 at 18:40
  • luckily https://zeppelin.incubator.apache.org/docs/install/install.html was tested on Cent OS and MacOSX so should work... :) – Vasilis Vagias Jul 07 '15 at 18:42
  • That was one of the first workarounds we tried, but it also crashed at zeppelin-web. The goals are for building hadoop and spark, but zeppelin-web is crashing with bower, grunt, and NPM issues. – parrottsquawk Jul 07 '15 at 19:02
  • That is very strange. what are the MAVEN_OPTS in your env? I did have a fail at the zeppelin-web and this got me through that point. export MAVEN_OPTS="-Xms1024m -Xmx4096m -XX:PermSize=1024m" Just throwing ideas around. – Vasilis Vagias Jul 07 '15 at 19:10
  • 1
    Darn! So close :) I shouldn't have assumed you were running with root privileges. Glad you figured it out! – Vasilis Vagias Jul 10 '15 at 14:38
  • I tried it with root privileges and without root privileges; the trick was to run it as a user that WASN'T root AND with sudo AND to start it again when the zeppelin-web project failed the first time. go figure... /facepalm – parrottsquawk Jul 10 '15 at 22:18
  • 1
    I know that story all to well! Glad you figured it out! Few more points and I'll be able to up vote your answer :) for now +1 haha – Vasilis Vagias Jul 15 '15 at 20:29
1

The solution to my problem was to run the following as a non-root user:

sudo env "PATH=$PATH" mvn clean install -Pspark-1.4 -Pspark.version=1.4.0 -Phadoop.version=2.7.0 -DskipTests
sudo env "PATH=$PATH" mvn clean install -Pspark-1.4 -Pspark.version=1.4.0 -Phadoop.version=2.7.0 -DskipTests -rf :zeppelin-web
parrottsquawk
  • 83
  • 8