2

I am trying to start the spark-job-server on my linux machine. I did the following:

  1. Installed the cloudera distriution CDH(5.x) and gotten it up and running
  2. Downloaded spark-job-server from the above mentioned github
  3. Extracted the project into some folder
  4. Installed sbt-0.13.8 and set path variable and able to run the sbt command
  5. Went to the project directory (spark-job-server/) and ran the sbt command
  6. From SBT shell, I have typed "re-start", which is suppposed to start the spark-job-server.

Then, I got this Exception:

  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[warn]  :: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[warn]  :: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[warn]  :: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn]  Note: Unresolved dependencies path:
[warn]          org.apache.hadoop:hadoop-mapreduce-client-app:2.2.0
[warn]            +- org.apache.hadoop:hadoop-client:2.2.0
[warn]            +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn]            +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn]          org.apache.hadoop:hadoop-yarn-api:2.2.0
[warn]            +- org.apache.hadoop:hadoop-client:2.2.0
[warn]            +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn]            +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn]          org.apache.hadoop:hadoop-mapreduce-client-core:2.2.0
[warn]            +- org.apache.hadoop:hadoop-client:2.2.0
[warn]            +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn]            +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn]          org.apache.hadoop:hadoop-mapreduce-client-jobclient:2.2.0
[warn]            +- org.apache.hadoop:hadoop-client:2.2.0
[warn]            +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn]            +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn] Credentials file /root/.bintray/.credentials does not exist
[warn] Credentials file /root/.bintray/.credentials does not exist
[warn] Credentials file /root/.bintray/.credentials does not exist
[trace] Stack trace suppressed: run last job-server-api/*:update for the full output.
[trace] Stack trace suppressed: run last akka-app/*:update for the full output.
[trace] Stack trace suppressed: run last job-server-api/*:update for the full output.
[trace] Stack trace suppressed: run last akka-app/*:update for the full output.
[error] (job-server-api/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] (akka-app/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] (job-server-api/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] (akka-app/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] Total time: 5 s, completed Aug 6, 2015 1:29:26 PM

Some more information:

  • I have manually downloaded these dependencies and placed them
  • when I rerun restart command again still sbt is not able to find the same dependency which i have copied in .ivy2 folder

How can I fix this?

Sumurai8
  • 20,333
  • 11
  • 66
  • 100
Pradeep Poojari
  • 155
  • 1
  • 1
  • 9
  • 4
    To help other people to help you, it is necessary that you post a minimal, correct, verifiable, example that others can reproduce that demonstrates the problem. You can click the "edit" link below your question to add the necessary details. – Paul Aug 06 '15 at 03:18
  • You are getting the exception when you run *what*, exactly? No one knows, from what is written, which of the steps on the github page is causing that problem. We also don't know which steps you have completed. – Paul Aug 06 '15 at 03:37
  • This is the step "Development mode" in the Git hub page which I mentioned in question. When I run reStart command from sbt shell i am getting this error. – Pradeep Poojari Aug 06 '15 at 04:46
  • looks like you can only fetch these JAR files if you have proper credentials for the repo. do you have these credentials? – Ashalynd Aug 06 '15 at 17:18
  • I was able to resolve most of the dependency, but not this. Yes I have a proper credentials. – Pradeep Poojari Aug 07 '15 at 03:18
  • I have downloaded these dependency separately and placed it inside .ivy2 folder. that not resolving my problem – Pradeep Poojari Aug 07 '15 at 03:24
  • One more thing we have used to HTTP proxy here to download dependency – Pradeep Poojari Aug 07 '15 at 03:26

0 Answers0