I want a RDD from a spark job to be persisted, so that it can be used by all subsequent jobs using Spark Job Server. Here is what i have tried:
Job 1:
package spark.jobserver
import com.typesafe.config.{Config, ConfigFactory}
import…
I have configured Spark jobserver to run on YARN.
I am able to send spark jobs to YARN but even after the job finishes it does not quit on YARN
For eg:
I tried to make a simple spark context.
The context is reflecting in jobserver but YARN is still…
I'm trying out the Spark job server - specifically, the docker container option. I was able to run the WordCountExample app in spark local mode. However, I ran into an exception when I tried to point the app to a remote Spark master.
Following are…
I've just got a weird problem when I attempt to instantiate the Spark Jobserver.
It seems it's looking at a wrong directory. Does anybody know something about this?. Maybe I am missing something but the directory /usr/share/dse/bin/dse doesnt't…
I am running spark-job-server 0.5.3 from ooyala. I have followed their official documents and it works fine when it is started by sbt using reStart command. But i can't
make it work using server_start.sh script.
unable to run it on a standalone…
I'm trying to start a spark jobserver, here are the steps I'm following:
I configure the local.sh based on the template.
Then I run ./bin/server_deploy.sh and it finishes without any error.
Configure local.conf.
Run ./bin/server_start.sh in the…
So I'm trying to create a local instance of spark jobserver to test jobs on and I can't even get it to run.
So the first thing I do when I got into my vagrant instance is I start spark. I know this works because I submit jobs to spark with the…
I want to deploy Spark Job Server (in a Docker container) on a different host to the Spark Master. However the server_start.sh script seems to assume that it is being run on the same machine as the Spark Master. E.g.:
if [ -z "$SPARK_CONF_DIR" ];…
I'm experimenting with the JobServer and would like to use it in our production environment.
I want to use mllib and spark-jobserver together, but i got a error (at the spark-jobserver, when a job is sended).
job-server[ERROR] Uncaught error from…
I have a problem with Apache job-server and my .jar with SparkJob.
I have VirtualBox with DataStax. There are Cassandra and Spark. I install Apache job-server from git job-server. I want run examples so I write sbt job-server-tests/package and next…
We are using DataStax Enterprise which has support for Spark and Spark Job server.
We have 3 node olap casandra cluster with each node configuration of 8 core processor and 32G of RAM
As far our spark job is concerned with above configuration I was…
I have streaming query with trigger once.
But as per spark documentation once all records are read from event-hub,spark job should stop.
But this is not…
I'm trying to understand the spark cache manager behavior as I deployed my test code to spark job server to have long running context and want to test the behavior by executing the same job multiple time after each other to see how caching is.
val…
I'm currently working with spark-jobserver, and when the spark-jobserver goes down my app just stop working but I don't get notified
There is a health check to spark-jobserver?
This is a simple Java code as a Spark job, mentioned in Spark job-server github repo
package com.sample.wordcount;
import com.typesafe.config.Config;
import com.typesafe.config.ConfigFactory;
import…
