SparkR sql Context Error after initiating spark R Context job

Question

I have installed the sparkR package and I am able to run other computation jobs like pi count or numbers of word counts in a document .But when I am trying to initiate sparkRSql job,it gives an error .Can anyone help me out ? I am using R version 3.2.0 and Spark 1.3.1

> library(SparkR)
> sc1 <- sparkR.init(master="local")
Launching java with command  /usr/lib/jvm/java-7-oracle/bin/java   -Xmx1g -cp '/home/himaanshu/R/x86_64-pc-linux-gnu-library/3.2/SparkR/sparkr-assembly-0.1.jar:' edu.berkeley.cs.amplab.sparkr.SparkRBackend /tmp/Rtmp0tAX4W/backend_port614e1c1c38f6 
15/07/09 18:05:51 WARN Utils: Your hostname, himaanshu-Inspiron-5520 resolves to a loopback address: 127.0.0.1; using 172.17.42.1 instead (on interface docker0)
15/07/09 18:05:51 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
15/07/09 18:05:52 INFO Slf4jLogger: Slf4jLogger started
15/07/09 18:05:54 WARN SparkContext: Using SPARK_MEM to set amount of memory to use per executor process is deprecated, please use spark.executor.memory instead.
> sqlContext <- sparkRSQL.init(sc1)
Error: could not find function "sparkRSQL.init"
````

I know but that will cause error too i guess .Are you sure that it should work? — user459, Jul 09 '15 at 13:03
well ,sparkR become officially part of Spark in 1.4.0 so I guess its a pretty good bet it will work.. — milos.ai, Jul 09 '15 at 14:13
I am getting the same error with Spark 1.4 any other thing that I should try? — user459, Jul 11 '15 at 02:16
I tried to use sparkR through terminal .When i used that sqlcontext init command through terminal I got this error - Error: 'sparkRSQL.init' is not an exported object from 'namespace:SparkR' — user459, Jul 11 '15 at 02:35
How did you install `SparkR`? Does it work with SparkR and R shell? — zero323, Jul 12 '15 at 11:00

score 1 · Answer 1 · answered Jul 15 '15 at 03:42

1

You SparkR version is wrong. sparkr-assembly-0.1.jar has not contained sparkRSQL.init yet.

answered Jul 15 '15 at 03:42

coladad

26
3

Could you share the link from where I can install SpARKR of the required version – user459 Jul 15 '15 at 18:58

SparkR sql Context Error after initiating spark R Context job

1 Answers1