1

I'm working my way up to a "Hello World" kind of SnappyData application, which I would like to be able to build and run in IntelliJ. My cluster so far is one locator, one lead, and one server on the local machine. I just want to connect to it, serialize a trivial bit of data or maybe a DataFrame, and see that it's working.

The Documentation says I should be able to do something like this:

val spark: SparkSession = SparkSession
  .builder()
  .appName("SnappyTest")
  .master("xxx.xxx.xxx.xxx:xxxx")
  .getOrCreate()
val snappy = new SnappySession(spark.sparkContext)

However, I get "Cannot resolve symbol SnappySession."

Here's what I have in my build.sbt:

name := "snappytest"

version := "0.1"

scalaVersion := "2.11.11"

// https://mvnrepository.com/artifact/io.snappydata/snappy-spark-core_2.11
libraryDependencies += "io.snappydata" % "snappy-spark-core_2.11" % "2.1.1.1"
// https://mvnrepository.com/artifact/io.snappydata/snappy-spark-sql_2.11
libraryDependencies += "io.snappydata" % "snappy-spark-sql_2.11" % "2.1.1.1"

(I refreshed the project after adding those.)

I gather that, when I import something Spark-related such as:

import org.apache.spark.sql.SparkSession

I'm really importing the extended SnappyData version from the dependencies in my build.sbt, not the canonical org.apache.spark version. So that should mean that I could also:

import org.apache.spark.sql.SnappySession

However, I get "Cannot resolve symbol SnappySession." And I don't see anything Snappy-related in the code completion drop-downs when I'm typing. It looks for all the world like vanilla Spark.

What am I missing here? I assume I'm missing something obvious. I can't find examples of the import headers or build statements in the SnappyData documentation, I assume because those kinds of details were too obvious to mention. Except to me. Is anyone here willing to help de-n00batize me on this matter?

Joseph Pride
  • 165
  • 11

2 Answers2

1

I am assuming you are trying to connect to an existing SnappyData cluster with a Spark application. Can you please check http://snappydatainc.github.io/snappydata/howto/spark_installation_using_smart_connector/ to see the maven coordinates parameters to do so.

Rishitesh Mishra
  • 459
  • 5
  • 9
  • Additionally, refer to this link for specifying SBT dependencies: http://snappydatainc.github.io/snappydata/affinity_modes/local_mode/#local-mode // https://mvnrepository.com/artifact/io.snappydata/snappydata-cluster_2.11 libraryDependencies += "io.snappydata" % "snappydata-cluster_2.11" % "1.0.0" – Amogh Oct 23 '17 at 09:16
  • 1
    According to those docs, it appears that `libraryDependencies += "io.snappydata" % "snappydata-cluster_2.11" % "1.0.0"` is the relevant line – plamb Oct 23 '17 at 15:47
  • Thanks for taking a stab at this, guys!Amogh: Where are you seeing the SBT dependencies in those links? I've been through those pages a few times and haven't seen any build.sbt examples. The linked example is to the scala code, but I can't make the same imports in that code work. – Joseph Pride Oct 24 '17 at 05:37
  • Also, as far as cluster design, I'm actually trying to use Embedded mode, not Smart Connector mode. In Embedded mode, I'm attempting to run the app on my machine rather than submit (as you can do in Spark). plambre: Seems correct! The object resolves. How can I find out that sort of information in the future? – Joseph Pride Oct 24 '17 at 05:47
  • 1
    Sorry, the comment wasn't well formatted. If you search for **SBT dependency** on [that page](http://snappydatainc.github.io/snappydata/affinity_modes/local_mode/#local-mode), you would find the example posted in the comment (also same as what @plambre posted above). – Amogh Oct 24 '17 at 12:23
  • 1
    Joesph, are you talking about SnappyData's API reference? Perhaps you mean this: http://snappydatainc.github.io/snappydata/apidocs/#package – plamb Oct 24 '17 at 15:38
1

you should import snappydata-core to resolve that. like this maven dependency:

<dependency>
  <groupId>io.snappydata</groupId>
  <artifactId>snappydata-core_${scala.version.major}</artifactId>
  <version>1.0.3</version>
</dependency>
Duc
  • 11
  • 1