1

As a linux noob, I recently set up the spark and shark to play around. There is an API sql2rdd that I want to use to pull data from shark in to rdd. However, I don't know where is the sql2rdd library is and how to link with the Spark Interactive Shell. I read somewhere on the web said that the Shark API needs to be sbt into Spark directory without any instruction. If anyone knows, please advice. Thanks in advance.

ChrisGPT was on strike
  • 127,765
  • 105
  • 273
  • 257
user3267815
  • 11
  • 2

1 Answers1

0

Go to shark installation directory. And the the following:

Step 1: ./bin/shark-shell

Step 2: val testRdd = sc.sql2rdd("select * from temp") // sc - SharkContext in shark-shell

Step 3: Now if you do an action, "testRdd.count" you can view the number of 
        records
ra.
  • 1,798
  • 14
  • 15
Maverick
  • 484
  • 2
  • 9
  • 20