-2

I have written the following code, which returns a "Class not found" exception. I'm not sure what I need to do to load data from a csv file into SparkSQL.

import org.apache.spark.SparkContext

/**

 * Loading sales csv using DataFrame API

 */

object CsvDataInput {

  def main(args: Array[String]) {    

    val sc = new SparkContext(args(0), "Csv loading example")

    val sqlContext = new org.apache.spark.sql.SQLContext(sc)

    val df = sqlContext.load("com.databricks.spark.csv", Map("path" -> args(1),"header"->"true"))

    df.printSchema()

    df.registerTempTable("data")

    val aggDF = sqlContext.sql("select * from data")

    println(aggDF.collectAsList())

  } 
}
Gillespie
  • 2,228
  • 2
  • 18
  • 25
Ravi Reddy
  • 23
  • 1
  • 8
  • Hi there, can you provide some more information on: How you are executing this code, and the full error you are getting? – Gillespie Sep 08 '15 at 20:47

1 Answers1

0

Try replacing this line

import org.apache.spark.SparkContext

with this

import org.apache.spark.*

You are importing just part of the library, but using classes from outside this part. Also, your import is actually misspelled - it should read org.apache.spark.sql.SQLContext, and you used some other package, not related to the code presented.

Kelevandos
  • 7,024
  • 2
  • 29
  • 46