0

When I try to read the csv file in the spark dataframe using the following statement:

val df = spark.read.option("header","true").csv("/home/labuser/Desktop/Project/play-h1b-scoring/H-1B_Disclosure_Data_FY17.csv")

I get this Error :org.apache.spark.sql.AnalysisException: Path does not exist : hdfs://cdhserver:8020/home/labuser/Desktop/Project/play-h1b-scoring/H-1B_Disclosure_Data_FY17.csv .

When I try to read the csv file locally in the spark dataframe using file // by the following statement:

val df = spark.read.option("header","true").csv("file:///home/labuser/Desktop/Project/play-h1b-scoring/H-1B_Disclosure_Data_FY17.csv")

I get this Error :java.io.FileNotFoundException: File file:/home/labuser/Desktop/Project/play-h1b-scoring/H-1B_Disclosure_Data_FY17.csv does not exist.

Kindly help me with the error

Robert Kossendey
  • 6,733
  • 2
  • 12
  • 42

0 Answers0