-1

Below is my Scala code:

import scala.io.Source

case class TempData(day:Int , DayOfYear:Int , month:Int , year:Int ,
                    precip:Double , snow:Double , tave:Double, tmax:Double, tmin:Double)
object TempData {
 def main(args:Array[String]) : Unit ={
   val source = Source.fromFile("C:///DataResearch/SparkScala/MN212142_9392.csv")
   val lines = source.getLines().drop(1)
   val data= lines.map{ line =>
     val p = line.split(",")
     TempData(p(0).toInt , p(1).toInt, p(2).toInt, p(4).toInt
     , p(5).toDouble, p(6).toDouble , p(7).toDouble, p(8).toDouble,p(9).toDouble)
   }.toArray
   source.close()
   data.take(5).foreach(println)
 }
}

Below is the error:

C:\Java\jdk1.8.0_144\bin\java "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2017.3.3\lib\idea_rt.jar=2298:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2017.3.3\bin" -Dfile.encoding=UTF-8 -classpath C:\Java\jdk1.8.0_144\jre\lib\charsets.jar;C:\Java\jdk1.8.0_144\jre\lib\deploy.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\access-bridge-64.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\cldrdata.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\dnsns.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\jaccess.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\jfxrt.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\localedata.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\nashorn.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\sunec.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\sunjce_provider.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\sunmscapi.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\sunpkcs11.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\zipfs.jar;C:\Java\jdk1.8.0_144\jre\lib\javaws.jar;C:\Java\jdk1.8.0_144\jre\lib\jce.jar;C:\Java\jdk1.8.0_144\jre\lib\jfr.jar;C:\Java\jdk1.8.0_144\jre\lib\jfxswt.jar;C:\Java\jdk1.8.0_144\jre\lib\jsse.jar;C:\Java\jdk1.8.0_144\jre\lib\management-agent.jar;C:\Java\jdk1.8.0_144\jre\lib\plugin.jar;C:\Java\jdk1.8.0_144\jre\lib\resources.jar;C:\Java\jdk1.8.0_144\jre\lib\rt.jar;C:\Users\Dell\IdeaProjects\ScalaSpark\target\scala-2.11\classes;C:\Users\Dell\.ivy2\cache\org.scala-lang\scala-library\jars\scala-library-2.11.12.jar TempData
Exception in thread "main" java.io.FileNotFoundException: C:\DataResearch\SparkScala\MN212142_9392.csv (The system cannot find the file specified)
    at java.io.FileInputStream.open0(Native Method)
    at java.io.FileInputStream.open(FileInputStream.java:195)
    at java.io.FileInputStream.<init>(FileInputStream.java:138)
    at scala.io.Source$.fromFile(Source.scala:91)
    at scala.io.Source$.fromFile(Source.scala:76)
    at scala.io.Source$.fromFile(Source.scala:54)
    at TempData$.main(TempData.scala:7)
    at TempData.main(TempData.scala)

Process finished with exit code 1

My file(MN212142_9392.csv) is in C drive inside folder DataResearch and SparkScala. I have tried all possible changes but nothing of any help.

halfer
  • 19,824
  • 17
  • 99
  • 186
Harshit Kakkar
  • 117
  • 2
  • 12

2 Answers2

3

Try to use File class instead of Source.

new File("path")

It should solve your problem :)

Or use class Files :) Below you can find links to documentation: https://docs.oracle.com/javase/7/docs/api/java/io/File.html https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html

Double check your path. In my opinion better option is to use a relative path instead of absolute. Put this file in a project and try to use the relative path. Your application can't find your file :)

Edit:

In created a simple file TempData.scala which contains:

import scala.io.Source 

object TempData {
  def main(args:Array[String]) : Unit ={
    val source = Source.fromFile("test.csv")
    val lines = source.getLines()
  }
}

I created file test.csv

I used scalac TempData.scala and after that scala TempData. Everything is okay, because you use methods in correct way. You have a problem with path. So my propose is try to use file that you put in the project and if everything is okay check your previous path :)

Patryk Rudnicki
  • 755
  • 1
  • 8
  • 21
  • I am not sure about File , Source is a library in Scala (import scala.io.Source) to load file. Not sure about library for File. – Harshit Kakkar May 19 '18 at 13:37
  • Yes, but for working with files in Scala the better option is use Java classes for it :) Try, I'm sure that it's better option. Here: https://www.tutorialspoint.com/java/java_files_io.htm you can find example how to proceed files in Java. Scala class for it is not good :) – Patryk Rudnicki May 19 '18 at 13:39
  • Yes you might be right , but what if I don't want to use any java libraries..Does scala libraries will not help? and using java libray will lead me to do some more code changes since command like getLines() not working. – Harshit Kakkar May 19 '18 at 13:43
  • Why? If something is better in Java, in my opinion, we should use it. If you want to use scala library try to put this file inside project and use relative path :) – Patryk Rudnicki May 19 '18 at 13:44
  • Your error is related to an incorrect path, so, first of all, you need to check path. If you have a file inside you can use a relative path. In practice is better to have a control on that :) – Patryk Rudnicki May 19 '18 at 13:46
  • Does scala libraries will not help? and using java libray will lead me to do some more code changes since command like getLines() not working. – Harshit Kakkar May 19 '18 at 13:47
  • Try to use a relative path. Put this file into the project. – Patryk Rudnicki May 19 '18 at 13:49
  • Yes file is in correct path , but using java library i am not able to use scala function like getLines() , map() etc..and I only want to write scala code , not wanna go into java – Harshit Kakkar May 19 '18 at 13:49
  • Okay, so please go to my comment. I'll show you an example in an edit. Give me 5 minutes – Patryk Rudnicki May 19 '18 at 13:54
  • Ok so how do i find the relative path , i have put the file inside C:\Users\Dell\IdeaProjects\ScalaSpark , still not working , do i need to put some where else? – Harshit Kakkar May 19 '18 at 14:02
  • Okay, so if you put this file inside `ScalaSpark` Please try this `Source.fromFile("MN212142_9392.csv")` – Patryk Rudnicki May 19 '18 at 14:06
  • Hello , I got the issue , actually my file name was MN212142_9392.csv but it was expecting.txt extension also so now doing ("MN212142_9392.csv.txt") worked for me. this is also working :"C:///DataResearch/SparkScala/MN212142_9392.csv.txt") – Harshit Kakkar May 19 '18 at 14:06
  • Nice, so the problem is with Path :) – Patryk Rudnicki May 19 '18 at 14:07
  • Thank you patryk :) – Harshit Kakkar May 19 '18 at 14:07
  • No problem, please set this question as answered – Patryk Rudnicki May 19 '18 at 14:08
-1

I have been facing issues with similar error: Find sample code which i have done:

  object CSVDemo extends App {
         val bufferedSource = io.Source.fromFile("C:\\Users\\dell\\IdeaProjects\\SbtExampleProject\\project\\job_test.txt")
        for (line <- bufferedSource.getLines()) {
          val cols = line.split(",").map(_.trim)
          println(s"${cols(0)}|${cols(1)}|${cols(2)}|${cols(3)}|${cols(4)}")
        }

NOTE: I had to create sample file in the project directory and when i tried to see its location i noticed that the path is created with "\" try it and see how it goes

ntambara peter
  • 1
  • 1