Scala case class ignoring import in the Spark shell

Question

I hope there is an obvious answer to this question!

I've just upgraded to Spark v2.0 and have an odd problem with the spark-shell (Scala 2.11 build).

If I enter the following minimal Scala,

import java.sql.Timestamp

case class Crime(caseNumber: String, date: Timestamp, description: String, detail: String, arrest: Boolean)

I get the following error,

<console>:11: error: not found: type Timestamp

If I use the Java Timestamp class elsewhere, e.g. in a function, then no errors are generated (as you would expect because of the import).

If I fully qualify and use java.sql.Timestamp in the case class it works!

Am I missing something obvious?

I have the same problem within Zeppelin. How do you fix it there? — wrschneider, Sep 08 '17 at 20:56

score 12 · Accepted Answer · answered Aug 02 '16 at 16:54

12

It's just that the Timestamp is not loaded in the case class declaration, to fix this you can:

:paste
import java.sql.Timestamp
case class Crime(caseNumber: String, date: Timestamp, description: String, detail: String, arrest: Boolean)

or

case class Crime(caseNumber: String, date: java.sql.Timestamp, description: String, detail: String, arrest: Boolean)

answered Aug 02 '16 at 16:54

Mikel San Vicente

Thanks! That's what I did, but it used to work in the 1.6 shell... Do you know what's changed? – Max van Daalen Aug 02 '16 at 17:07
How do you fix this within Zeppelin? – wrschneider Sep 08 '17 at 20:56

1 Answers1