How to filter multiple condition in scala

Question

Do have a question in Scala.

val spark = SparkSession.builder
    .master("local[1]")
    .appName("SparkByExamples.com")
    .getOrCreate()

val data = Seq(("James","Smith","USA","CA"),
  ("Michael","Rose","USA","NY"),
  ("Robert","Williams","USA","CA"),
  ("Maria","Jones","USA","FL")
  )

val columns = Seq("firstname","lastname","country","state")
import spark.implicits._
val df = data.toDF(columns:_*)

df.show(false)

firstname	lastname	country	state
James	Smith	USA	CA
Michael	Rose	USA	NY
Robert	Williams	USA	CA
Maria	Jones	USA	FL

I would like to filter this something like this using state to filter with CA and FL only. any idea? What I'm looking for is something filter with Seq any possible? THanks

df.filter("state" === Seq("CA", "FL"))

firstname	lastname	country	state
James	Smith	USA	CA
Robert	Williams	USA	CA
Maria	Jones	USA	FL

did you mean I could use this? df.filter($"state".isin(Seq("CA","NY"))) Thanks — KKK, Jul 21 '22 at 19:43

score 0 · Answer 1 · answered Jul 21 '22 at 19:56

0

Got it,

val selected = Seq("CA","NY")
df.filter($"state".isin(selected:_*)).show

answered Jul 21 '22 at 19:56

KKK

27
7

How to filter multiple condition in scala

1 Answers1