How do I write case statements in Pyspark using Palantir Foundry

Question

Trying to take a case statement from macros in VBA and transfer into Pyspark. How would I go about writing something like this in Pyspark?

 Select Case dMinutes
                Case 0 To 30
                    xnum = 1
                Case 31 To 60
                    xnum = 2
                Case 61 To 90
                    xnum = 3
                Case 91 To 120
                    xnum = 4
                Case Else
                    xnum = 5

Does this answer your question? [Apache spark dealing with case statements](https://stackoverflow.com/questions/39982135/apache-spark-dealing-with-case-statements) — Kate, Oct 26 '21 at 21:12

score 2 · Accepted Answer · answered Oct 26 '21 at 21:09

The following PySpark should achieve what you want!

import pyspark.sql.functions as F

dataset = dataset.withColumn('xnum',
    F.when(F.col('dMinutes').between(0,30), 1)
    .when(F.col('dMinutes').between(31,60), 2)
    .when(F.col('dMinutes').between(61,90), 3)
    .when(F.col('dMinutes').between(91,120), 4)
    .otherwise(5)
)

How do I write case statements in Pyspark using Palantir Foundry

1 Answers1