Case statement issue in Pyspark

Question

I'm currently writing this code in Pyspark but I'm getting errors and I know it's wrong but I'm not sure how to fix it. I'm trying to say that if 1 then I want to use a certain column with the value inside of it, if 2 this other column with the value inside of it and so on.

    ABC_SCHED = ABC_SCHED.withColumn("ABC123",
    F.when(F.col("MINS_TRA" == "1"), ABC_SCHED.T1_0to30)
    .when(F.col("MINS_TRA" == "2"), ABC_SCHED.T2_31to60)
    .when(F.col("MINS_TRA" == "3"), ABC_SCHED.T3_61to90)
    .when(F.col("MINS_TRA" == "4"), ABC_SCHED.T4_91to120)
    .when(F.col("MINS_TRA" == "5"), ABC_SCHED.T4_120ORMORE)
    .otherwise(F.lit(None))
    )

looks like this column name is invalid `T4_120+`, try `ABC_SCHED['T4_120+']`? You might want to change all other columns too. — pltc, Nov 01 '21 at 19:38

score 1 · Answer 1 · answered Nov 01 '21 at 20:23

I found the answer! I just had parentheses in the wrong places

    ABC_SCHED = ABC_SCHED.withColumn("ABC123",
    F.when((F.col("MINS_TRA") == "1"), ABC_SCHED.T1_0to30)
    .when((F.col("MINS_TRA") == "2"), ABC_SCHED.T2_31to60)
    .when((F.col("MINS_TRA") == "3"), ABC_SCHED.T3_61to90)
    .when((F.col("MINS_TRA") == "4"), ABC_SCHED.T4_91to120)
    .when((F.col("MINS_TRA") == "5"), ABC_SCHED.T5_120ORMORE)
    .otherwise(F.lit(None))
    )

Case statement issue in Pyspark

1 Answers1