NPE in Spark Java toBytes(row.getAs("value2"))

Question

Why does the following generate a NPE?

   final SparkSession spark = SparkSession.builder()
                                           .appName(" test")
                                           .getOrCreate();

    spark.createDataFrame(singletonList(new GenericRow(new Object[] { 1L, null, 3L })),
                          new StructType(new StructField[] { DataTypes.createStructField("value1", DataTypes.LongType, true),
                                                             DataTypes.createStructField("value2", DataTypes.LongType, true),
                                                             DataTypes.createStructField("value3", DataTypes.LongType, true)}))
         .foreach((ForeachFunction<Row>) row -> {

             System.out.println("###" + row.getAs("value1"));
             System.out.println(row.<Long>getAs("value2"));
             System.out.println(toBytes(row.<Long>getAs("value2")));
             System.out.println("###" + row.getAs("value3"));
         });

I think this does not occur in Spark 1.6, but unsure, could just be better test data.

Answer 1

So line System.out.println(toBytes(row.<Long>getAs("value2"))); The line

row.<Long>getAs(“value2”)) 

returns a null “Long” Object

but then

toBytes(long l)

wants a “long”, so java unboxes the Long into a null long => NPE as shown in this answer.

To protect against this we can use Java Optional

toBytes(Optional.ofNullable(row.<Long>getAs(name)).orElse(0L)));