After you have loaded a CSV file into Spark. And you are exploring from the Scala REPL. How can you tell which Columns have allowed Nulls and which ones haven't?
Asked
Active
Viewed 100 times
0
-
3`df.printSchema` will show this informations: `id: long (nullable = false)` – chlebek Feb 25 '21 at 22:37
-
1I guess the only weird thing is the 98 something fields all show nullable = true... and I know some of them do not have missing data in the CSV – BuddyJoe Feb 26 '21 at 00:09