I'm working with a Spark dataframe and trying to convert string to date. Here's what I did
df.select('transactionDate',from_unixtime(unix_timestamp('transactionDate', 'EEE MMM dd HH:mm:ss z YYYY')).alias('new_transactionDate')).show(5, truncate=False)
Here's the result:
Notice that the month and date are incorrect. Why is this? How do I fix it?
I'm using PySpark in Jupyter Notebook. (Spark 2.1)