I am having an issue with a line of code that used to work fine in Spark 1.6 and doesn’t work in Spark 2.2. The error is java.io.FileNotFoundException: File does not exist:
Note there is a white space in the file path. The space is after the yyyy-mm-dd.
hdfs://hadoop/path/part_date=2018-04-20 15%3A01%3A21/000000_0
That might be causing the problem. How can I get around this.
df = spark.read.format('orc').load('hdfs://hadoop/path/part_date=2018-04-20%2015%253A01%253A21/000000_0')
df.show()