pyspark dataframe is giving an error with show()

Question

I am using Zeppelin notebook with %livy.pyspark interpreter. I am running a SQL query on Hadoop Hive table and want to see few lines to the table. I am using below code:

'''
%livy.pyspark
from pyspark.sql import HiveContext
sqlContext = HiveContext(sc)
df = sqlContext.sql("Select * from Orders")
df.printSchema()
df.show()
'''

It is showing the table schema with df.printSchema(). However, giving following error with df.show()

'''An error occurred while calling showString.
: java.lang.RuntimeException: serious problem'''

Could you please help me to resolve this issue?

Here Orders table is a transaction table. show() is working on other table. — user151444, Sep 25 '19 at 08:23

score 0 · Answer 1 · answered Sep 25 '19 at 07:35

0

This is the issue with HIVE(https://issues.apache.org/jira/browse/HIVE-13120). Workaround is to set set hive.fetch.task.conversion=none

answered Sep 25 '19 at 07:35

Sudhansu Kumar

129
1
3
10

pyspark dataframe is giving an error with show()

1 Answers1