1

I have loaded a CSV file into RDD format using sc.textFile

gExp = sc.textFile("/mnt/%s/RNA-Seq/GSE10846_Gene_Expression_Data.csv" % MOUNT_NAME)

I want to convert this to a Spark DataFrame

header = gExp.take(1) 
data = gExp.filter(lambda row : row != header).toDF(header)

Here, I am receiving an error:

TypeError: Can not infer schema for type:

j1897
  • 1,507
  • 5
  • 21
  • 41

0 Answers0