Losing columns names when writing sparkdataframe with sparkR write.df

Asked Mar 08 '18 at 13:42

Active Mar 08 '18 at 14:17

Viewed 536 times

Context

I'm working on an azure HDI R server cluster with rstudio and sparkR package. I'm reading file, modifying it and then i want to write it with write.df, but the problem is that when i write the file, my column names disappear.

My code is the following:

write.df(spdf,"///Output/File","csv","overwrite",header=T)

here is the file i want to write in csv format

Num,Letter
5.5,a
9.,b
5.5,c
9,d
5.5,e
9,f
5.5,g
9,h
5.5,i
9,j

here is the file i get:

    0,1
    5.5,a
    9.,b
    5.5,c
    9,d
    5.5,e
    9,f
    5.5,g
    9,h
    5.5,i
    9,j

edited Mar 08 '18 at 14:17

asked Mar 08 '18 at 13:42

Orhan Yazar

2

Put your code here, you have to set "header" option true to have the header in your output file. Like this: write.df(..., header = "true",...). – Mobin Ranjbar Mar 08 '18 at 13:51
Thank you my friend – Orhan Yazar Mar 08 '18 at 14:17
4

sucks that this is completely undocumented :\ – MichaelChirico Aug 16 '18 at 03:15

Losing columns names when writing sparkdataframe with sparkR write.df

0 Answers0