0

I'd like to append columns from one pyspark dataframe to another.

In pandas, the command would look like

df1 = pd.DataFrame({'x':['a','b','c']})
df2 = pd.DataFrame({'y':[1,2,3]})

pd.concat((df1, df2), axis = 1)

Is there a way to accomplish this in pyspark? All I can find is either concatenating the contents of a column or doing a join.

Demetri Pananos
  • 6,770
  • 9
  • 42
  • 73
  • You have to do a join, but first you need to have a join key (like the index in a pandas dataframe). – pault Aug 08 '18 at 17:20
  • 1
    Possible duplicate of [Concatenate two dataframes pyspark](https://stackoverflow.com/questions/44305012/concatenate-two-dataframes-pyspark) and [How to concatenate/append multiple Spark dataframes column wise in Pyspark?](https://stackoverflow.com/q/44320699/5858851) – pault Aug 08 '18 at 17:23

0 Answers0