How to combine two dataset into one in Pyspark

Question

Let's say we have two dataset:

Dataset 1:

id  name    
1   James
2   Tom

Dataset 2:

id   name
1    Bond
2    Jerry

I want dataset 3 in the following format

Dataset 3:

id   name
1    James
2    Tom
3    Bond
4    Jerry

Thanks in advance!!

Possible duplicate of [Concatenate two PySpark dataframes](https://stackoverflow.com/questions/37332434/concatenate-two-pyspark-dataframes) — razdi, Oct 25 '19 at 03:03

score 0 · Accepted Answer · edited Oct 25 '19 at 07:47

0

Fairly straightforward,

df3 = df1.union(df2)

edited Oct 25 '19 at 07:47

Bala

answered Oct 25 '19 at 06:48

thegooner

1 Answers1