How merge three DataFrame in Scala

Question

How merge 3 DataFrame in Spark-Scala? I completly don't have any Idea how I can make this. On stackOverFlow I can't found similar example.

I have 3 similar DataFrames. The same name of Column, and the same number of them. Difference is only a value on rows.

DataFrame1:

+----+------+----+---+
|type| Model|Name|ID |
+----+------+----+---+
|  1 |wdasd |xyzd|111|
|  1 |wd    |zdfd|112|
|  1 |bdp   |2gfs|113|
+----+------+----+---+

DataFrame2:

+----+------+----+---+
|type| Model|Name|ID |
+----+------+----+---+
|  2 |wdasd |xyzd|221|
|  2 |wd    |zdfd|222|
|  2 |bdp   |2gfs|223|
+----+------+----+---+

DataFrame3:

+----+------+----+---+
|type| Model|Name|ID |
+----+------+----+---+
|  3 |AAAA  |N_AM|331|
|  3 |BBBB  |NA_M|332|
|  3 |CCCC  |MA_N|333|
+----+------+----+---+

And I want to this type of DataFrame

MergeDataFrame:

+----+------+----+---+
|type| Model|Name|ID |
+----+------+----+---+
|  1 |wdasd |xyzd|111|
|  1 |wd    |zdfd|112|
|  1 |bdp   |2gfs|113|
|  2 |wdasd |xyzd|221|
|  2 |wd    |zdfd|222|
|  2 |bdp   |2gfs|223|
|  3 |AAAA  |N_AM|331|
|  3 |BBBB  |NA_M|332|
|  3 |CCCC  |MA_N|333|
+----+------+----+---+

Possible duplicate of [How can I combine(concatenate) two data frames with the same column name in java](https://stackoverflow.com/questions/40397206/how-can-i-combineconcatenate-two-data-frames-with-the-same-column-name-in-java) — philantrovert, Mar 15 '18 at 10:14
Also, this : https://stackoverflow.com/questions/37332434/concatenate-two-pyspark-dataframes — philantrovert, Mar 15 '18 at 10:15
and also this https://stackoverflow.com/questions/43554601/efficiently-using-union-in-spark — Ramesh Maharjan, Mar 15 '18 at 10:18

score 13 · Accepted Answer · answered Mar 15 '18 at 09:37

Spark provides a union and unionAll. Looks like they are deprecating the unionAll function so I would use the union function as below:

dataFrame1.union(dataFrame2).union(dataFrame3)

Note that in order to union data frames the data frames must have the exact same column names in the exact same order.

See the spark docs here

How merge three DataFrame in Scala

DataFrame1:

DataFrame2:

DataFrame3:

MergeDataFrame:

1 Answers1