Spark dataset related

Question

My data set is looking like this by following code

+------+---------------+----+
|  City|      Timestamp|Sale|
+------+---------------+----+
|City 3|6/30/2017 16:04|  28|
|City 4| 7/4/2017 16:04|  12|
|City 2|7/13/2017 16:04|   8|
|City 4|7/16/2017 16:04|  21|
|City 4| 7/3/2017 16:04|  24|
|City 2|7/17/2017 16:04|  34|
|City 3| 7/9/2017 16:04|  13|
|City 3|7/18/2017 16:04|  26|
|City 3| 7/6/2017 16:04|  16|
|City 3|7/15/2017 16:04|  29|
|City 4|7/18/2017 16:04|  39|
|City 2| 7/1/2017 16:04|  19|
|City 2|7/18/2017 16:04|  19|
|City 4| 7/4/2017 16:04|  24|
|City 2| 7/4/2017 16:04|   9|
|City 4|7/15/2017 16:04|  20|
|City 3|7/12/2017 16:04|  19|
|City 1| 7/9/2017 16:04|  13|
|City 1|7/13/2017 16:04|  25|
|City 4|7/10/2017 16:04|  10|
+------+---------------+----+

We need to calculate the sum of Sale of each City in week base.

provide solution in spark scala – Ved Prakash Jul 06 '17 at 05:09 — Ved Prakash, Jul 06 '17 at 05:09

score 0 · Answer 1 · answered Jul 06 '17 at 05:10

0

You can group by City and Time stamp and sum the Sales

data.groupBy("City", "TimeStamp").agg(sum(col("Sale")).as("TotalSale")).show

Hope this helps!

answered Jul 06 '17 at 05:10

koiralo

22,594
6
51
72

Hi,it working fine BUT i need week wise ie. week wise sum of sale of each city – Ved Prakash Jul 06 '17 at 05:34
hi ,it working fine but i need week wise sum ie for each week i need sum of sale in each city [city1 Fri 01/06/2017 38] – Ved Prakash Jul 06 '17 at 05:36
How could some one determine weeks from your given input ? – koiralo Jul 10 '17 at 05:52
did you get how to determine weeks ? – koiralo Jul 12 '17 at 09:48

Spark dataset related

1 Answers1