Scala flatmap saving to cassandra via spark in loop

Question

I have loop with Names{Real Test!, Real Test1!} and inside it loop throw days (here as example iteration is 1 day only), in this loop some operations with data, and then I flatMap RDD(with 2 rows) to RDD with 5 rows. And on collect().foreach(println) it gave me this:

(Real Test!,2016-07-25,session_end,63)
(Real Test!,2016-07-25,app_open,63)

for first Name - and

(Real Test1!,2016-07-25,app_open,63)
(Real Test1!,2016-07-25,app_install,0)
(Real Test1!,2016-07-25,session_end,0)

for second. When i try to saveToCassandra all 5 rows in loop - in DB I only have last rows of flatMap.

id | name        | date      | count | event
1  | Real Test!  |2016-07-25 |  63  | session_end
2  | Real Test1! |2016-07-25 |  0   | session_end

Why it saves me only last row from flatMap, and how I can save ALL 5 rows?

If your primary key is (id,name,date) then this is the expected behaviour as the data with same primary key overrides the existing data. — Knight71, Jul 25 '16 at 07:29

score 0 · Accepted Answer · edited Aug 03 '16 at 06:54

0

If the primary key is (id,name,date) then this is the expected behaviour as the data with same primary key overrides the existing data.

edited Aug 03 '16 at 06:54

eliasah

39,588
11
124
154

answered Aug 03 '16 at 06:50

ANTVirGEO

124
1
16

Your answer was poor quality. Please read about how to answer question on SO. – eliasah Aug 03 '16 at 06:54

Scala flatmap saving to cassandra via spark in loop

1 Answers1