I'm trying to create a spark structured streaming application with arbitrary state, when I add groupByKey and mapGroupWithState it gives me an error after starting the first task.
.groupByKey(_.user_id)
.mapGroupsWithState(GroupStateTimeout.NoTimeout)(sessionState.updateAcrossEvents)
Error: Lost task 5.0 in stage 1.0 (TID 5, node, executor 1): java.lang.NoClassDefFoundError: Could not initialize class com.test.Run$
Run is the main class which contains the main method.
Can someone tell me why this is happening?
PS: if i write directly to console without groupByKey and mapGroupsWithState it works fine.