0
case class employeeObj(fn:String, ln:String)

//don't worry too much about processEmps function as I am just returning first object to test functionality

def processEmps(m1:employeeObj, m2:employeeObj):employeeObj ={
  m1
}

var o1 = employeeObj("john", "king")
var o2 = employeeObj("john", "king")
var o3 = employeeObj("jason", "king")`

var x = sc.parallelize(Array(("a", o1), ("a", o2), ("b", o3)))
x.reduceByKey(processEmps)

When I call x.reduceByKey(processEmps), it is failing by generating type mismatch error.

Please let me know what I am doing wrong here.

enter image description here

Brandon McKenzie
  • 1,655
  • 11
  • 26
SpkQuestions
  • 11
  • 2
  • What version of Apache Spark are you using? Have you tried to clean and recompile your whole code? Everything that you've posted works on Spark 1.6.1. – Paweł Jurczenko Jun 03 '16 at 13:23
  • 4
    Possible duplicate of ["error: type mismatch" in Spark with same found and required datatypes](http://stackoverflow.com/questions/37476790/error-type-mismatch-in-spark-with-same-found-and-required-datatypes) – zero323 Jun 03 '16 at 13:29
  • @zero323 spot on! I am able to run this code without any errors. Getting the 'type mismatch' error after redefining the case class. – Pranav Shukla Jun 03 '16 at 13:43
  • It works, when I use the following: var x = sc.parallelize(Array(("a",employeeObj("john", "king")), ("a", employeeObj("john", "king")))) x.reduceByKey(processEmps) – SpkQuestions Jun 03 '16 at 22:46

0 Answers0