0
    val rddData1 = sc.makeRDD(1 to 10, 2)
    println(rddData1.glom.collect)

code in idea or spark-shell will output [[I@34a0ef00

but rddData1.glom.collect in spark-shell will output Array[Array[Int]] = Array(Array(1, 2, 3, 4, 5), Array(6, 7, 8, 9, 10))

how do i can get Array[Array[Int]] = Array(Array(1, 2, 3, 4, 5), Array(6, 7, 8, 9, 10)) in idea

chinayangyongyong
  • 37
  • 1
  • 9

2 Answers2

0
val conf = new SparkConf()
  .setMaster("local[*]")
  .setAppName("myAppName")
val sc = new SparkContext(conf)
val rddData1 = sc.makeRDD(1 to 10,2)
val rddData2 = sc.makeRDD(20 to 25,2)
println(rddData1.glom().collect())
println(java.util.Arrays.deepToString( rddData1.glom().collect().map(_.asInstanceOf[Object])))
// output [[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]]
// 结束任务
sc.stop()
chinayangyongyong
  • 37
  • 1
  • 9
0
val conf = new SparkConf()
  .setMaster("local[*]")
  .setAppName("myAppName")
val sc = new SparkContext(conf)
val rddData1 = sc.makeRDD(1 to 10,2)
println(java.util.Arrays.toString( rddData1.collect()))
// output [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
// 结束任务
sc.stop()
chinayangyongyong
  • 37
  • 1
  • 9
  • Thank you for your contribution to the community. This may be a correct answer, but it’d be useful to provide additional explanation of your code so developers can understand your reasoning. This is especially useful for new developers who may not be familiar with the syntax. Further, it can help reduce the need for follow-up questions. – Jeremy Caney May 08 '20 at 09:43