-1

I have a Scala for loop that goes like this:

val a = sc.textFile("path to file containing 8 elements")
for(i <- 0 to a.count.toInt)
{
    println((a.take(i).last))
}

But it throws java.lang.NoSuchElementException error. I am not able to understand what's wrong and how to resolve it?

Tzach Zohar
  • 37,442
  • 3
  • 79
  • 85
user3663737
  • 3
  • 7

1 Answers1

0

There are two problems

1) The "to" operator for defining range (in 0 to a.count.toInt) is a problem here as it is inclusive range from 0 to 8. In a collection of 8 elements, it is trying to access element at index 8.

You can use 0 until a.count.toInt.

2) Second problem is the way "last" operator is called. When i=0, the expression a.take(i) is an empty collection and hence calling "last" on it results in NoSuchElementException.

Why would you iteratively take 1, 2, 3...8 elements from a collection just to take the last element everytime?

It is ok to do what you are doing with a collection of 8 elements but if you wanted to do this on a larger RDD, you should consider caching the RDD if you want to do something like this on a larger RDD.

Pranav Shukla
  • 2,206
  • 2
  • 17
  • 20