0

I'm trying to understand how does the LazyList.fill actually works. I implemented a retry logic using LazyList.fill(n). But seems like it is not working as expected.

def retry[T](n: Int)(block: => T): Try[T] = {
  val lazyList = LazyList.fill(n)(Try(block))
  lazyList find (_.isSuccess) getOrElse lazyList.head
}

Considering the above piece of code, I am trying to execute block with a retry logic. If the execution succeeds, return the result from block else retry until it succeeds for a maximum of n attempts.

Is it like LazyList will evaluate the first element and if it finds true, it skips the evaluation for the remaining elements in the list?

Dipu Muraleedharan
  • 1
  • 2
  • 2
    I think the issue is that `Try(block)` is being cached, so it is actually evaluating the `block` only once. You probably can fix it using a couple of indirections like `val execute: = () => block` and then `Try(execute())` – Luis Miguel Mejía Suárez Nov 28 '22 at 16:07
  • And that is exactly what a `LazyList` is supposed to do. The actual elements are materialized/computed in lazy manners, which means only when there is an actual demand for it. `find` stops consuming as soon as it finds the first match, so the element materialization stops as well. Also, why would you want a `retry` method to keep on trying even after success ? Do you actually want a `repeat` method instead ? – sarveshseri Nov 28 '22 at 16:30
  • @LuisMiguelMejíaSuárez Actually that block is an another method which does write the results to a deltalake sink. It is similar to the indirections which you have mentioned. – Dipu Muraleedharan Nov 28 '22 at 16:38
  • @sarveshseri I want to executes this method until it gets succeeded(at most n times).If I am correct, this find will execute the block, if it succeeds it immediately stops the processing. If it fails, it goes to the next execution of same block until it find succeed. – Dipu Muraleedharan Nov 28 '22 at 16:41
  • Yes. It will try to execute it successfully to a maximum of n tries. And will stop at either the `1`st successfull execution or `n`th failed/successful execution. – sarveshseri Nov 28 '22 at 16:44
  • But Spark and Delta will add more things to consider. For example, you will need to ensure that either the `overwrite` flag is on or every attempt tries to write at diffetent path while writing to DeltaLake. Otherwise the partial data created by first attempt will result in failures for all further attempts. – sarveshseri Nov 28 '22 at 16:47
  • Actually, this seems to work just as expected: https://scastie.scala-lang.org/BalmungSan/5ezweW1KRXuOz0X8TVmTlA/4 - So, I guess the problem is your `block` you are probably passing a **Spark** thing which is not running yet; thus the exception is happening elsewhere. – Luis Miguel Mejía Suárez Nov 28 '22 at 17:35
  • 1
    @LuisMiguelMejíaSuárez There is no problem with OP's code/implementation. OP probably just wanted to verify that this `retry` is written correctly and will work reliably. – sarveshseri Nov 28 '22 at 21:05

1 Answers1

0

As I already mentiond in the comment, this is exactly what a LazyList is supposed to do.

The elements in a LazyList are also materialized/computed only when there is demand from an actual consumer.

And find method of LazyList respect this lazyness. You can find it cleary written in documentation as well - https://www.scala-lang.org/api/2.13.x/scala/collection/immutable/LazyList.html#find(p:A=%3EBoolean):Option[A]

def find(p: (A) => Boolean): Option[A]

// Finds the first element of the lazy list satisfying a predicate, if any.

// Note: may not terminate for infinite-sized collections.

// This method does not evaluate any elements further than the first element matching the predicate.

So, If the first element succeeds, it will stop at the first element itself.

Also, if you are writing a retry method then you probably also want to stop at first success. Why would you want to continue evaluating the block even after the suceess.

You might want to better clarify your exact requirements to get a more helpful answer.

sarveshseri
  • 13,738
  • 28
  • 47
  • I am not fully clear about this concept. "Also, if you are writing a retry method then you probably also want to stop at first success. Why would you want to continue evaluating the block even after the success" . If it is evaluated only once as per the predicate, It will stop there right or am I missing something here ? – Dipu Muraleedharan Nov 28 '22 at 19:45
  • Yes, it will stop at first success. As that is what a method named something like `retry` is generally supposed to do. – sarveshseri Nov 28 '22 at 21:02