Scala Cats Accumulating Errors and Successes with Ior

Question

I am trying to use Cats datatype Ior to accumulate both errors and successes of using a service (which can return an error).

def find(key: String): F[Ior[NonEmptyList[Error], A]] = {
  (for {
      b <- service.findByKey(key)
    } yield b.rightIor[NonEmptyList[Error]])
  .recover {
      case e: Error => Ior.leftNel(AnotherError)
    }
}

def findMultiple(keys: List[String]): F[Ior[NonEmptyList[Error], List[A]]] = {
  keys map find reduce (_ |+| _)
}

My confusion lies in how to combine the errors/successes. I am trying to use the Semigroup combine (infix syntax) to combine with no success. Is there a better way to do this? Any help would be great.

score 5 · Accepted Answer · answered Dec 12 '19 at 10:21

I'm going to assume that you want both all errors and all successful results. Here's a possible implementation:

class Foo[F[_]: Applicative, A](find: String => F[IorNel[Error, A]]) {
  def findMultiple(keys: List[String]): F[IorNel[Error, List[A]]] = {
    keys.map(find).sequence.map { nelsList =>
      nelsList.map(nel => nel.map(List(_)))
        .reduceOption(_ |+| _).getOrElse(Nil.rightIor)
    }
  }
}

Let's break it down:

We will be trying to "flip" a List[IorNel[Error, A]] into IorNel[Error, List[A]]. However, from doing keys.map(find) we get List[F[IorNel[...]]], so we need to also "flip" it in a similar fashion first. That can be done by using .sequence on the result, and is what forces F[_]: Applicative constraint.

N.B. Applicative[Future] is available whenever there's an implicit ExecutionContext in scope. You can also get rid of F and use Future.sequence directly.

Now, we have F[List[IorNel[Error, A]]], so we want to map the inner part to transform the nelsList we got. You might think that sequence could be used there too, but it can not - it has the "short-circuit on first error" behavior, so we'd lose all successful values. Let's try to use |+| instead.

Ior[X, Y] has a Semigroup instance when both X and Y have one. Since we're using IorNel, X = NonEmptyList[Z], and that is satisfied. For Y = A - your domain type - it might not be available.

But we don't want to combine all results into a single A, we want Y = List[A] (which also always has a semigroup). So, we take every IorNel[Error, A] we have and map A to a singleton List[A]:

nelsList.map(nel => nel.map(List(_)))

This gives us List[IorNel[Error, List[A]], which we can reduce. Unfortunately, since Ior does not have a Monoid, we can't quite use convenient syntax. So, with stdlib collections, one way is to do .reduceOption(_ |+| _).getOrElse(Nil.rightIor).

This can be improved by doing few things:

x.map(f).sequence is equivalent to doing x.traverse(f)
We can demand that keys are non-empty upfront, and give nonempty result back too.

The latter step gives us Reducible instance for a collection, letting us shorten everything by doing reduceMap

class Foo2[F[_]: Applicative, A](find: String => F[IorNel[Error, A]]) {
  def findMultiple(keys: NonEmptyList[String]): F[IorNel[Error, NonEmptyList[A]]] = {
    keys.traverse(find).map { nelsList =>
      nelsList.reduceMap(nel => nel.map(NonEmptyList.one))
    }
  }
}

Of course, you can make a one-liner out of this:

keys.traverse(find).map(_.reduceMap(_.map(NonEmptyList.one)))

Or, you can do the non-emptiness check inside:

class Foo3[F[_]: Applicative, A](find: String => F[IorNel[Error, A]]) {
  def findMultiple(keys: List[String]): F[IorNel[Error, List[A]]] = {
    NonEmptyList.fromList(keys)
      .map(_.traverse(find).map { _.reduceMap(_.map(List(_))) })
      .getOrElse(List.empty[A].rightIor.pure[F])
  }
}

montrivo · Answer 2 · 2020-02-26T13:40:11.710

Ior is a good choice for warning accumulation, that is errors and a successful value. But, as mentioned by Oleg Pyzhcov, Ior.Left case is short-circuiting. This example illustrates it:

scala> val shortCircuitingErrors = List(
  Ior.leftNec("error1"),
  Ior.bothNec("warning2", 2),
  Ior.bothNec("warning3", 3)
).sequence

shortCircuitingErrors: Ior[Nec[String], List[Int]]] = Left(Chain(error1))

One way to accumulate both errors and successes is to convert all your Left cases into Both. One approach is using Option as right type and converting Left(errs) values into Both(errs, None). After calling .traverse, you end up with optList: List[Option] on the right side and you can flatten it with optList.flatMap(_.toList) to filter out None values.

class Error
class KeyValue

def find(key: String): Ior[Nel[Error], KeyValue] = ???

def findMultiple(keys: List[String]): Ior[Nel[Error], List[KeyValue]] =
  keys
    .traverse { k =>
      val ior = find(k)
      ior.putRight(ior.right)
    }
    .map(_.flatMap(_.toList))

Or more succinctly:

def findMultiple(keys: List[String]): Ior[Nel[Error], List[KeyValue]] =
  keys.flatTraverse { k =>
    val ior = find(k)
    ior.putRight(ior.toList) // Ior[A,B].toList: List[B]
  }

Scala Cats Accumulating Errors and Successes with Ior

2 Answers2

Linked