How to efficiently traverse recursive case-class trees

Question

I want to create a macro that generates a recursive traversal of a tree of case class instances similar to the visitor pattern. The generated code should recurse for all fields whose type is derived from one of multiple base types.

I thought about some code that could be used similar to this:

trait A
trait B
trait C

case object D extends A { }
case object E extends C { }
case class F (a: A, b: B, c: Int) extends A
case class G (d: C, e: String) extends B
val instanceOfA = F (D, G (E, "Foo"), 1)

// expected type   extra types that should be included in traversal
//       vv        vv
traverse[A,        B, C] (handlerForA, handlerForB, handlerForC) (instanceOfA)

The type of traverse would be something like the following (I know this isn't valid syntax but couldn't come up with any better):

def traverse[T, U...] (handleT : T => Unit, handleU... : U... => Unit) : T => Unit = macro traverseImpl

I know that Scala can't support variadic generics but I don't see why there shouldn't be a way to achieve something like this with macros. But I haven't found a way that allows to pass multiple types as arguments to a def macro.

Answer 1

After some investigation is appears to me that your problem is slightly more complicated than it seems on the first sight. I drafted a solution where instead of macros I used DSL based on Monocle:

import monocle.Lens
import monocle.macros.GenLens

case class Nested[A, B](aToB: Lens[A, B], handleB: B => Unit) {

  def from[C](cToA: Lens[C, A]): Nested[C, B] = Nested(cToA composeLens aToB, handleB)

  def nest(nested: Nested[B, _]*): Seq[Nested[A, _]] = this +: nested.map(_.from(aToB))
}

def traverse[A](handleA: A => Unit)(configure: Nested[A, A] => Seq[Nested[A, _]]): A => Unit =
  value => configure(Nested(Lens.id[A], handleA)).foreach { case Nested(lens, handler) =>
    handler(lens.get(value))
  }

However, as soon as I started testing it I figured out an issue:

val t = traverse[F](handlerForA)(_.nest(
  Nested(GenLens[F](_.a), handlerForA),
  Nested(GenLens[F](_.b), handlerForB),
  // how to put handlerForC?
))

t(instanceOfA)

I was usable to use handlerForC... because your original types were not product types-only. A might have different implementations, B and C as well.

So, could we try to generate some more complex solution where coproducts are taken into account? Well, with your exact example not - compiler is only able to derive known direct subclasses if your class/trait is sealed and all (direct) implementations have to be implemented in the same file.

But let's say you made A, B and C sealed. In such case I would somehow try to use Prism, in order to step inside only when possible (or use pattern matching or any other equivalent solution). But that complicates the DSL - the reason I guess why do decided to look at the macros in the first place.

So, lets rethink the issue anew.

Code that you would have to write would:

• take into account extracting values from products types
• take into account branching on coproduct types
• make use of provided handlers for each (coproduct) type
• minimize the amount of code written

This requirements sounds difficult to achieve. Right now I am thinking if you could achieve your goal faster by using recursive schemes if you created a common parent for all traits, "lifted" all your classes to Id and then writing one traversing function and one which applies handlers.

Of course all of that could be implemented one way or the other using def macros, but the thing is will requirements in the way they are now I would be extremely difficult to make sure that it won't do something bad, as we have pretty strict requirements for the output (for each handled cases find them even if they are nested) while very relaxed requirements on the input (least upper bound is Any/AnyRef, hierarchies are not sealed). I am not sure if runtime reflection wouldn't be easier way to achieve your goals as long as you don't want to change any assumption or requirements.

Answer 2

I found another completely different solution with its own drawbacks. I added two traits Traversable and TraverseHandler:

trait Traversable[+T] {
    def traverse (handler: TraverseHandler) : T
}
trait TraverseHandler {
    def handleA (a: A) : A
    def handleB (b: B) : B
    def handleC (c: C) : C
}

A, B and C extend Traversable[A]... and their traverse method is implemented in all case classes. I'm using annotation macros (wanted to avoid them, but they seams to be the easiest solution) to to the following code expansion:

@traversable [TraverseHandler]
case class F (@expand a: A, @expand b: B, c: Int) extends A

to

case class F (a: A, b: B, c: Int) extends A {
    def traverse (handler: TraverseHandler) : F = {
        val newA = handler.handleA(a)
        val newB = handler.handleB(b)
        if ((newA ne a) || (newB ne b))
            copy (a = newA, b = newB)
        else
            this
    }
}

This avoids the need for sealed traits and allows different handlers while not adding to much overhead to the classes themselves.

Of course it would have been nice if there was something easier or something which wouldn't require changes to the classes themselves. It also requires the handlers to do the recursion (which on the other side gives the power to delete individual elements without processing them or allows for custom pre/post-processing).

Maybe a even nicer solution might have been to create a method in a companion object which provides a class specific constructor for lenses. But I still fear that the recursive nature of this structure might complicate use of monocle. How would one create a lens that depends on a lens which in turn depends on the original lens? Or in other words: Is there a fixed point combinator for lenses?