Infinite recursion with Shapeless select[U]

Question

I had a neat idea (well, that's debatable, but let's say I had an idea) for making implicit dependency injection easier in Scala. The problem I have is that if you call any methods which require an implicit dependency, you must also decorate the calling method with the same dependency, all the way through until that concrete dependency is finally in scope. My goal was to be able to encode a trait as requiring a group of implicits at the time it's mixed in to a concrete class, so it could go about calling methods that require the implicits, but defer their definition to the implementor.

The obvious way to do this is with some kind of selftype a la this psuedo-scala:

object ThingDoer {
  def getSomething(implicit foo: Foo): Int = ???
}

trait MyTrait { self: Requires[Foo and Bar and Bubba] =>
  //this normally fails to compile unless doThing takes an implicit Foo
  def doThing = ThingDoer.getSomething
}

After a few valiant attempts to actually implement a trait and[A,B] in order to get that nice syntax, I thought it would be smarter to start with shapeless and see if I could even get anywhere with that. I landed on something like this:

import shapeless._, ops.hlist._

trait Requires[L <: HList] {
  def required: L
  implicit def provide[T]:T = required.select[T]
}

object ThingDoer {
  def needsInt(implicit i: Int) = i + 1
}

trait MyTrait { self: Requires[Int :: String :: HNil] =>
  val foo = ThingDoer.needsInt
}

class MyImpl extends MyTrait with Requires[Int :: String :: HNil] {
  def required = 10 :: "Hello" :: HNil
  def showMe = println(foo)
}

I have to say, I was pretty excited when this actually compiled. But, it turns out that when you actually instantiate MyImpl, you get an infinite mutual recursion between MyImpl.provide and Required.provide.

The reason that I think it's due to some mistake I've made with shapeless is that when I step through, it's getting to that select[T] and then steps into HListOps (makes sense, since HListOps is what has the select[T] method) and then seems to bounce back into another call to Requires.provide.

My first thought was that it's attempting to get an implicit Selector[L,T] from provide, since provide doesn't explicitly guard against that. But,

1. The compiler should have realized that it wasn't going to get a Selector out of provide, and either chosen another candidate or failed to compile.
2. If I guard provide by requiring that it receive an implicit Selector[L,T] (in which case I could just apply the Selector to get the T) then it doesn't compile anymore due to diverging implicit expansion for type shapeless.ops.hlist.Selector[Int :: String :: HNil], which I don't really know how to go about addressing.

Aside from the fact that my idea is probably misguided to begin with, I'm curious to know how people typically go about debugging these kinds of mysterious, nitty-gritty things. Any pointers?

Answer 1

When I get confused about something related to implicits / type-level behaviour, I tend to find the reify technique useful:

scala> import scala.reflect.runtime.universe._
import scala.reflect.runtime.universe._
scala> val required: HList = HNil
required: shapeless.HList = HNil
scala> reify { implicit def provide[T]:T = required.select[T] }
res3: reflect.runtime.universe.Expr[Unit] =
Expr[Unit]({
  implicit def provide[T]: T = HList.hlistOps($read.required).select[T](provide);
  ()
})

At this point it's easy to see what's gone wrong - the compiler thinks provide can provide any arbitrary T (because that's what you've told it), so it just calls provide to get the required Selector[L, T]. At compile time it only resolves this once, so there is no diverging implicit, no confusion at compile time - only at run-time.

The diverging implicit expansion happens because the compiler looks for a Selector[Int :: String :: HNil], it thinks provide could give it one if given a Selector[Int :: String :: HNil, Selector[Int :: String :: HNil]], it thinks provide could give it one if given a Selector[Int :: String :: HNil, Selector[Int :: String :: HNil, Selector[Int :: String :: HNil]] and at some point it realises this is an infinite loop. Where/how are you expecting it to get the Selector it needs? I think your provide is misguided because it's too general. Try making the call to ThingDoer.needsInt with an explicit int work first before trying to make it all implicit.

This general approach does work - I've written applications that use it as a DI mechanism -though beware of quadratic compile times.