10

I've been reading some stuff on Scala type level programming. Mainly the Apocalisp blog, and also a youtube talk by Alexander Lehmann.

I am a bit stuck on something which I guess is probably very basic, which is the use of implicitly to compare two types as shown below:

implicitly[Int =:= Int]

Mark on the Apocalisp blog says:

This is useful for capturing an implicit value that is in scope and has type T.

I get how to make this work, but I don't really know why it works and so don't want to move on.

In the case above is there an implicit of type 'Int' in scope, that 'implicitly' plucks from the ether, allowing the code to compile? How does that fit with the 'function1' return type of?

res0: =:=[Int,Int] = <function1>

Also, where does this implicit come from? How about in the case of my trait 'Foo', why does

implicitly[Foo =:= Foo] 

compile? Where would the 'Foo' implicit come from in this case?

Apologies in advance if this is a very dumb question, and thanks for any help!

bobbyr
  • 226
  • 1
  • 9

1 Answers1

18

X =:= Y is just syntactic sugar (infix notation) for the type =:=[X, Y].

So when you do implicitly[Y =:= Y], you are simply looking up an implicit value of type =:=[X, Y]. =:= is a generic trait defined in Predef.

Also, =:= is a valid type name because type names (just like any identifier) can contain special characters.

From now on, let's rename =:= as IsSameType and remove the infix notation so as to make our code look more straightforward and less magic. This gives us implicitly[IsSameType[X,Y]]

Here is a simplified version of how this type is defined:

sealed abstract class IsSameType[X, Y]
object IsSameType {
   implicit def tpEquals[A] = new IsSameType[A, A]{}
}

Notice how tpEquals provides an implicit value of IsSameType[A, A] for any type A. In other words, it provides an implicit value of IsSameType[X, Y] if and only if X and Y are the same type. So implicitly[IsSameType[Foo, Foo]] compiles fine. But implicitly[IsSameType[Int, String]] does not, as there is no implicit in scope of type IsSameType[Int, String], given that tpEquals is unapplicable here.

So with this very simple construct we are able to statically check that some type X is the same as another type Y.


Now here is an example of how it might be useful. Say I want to define a Pair type (ignoring the fact that it already exists in the standard library):

case class Pair[X,Y]( x: X, y: Y ) {
  def swap: Pair[Y,X] = Pair( y, x )
}

Pair is parameterized with the types of its 2 elements, which can be anything, and most importantly are unrelated. Now what if I want to define a method toList that converts the pair to a 2 elements list? This method would only really make sense in the case where X and Y are the same, otherwise I would be forced to return a List[Any]. And I certainly don't want to change the definition of Pair to Pair[T]( x: T, y: T ) because I really want to be able to have pairs of heterogeneous types. After all, it is only when calling toList that I need that X == Y. All other methods (such as swap) should be callable on any kind of heterogenous pair. So in the end I really want to statically ensure that X == Y, but only when calling toList, in which case it becomes possible and consistent to return a List[X] (or a List[Y], which is the same thing):

case class Pair[X,Y]( x: X, y: Y ) {
  def swap: Pair[Y,X] = Pair( y, x )
  def toList( implicit evidence: IsSameType[X, Y] ): List[Y] = ???
}

But there is still a serious problem when it comes to actually implement toList. If I try to write the obvious implementation, this fails to compile:

def toList( implicit evidence: IsSameType[X, Y] ): List[Y] = List[Y]( x, y )

The compiler will complain that x is not of type Y. And indeed, X and Y are still different types as far as the compiler is concerned. It is only by careful construction that we can be statically sure that X == Y (namely, the fact that toList takes an implicit value of type IsSameType[X, Y], and that they are provided by the method tpEquals only if X == Y). But the compiler certainly won't decipher this astute construction to conclude that X == Y.

What we can do to fix this situation is to provide an implicit conversion from X to Y provided that we know that X == Y (or in other words, that we have an instance of IsSameType[X, Y] in scope).

// A simple cast will do, given that we statically know that X == Y
implicit def sameTypeConvert[X,Y]( x: X )( implicit evidence: IsSameType[X, Y] ): Y = x.asInstanceOf[Y]

And now, our implementation of toList finally compiles fine: x will simply be converted to Y through the implicit conversion sameTypeConvert.

As a final tweak, we can simplify things even further: given that we are taking an implicit value (evidence) as a parameter already, why not have THIS value implement the conversion? Like this:

sealed abstract class IsSameType[X, Y] extends (X => Y) {
  def apply( x: X ): Y = x.asInstanceOf[Y]
}
object IsSameType {
   implicit def tpEquals[A] = new IsSameType[A, A]{}
}    

We can then remove the method sameTypeConvert, as the implicit conversion is now provided by the IsSameType instance itself. Now IsSameType serves a double purpose: statically ensuring that X == Y, and (if it is) providing the implicit conversion that actually allows us to treat instances of X as instances of Y.

We have now basically reimplemented the type =:= as defined in Predef


UPDATE: From the comments, it seems apparent that the use of asInstanceOf is bothering people (even though it really is just an implementation detail, and no user of IsSameType needs to ever do a cast). It turns out that it is easy to get rid of it even in the implementation. Behold:

sealed abstract class IsSameType[X, Y] extends (X => Y) {
  def apply(x: X): Y
}
object IsSameType {
  implicit def tpEquals[A] = new IsSameType[A, A]{
    def apply(x: A): A = x
  }
}

Basically, we just leave the apply abstract, and only implement it right in tpEquals where we (and the compiler) know that both the passed argument and the return value really have the same type. Hence no need for any cast. That's it really.

Note that in the end, the same cast is still present in the generated bytecode, but is now absent from the source code, and provably correct from the point of view of the compiler. And though we did introduce an additional (anonymous) class (and thus an additional indirection from the abstract class to the concrete class), it should run just as fast on any decent virtual machine because we are in the simple case of a "monomorphic method dispatch" (look it up if you are interested in the inner workings of virtual machines). Although it might still make it harder for the virtual machine to inline the call to apply (runtime virtual machine optimization is something of a black art and it's hard to make definite statements).

As a final note, I have to stress that it's really not a big deal to have a cast in the code anyway if it's provably correct. And after all, the standard library itself had this very same cast up until recently (now the whole implementation has been revamped to make it semmingly more powerful, but still contains casts in other places). If it's good enough for the standard library, it's good enough for me.

Régis Jean-Gilles
  • 32,541
  • 5
  • 83
  • 97
  • How does the `apply` method act as an implicit conversion? In a repl, if I do `implicit def blah = new SomeClass` where `SomeClass` has an `apply` method that does the conversion that I want, it doesn't work- there's no implicit conversion available. – Rag Jun 06 '15 at 00:56
  • Very good question, you have actually just found a bug in my code. In scala, merely defining `apply` is enough to use an object as a function (thanks to simple syntactic sugar), but the object is still not a proper function and as such is not eligible as an implicit conversion even if it has an `apply` method. So the fix in my code is to have `IsSameType` extend `(X => Y)` (and sure enough, if you look at the definition of `=:=` in `Predef`, it does extend `From => To`). Thanks for pointing it out, I've updated my answer. – Régis Jean-Gilles Jun 08 '15 at 08:05
  • This answer should be updated to remove calls to `.asInstanceOf`. Either `A =:= B extends A => B` should be used directly and called for value type conversion, or `IsSameType` should explicitly extend `A => B`. The whole point of `=:=` is avoiding calling `.asInstanceOf`, and references to `.asInstanceOf` in this answer only create confusion. – justinpc Oct 31 '16 at 12:04
  • 2
    I think there is some misunderstanding here. My answer takes the approach to pretend that `=:=` does not already exist in the standard library, and to implement it from scratch, step by step. In the process, I expose the internal working of `=:=` (in my case named `IsSameType`), which certainly **does** need `asInstanceOf` (see the original here: https://github.com/scala/scala/blob/v2.11.8/src/library/scala/Predef.scala#L406). – Régis Jean-Gilles Nov 02 '16 at 09:04
  • And the reason why the cast is required is because it is only by construction that the evidence `=:=[X, Y]` guarantees that `X` and `Y` really are the same type, the compiler cannot figure it out on its own. Thus we have to provide the conversion ourself (simply doing a cast, which will always succeed and is thus safe). But rest assured that as a simple **user** of `=:=`, you do **not** need to mess with `asInstanceOf` yourself. – Régis Jean-Gilles Nov 02 '16 at 09:04
  • You don't need `asInstanceOf`, if you define a pair of methods for `IsSameType[X,Y]`: `def to(x : X) : Y` and `def from(y: Y) : X `. https://gist.github.com/edgarklerks/b57dbf554bcee21ae389f2aee45b5bba – Edgar Klerks Oct 10 '19 at 09:40
  • 1
    @Edgar Klerks : There really is no need for a pair of methods to get rid of the cast, and the idea anyway was for the conversion to be implicit. But you are entirely right, the cast is not really required, it is just the most natural implementation given that we know for a fact that the two types are actually the same, so we just need to convince the compiler to get off our lawn. For a straightforward way to do it without a cast, see my update. The short answer is that the cast is required only if we want to implement the `apply` method right in `IsSameType` – Régis Jean-Gilles Oct 21 '19 at 10:03