How to implement types like MapK in Scala 3 / Dotty?

Question

I am trying and failing to get something like this to work in Scala 3:

type TupleK[K[*], V[*], A] = (K[A], V[A]) 

final class MapK[K[*], V[*]] private (val rawMap: Map[K[?], V[?]]) {
    
  def foreach(f: TupleK[K, V, ?] => Unit): Unit = {
    rawMap.foreach(f.asInstanceOf[Tuple2[K[?], V[?]] => Any])
  }
}

object MapK {
  
  def apply[K[*], V[*]](entries: TupleK[K, V, ?]*): MapK[K, V] = {
    new MapK[K, V](Map(entries: _*))
  }
}

With usage like this:

class Key[A]()
  
type Id[A] = A
  
val intKey = Key[Int]
val strKey = Key[String]

MapK[Key, Id](intKey -> 1, strKey -> "a")

In Scala 2 that works, just need to adjust syntax by replacing * and ? with _ (except in _* of course).

In Scala 3 however basically every line errors with "unreducible application of higher-kinded type to wildcard arguments": Scastie.

The docs say that existential types have been dropped in Scala 3, however they don't really give any non-trivial examples of how to deal with this.

The docs mention that "Existential types largely overlap with path-dependent types" – can this MapK be implemented with path-dependent types? I've read this but didn't understand how to apply that, or whether it's possible in my case.

And, if not path dependent types... then what? It seems unlikely that Scala was "simplified" to the point where it's impossible to implement this functionality anymore, so I must be missing something.

ETA: In addition to my own answers below, I made this repo and wrote this article about the various approaches to encoding MapK in Scala 3.

Answer 1

I was able to produce a working, albeit incredibly annoying, implementation. This pointer was especially valuable.

First, a few notes:

• Type inference on this sucks on many levels. All the manual type ascribtions in the tests, and all of the implicit conversions below are needed for this to work.

• Apparently Scala is not smart enough to figure out that A and type Id[A] = A are functionally the same when looking for implicits, so a combinatorial explosion of ad-hoc implicit conversions are needed. Ugly and not very scalable.

• Observe the different options available in Scala 3: foreach, foreachT, and foreachK. All of them have stylistic tradeoffs.

• If you can improve on any of this, please let me know. This works, but it was so much nicer in Scala 2.

MapK implementation:

class MapK[K[_], V[_]] protected(protected val rawMap: Map[Type[K], Type[V]]) {

  def apply[A](key: K[A]): V[A] = {
    rawMap(key).asInstanceOf[V[A]]
  }

  def updated[A](key: K[A], value: V[A]): MapK[K, V] = {
    MapK.unsafeCoerce(rawMap.updated(key, value))
  }

  def updated[A](pair: (K[A], V[A])): MapK[K, V] = {
    MapK.unsafeCoerce(rawMap.updated(pair._1, pair._2))
  }

  def foreach[A](f: ((K[A], V[A])) => Unit): Unit = {
    rawMap.foreach(f.asInstanceOf[(([Type[K], Type[V]])) => Any])
  }

  def foreachT(f: Type.Tuple2[K, V] => Unit): Unit = {
    foreach { (k, v) => f((k, v)) }
  }

  def foreachK(f: [A] => (K[A], V[A]) => Unit): Unit = {
    foreach { (k, v) => f(k, v) }
  }

}

object MapK {

  def unsafeCoerce[K[_], V[_]](rawMap: Map[Type[K], Type[V]]): MapK[K, V] = {
    new MapK[K, V](rawMap)
  }

  def apply[K[_], V[_]](entries: Type.Tuple2[K, V]*): MapK[K, V] = {
    new MapK[K, V](Map(entries.asInstanceOf[Seq[(Type[K], Type[V])]]: _*))
  }
}

Other methods in MapK that you might want to implement basically follow the same patterns as foreach, foreachT, or foreachK.

And now, usage:

  def test(caption: String)(code: => Unit): Unit = code

  def assertEquals[A](a: A, b: A): Unit = assert(a == b)

  case class Key[A](label: String, default: A)

  val boolKey = Key[Boolean]("bool", false)

  val intKey = Key[Int]("int", 0)

  val strKey = Key[String]("str", "")

  val optionMap = MapK[Key, Option](boolKey -> Some(true), intKey -> Some(1), strKey -> Option("a"), strKey -> None)

  val idMap = MapK[Key, Id](boolKey -> true, intKey -> 1, strKey -> "hello")

  val expectedOptionValues = List[Type.Tuple3[Key, Option, Id]](
    (boolKey, Some(true), false),
    (intKey, Some(1), 0),
    (strKey, None, "")
  )

  val expectedIdValues = List[Type.Tuple3[Key, Id, Id]](
    (boolKey, true, false),
    (intKey, 1, 0),
    (strKey, "hello", "")
  )

  test("optionMap - apply & updated") {
    assertEquals(optionMap(intKey), Some(1))
    assertEquals(optionMap(strKey), None)
    assertEquals(optionMap.updated(strKey, Some("yo"))(strKey), Some("yo"))
  }

  test("optionMap - foreach") {
    var values: List[Type.Tuple3[Key, Option, Id]] = Nil

    optionMap.foreach { (k, v) =>
      values = values :+ (k, v, k.default)
    }

    assertEquals(values, expectedOptionValues)
  }

  test("optionMap - foreachT") {
    var values: List[Type.Tuple3[Key, Option, Id]] = Nil

    optionMap.foreachT { pair => // no parameter untupling :(
      values = values :+ (pair._1, pair._2, pair._1.default)
    }

    assertEquals(values, expectedOptionValues)
  }

  test("optionMap - foreachK") {
    var values: List[Type.Tuple3[Key, Option, Id]] = Nil

    optionMap.foreachK {
      [A] => (k: Key[A], v: Option[A]) => // need explicit types :(
        values = values :+ (k, v, k.default)
    }

    assertEquals(values, expectedOptionValues)
  }

  test("idMap - apply & updated") {
    assertEquals(idMap(intKey), 1)
    assertEquals(idMap(strKey), "hello")
    assertEquals(idMap.updated(strKey, "yo")(strKey), "yo")
  }

  test("idMap - foreach") {
    var values: List[Type.Tuple3[Key, Id, Id]] = Nil

    idMap.foreach { (k, v) =>
      values = values :+ (k, v, k.default)
    }

    assertEquals(values, expectedIdValues)
  }

  test("idMap - foreachT") {
    var values: List[Type.Tuple3[Key, Id, Id]] = Nil

    idMap.foreachT { pair =>
      values = values :+ (pair._1, pair._2, pair._1.default)
    }

    assertEquals(values, expectedIdValues)
  }

  test("idMap - foreachK") {
    var values: List[Type.Tuple3[Key, Id, Id]] = Nil

    idMap.foreachK {
      [A] => (k: Key[A], v: A) =>
        values = values :+ (k, v, k.default)
    }

    assertEquals(values, expectedIdValues)
  }

And now, the support cast that makes this work:

import scala.language.implicitConversions // old style, but whatever

type Id[A] = A

type Type[F[_]] <: (Any { type T })


object Type {

  type Tuple2[F[_], G[_]] <: (Any { type T })

  type Tuple3[F[_], G[_], H[_]] <: (Any { type T })
}


implicit def wrap[F[_], A](value: F[A]): Type[F] =
  value.asInstanceOf[Type[F]]

implicit def wrapT2[F[_], G[_], A](value: (F[A], G[A])): Type.Tuple2[F, G] =
  value.asInstanceOf[Type.Tuple2[F, G]]

implicit def wrapT2_P1[F[_], A](t: (F[A], A)): Type.Tuple2[F, Id] = wrapT2[F, Id, A](t)

implicit def wrapT3[F[_], G[_], H[_], A](value: (F[A], G[A], H[A])): Type.Tuple3[F, G, H] =
  value.asInstanceOf[Type.Tuple3[F, G, H]]

implicit def wrapT3_P1[F[_], G[_], A](value: (F[A], A, A)): Type.Tuple3[F, Id, Id] =
  value.asInstanceOf[Type.Tuple3[F, Id, Id]]

implicit def wrapT3_P1_P2[F[_], G[_], A](value: (F[A], G[A], A)): Type.Tuple3[F, G, Id] =
  value.asInstanceOf[Type.Tuple3[F, G, Id]]


implicit def unwrap[F[_]](value: Type[F]): F[value.T] =
  value.asInstanceOf[F[value.T]]

implicit def unwrapT2[F[_], G[_]](value: Type.Tuple2[F, G]): (F[value.T], G[value.T]) =
  value.asInstanceOf[(F[value.T], G[value.T])]

implicit def unwrapT3[F[_], G[_], H[_]](value: Type.Tuple3[F, G, H]): (F[value.T], G[value.T], H[value.T]) =
  value.asInstanceOf[(F[value.T], G[value.T], H[value.T])]

Answer 2

Here's an alternative solution, using dependent types. In general I like it better, it's more obvious to me what's going on.

import scala.language.implicitConversions

type Id[A] = A

implicit def wrapId[A](a: A): Id[A] = a

implicit def unwrapId[A](a: Id[A]): A = a


case class Key[A](caption: String, default: A)

val boolKey = Key[Boolean]("bool", false)
val intKey = Key[Int]("int", 0)
val strKey = Key[String]("str", "")


type KTuple[K[_], V[_]] = {
  type T;
  type Pair = (K[T], V[T]);
}

implicit def KTuple[K[_], V[_], A](value: (K[A], V[A])): KTuple[K, V]#Pair = value.asInstanceOf[KTuple[K, V]#Pair]

implicit def KTuple_P1[K[_], A](value: (K[A], A)): KTuple[K, Id]#Pair = value.asInstanceOf[KTuple[K, Id]#Pair]

class MapK[K[_], V[_]](rawMap: Map[K[Any], V[Any]]) {

  def foreachK(f: [A] => (K[A], V[A]) => Unit): Unit = {
    rawMap.foreach(f.asInstanceOf[((K[Any], V[Any])) => Unit])
  }

  def foreach(f: KTuple[K, V]#Pair => Unit): Unit = {
    rawMap.foreach { pair =>
      f(pair.asInstanceOf[KTuple[K, V]#Pair])
    }
  }
}

object MapK {

  def create[K[_], V[_]](pairs: KTuple[K, V]#Pair*): MapK[K, V] = {
    val x: List[KTuple[K, V]#Pair] = pairs.toList
    val y: List[(K[Any], V[Any])] = x.map(t => t.asInstanceOf[(K[Any], V[Any])])
    new MapK(Map(y: _*))
  }

}

val idMap = MapK.create[Key, Id](
  boolKey -> false,
  intKey -> 1,
  strKey -> "a",
)

val optionMap = MapK.create[Key, Option](
  intKey -> Some(1),
  strKey -> Some("a")
)

type T3[A] = (Key[A], A, A)

var log = List[KTuple[Key, Option]#Pair]()

idMap.foreach { (k, v) =>
  log = log.appended(KTuple(k, Some(v)))
}

def doSomething[A, V[_]](k: Key[A], v: V[A]): Unit = println(s"$k -> v")

optionMap.foreachK {
  [A] => (k: Key[A], v: Option[A]) => {
    doSomething(k, v.get)
    doSomething(k, v)
    log = log :+ KTuple((k, v))
  }
}

I wrote up a blog post with more details, will publish soon after some editing. Still looking for better approaches and improvements though.