How to properly register primitives and nulls in polymorphic serialization?

Question

I need to set up a serialization/deserialization mechanism for a polymorphic class hierarchy that also includes primitives and nulls. There are container classes containing collections with polymorphic objects, primitives, and nulls. And, the subclasses for these objects are spread across modules (therefore sealed is not an option).

I have been reading through the kotlinx.serialization polymorphism docs trying to come up with a solution. I've been able to make some incremental progress by working through that tutorial but I seem to still be hitting a wall when I try to put everything together.

The code I am posting here is a minimal example that brings together everything I need. If I can get this example to work, that should cover everything I need for my real project. This example does run without error but introduces some unnecessary readability and efficiency issues.

All classes in my custom class hierarchy are serializable data classes. The outermost container object that needs to be serialized/deserialized is a map wrapper. This map has keys which are each an instance of one of these data classes. And the values of this map can be primitives, nulls, or instances of one of my data classes. I think my main challenge here is to include those primitives and nulls in my polymorphic serialization in a clean way.

The goal of my code below is to represent this problem in the simplest way possible and to serialize and deserialize one container object successfully.

There are two main issues in the code:

1. I've had to replace null with FakeNull. Without this, I get null cannot be cast to non-null type kotlin.Any. This will reduce the readability and simplicity of my code and I suspect it could decrease efficiency as well.
2. I've had to add StringClassSerializer and DoubleClassSerializer and wrapper classes. I would also need to add serializers like these for every primitive class. If I don't register these primitives as subclasses of Any, I get Class 'String' is not registered for polymorphic serialization in the scope of 'Any'.. And if I try to register them with their default serializers (like subclass(String::class, String.serializer())) I get Serializer for String of kind STRING cannot be serialized polymorphically with class discriminator.. The problem with using serializers like StringClassSerializer and wrappers like StringWrapper is that it removes the efficiency and readability benefits of using primitives.

The json comes out looking like:

{"type":"MapContainer","map":[{"type":"SubA","data":1.0},{"type":"StringWrapper","s":"valueA"},{"type":"SubB","data":2.0},{"type":"DoubleWrapper","d":2.0},{"type":"SubB","data":3.0},{"type":"SubA","data":1.0},{"type":"SubB","data":4.0},{"type":"matt.play.FakeNull"}]}

I don't like the way this looks. I want the nulls to simply be null and the primitives to simply be primitives.

import kotlinx.serialization.KSerializer
import kotlinx.serialization.PolymorphicSerializer
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.descriptors.buildClassSerialDescriptor
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.json.Json
import kotlinx.serialization.modules.SerializersModule
import kotlinx.serialization.modules.polymorphic
import kotlinx.serialization.modules.subclass
import kotlin.collections.set

@Serializable
abstract class SuperClass

@Serializable
@SerialName("SubA")
data class SubA(val data: Double): SuperClass()

@Serializable
@SerialName("SubB")
data class SubB(val data: Double): SuperClass()

@Serializable
@SerialName("MapContainer")
data class MapContainer<K: SuperClass, V>(val map: Map<K, V>): Map<K, V> by map

@Serializable
@SerialName("StringWrapper")
data class StringWrapper(val s: String)

@Serializable
@SerialName("DoubleWrapper")
data class DoubleWrapper(val d: Double)

object StringClassSerializer: KSerializer<String> {
  override val descriptor = buildClassSerialDescriptor("string")
  override fun deserialize(decoder: Decoder) = decoder.decodeSerializableValue(StringWrapper.serializer()).s
  override fun serialize(encoder: Encoder, value: String) =
    encoder.encodeSerializableValue(StringWrapper.serializer(), StringWrapper(value))
}

object DoubleClassSerializer: KSerializer<Double> {
  override val descriptor = buildClassSerialDescriptor("double")
  override fun deserialize(decoder: Decoder) = decoder.decodeSerializableValue(DoubleWrapper.serializer()).d
  override fun serialize(encoder: Encoder, value: Double) =
    encoder.encodeSerializableValue(DoubleWrapper.serializer(), DoubleWrapper(value))
}

@Serializable
object FakeNull

fun main() {
  val theMap = mutableMapOf<SuperClass, Any?>()
  theMap[SubA(1.0)] = "valueA"
  theMap[SubB(2.0)] = 2.0
  theMap[SubB(3.0)] = SubA(1.0)
  theMap[SubB(4.0)] = FakeNull /*wish I could make this just `null`*/
  val theMapContainer = MapContainer(theMap)
  val format = Json {
    allowStructuredMapKeys = true
    ignoreUnknownKeys = true
    serializersModule = SerializersModule {
      polymorphic(SuperClass::class) {
        subclass(SubA::class)
        subclass(SubB::class)
      }
      polymorphic(Any::class) {


        /*I wish I could remove all of this primitive wrapper stuff*/
        default {
          when (it) {
            StringWrapper::class.simpleName -> StringClassSerializer
            DoubleWrapper::class.simpleName -> DoubleClassSerializer
            else                            -> throw RuntimeException("unknown type: ${it}?")
          }
        }
        subclass(String::class, StringClassSerializer)
        subclass(Double::class, DoubleClassSerializer)
        subclass(SubA::class)
        subclass(SubB::class)
        subclass(FakeNull::class)
      }

      polymorphic(
        MapContainer::class, MapContainer::class, actualSerializer = MapContainer.serializer(
          PolymorphicSerializer(SuperClass::class),
          PolymorphicSerializer(Any::class)
        ) as KSerializer<MapContainer<*, *>>
      )
    }
  }
  val encoded = format.encodeToString(PolymorphicSerializer(MapContainer::class), theMapContainer)
  println("\n\n${encoded}\n\n")
  val decoded = format.decodeFromString(PolymorphicSerializer(MapContainer::class), encoded)

  if (theMapContainer != decoded) {
    throw RuntimeException("the decoded object is not the same as the original")
  } else {
    println("success")
  }
}

Answer 1

Primitives (such as strings, numbers, and enums) by default are serialized as JSON primitives (e.g., "answer" or 42), not JSON objects ({ ... }). This is why they don't support polymorphic serialization; there is no "space" to place the type information in (the class discriminator).

There is no JSON object to place the class discriminator in, e.g., {"type": "fully.qualified.Name"} by default.

But, kotlinx serialization does allow you to write custom serializers, which allows you to work around this. I wrote a custom serializer for enums since I wanted to register enums as concrete types in polymophic serialization. It sounds like you should be able to do something similar. _{(Disclosure: I only read your problem description in detail; not your ongoing attempts/solution.)}

A serializer which supports registering [Enum]s as subclasses in polymorphic serialization when class discriminators are used. When class discriminators are used, an enum is not encoded as a structure which the class discriminator can be added to. An exception is thrown when initializing [Json]: " "Serializer for of kind ENUM cannot be serialized polymorphically with class discriminator." This serializer encodes the enum as a structure with a single value holding the enum value.

Use this serializer to register the enum in the serializers module, e.g.: subclass( <enum>::class, PolymorphicEnumSerializer( <enum>.serializer() )

This custom serializer can possibly be generalized to any primitive type and thus support your use case.