Read Hocon config as a Map[String, String] with key in dot notation and value

Question

I have following HOCON config:

a {
 b.c.d = "val1"
 d.f.g = "val2" 
}

HOCON represents paths "b.c.d" and "d.f.g" as objects. So, I would like to have a reader, which reads these configs as Map[String, String], ex:

Map("b.c.d" -> "val1", "d.f.g" -> "val2")

I've created a reader and trying to do it recursively:

import scala.collection.mutable.{Map => MutableMap}

  private implicit val mapReader: ConfigReader[Map[String, String]] = ConfigReader.fromCursor(cur => {
    def concat(prefix: String, key: String): String = if (prefix.nonEmpty) s"$prefix.$key" else key

    def toMap(): Map[String, String] = {
      val acc = MutableMap[String, String]()

      def go(
        cur: ConfigCursor,
        prefix: String = EMPTY,
        acc: MutableMap[String, String]
      ): Result[Map[String, Object]] = {
        cur.fluent.mapObject { obj =>
          obj.value.valueType() match {
            case ConfigValueType.OBJECT => go(obj, concat(prefix, obj.pathElems.head), acc)
            case ConfigValueType.STRING =>
              acc += (concat(prefix, obj.pathElems.head) -> obj.asString.right.getOrElse(EMPTY))
          }
          obj.asRight
        }
      }

      go(cur, acc = acc)
      acc.toMap
    }

    toMap().asRight
  })

It gives me the correct result but is there a way to avoid MutableMap here?

P.S. Also, I would like to keep implementation by "pureconfig" reader.

Ivan Stanislavciuc · Answer 1 · 2020-09-20T14:05:24.693

You can do the same without using recursion. Use method entrySet as following

import scala.jdk.CollectionConverters._

val hocon =
  """
  |a {
  | b.c.d = "val1"
  | d.f.g = val2 
  |}""".stripMargin
val config  = ConfigFactory.load(ConfigFactory.parseString(hocon))
val innerConfig = config.getConfig("a")

val map = innerConfig
  .entrySet()
  .asScala
  .map { entry =>
    entry.getKey -> entry.getValue.render()
  }
  .toMap

println(map)

Produces

Map(b.c.d -> "val1", d.f.g -> "val2")

With given knowledge, it's possible to define a pureconfig.ConfigReader that reads Map[String, String] as following

implicit val reader: ConfigReader[Map[String, String]] = ConfigReader.fromFunction {
  case co: ConfigObject =>
    Right(
      co.toConfig
        .entrySet()
        .asScala
        .map { entry =>
          entry.getKey -> entry.getValue.render()
        }
        .toMap
    )
  case value =>
    //Handle error case
    Left(
      ConfigReaderFailures(
        ThrowableFailure(
          new RuntimeException("cannot be mapped to map of string -> string"),
          Option(value.origin())
        )
      )
    )
}

Thanks for your answer, your solution is working, but probably I needed to be more precise in my question. I would like to implement it via "pureconfig" reader, as I mainly using it in my code. — user23, Sep 20 '20 at 08:38
@MikeMostetskyi you can use `entrySet` to define a custom reader. See update — Ivan Stanislavciuc, Sep 20 '20 at 14:06

Matthias Berndt · Accepted Answer · 2020-09-26T08:25:46.140

The solution given by Ivan Stanislavciuc isn't ideal. If the parsed config object contains values other than strings or objects, you don't get an error message (as you would expect) but instead some very strange output. For instance, if you parse a typesafe config document like this

"a":[1]

The resulting value will look like this:

Map(a -> [
    # String: 1
    1
])

And even if the input only contains objects and strings, it doesn't work correctly, because it erroneously adds double quotes around all the string values.

So I gave this a shot myself and came up with a recursive solution that reports an error for things like lists or null and doesn't add quotes that shouldn't be there.

  implicit val reader: ConfigReader[Map[String, String]] = {
    implicit val r: ConfigReader[String => Map[String, String]] =
      ConfigReader[String]
        .map(v => (prefix: String) => Map(prefix -> v))
        .orElse { reader.map { v =>
          (prefix: String) => v.map { case (k, v2) => s"$prefix.$k" -> v2 }
        }}
    ConfigReader[Map[String, String => Map[String, String]]].map {
      _.flatMap { case (prefix, v) => v(prefix) }
    }
  }

Note that my solution doesn't mention ConfigValue or ConfigReader.Result at all. It only takes existing ConfigReader objects and combines them with combinators like map and orElse. This is, generally speaking, the best way to write ConfigReaders: don't start from scratch with methods like ConfigReader.fromFunction, use existing readers and combine them.

It seems a bit surprising at first that the above code works at all, because I'm using reader within its own definition. But it works because the orElse method takes its argument by name and not by value.

score 0 · Answer 3 · answered Sep 18 '21 at 07:46

I did not want to write custom readers to get a mapping of key value pairs. I instead changed my internal data type from a map to list of pairs (I am using kotlin), and then I can easily change that to a map at some later internal stage if I need to. My HOCON was then able to look like this.

  additionalProperties = [
    {first = "sasl.mechanism", second = "PLAIN"},
    {first = "security.protocol", second = "SASL_SSL"},
  ]
  additionalProducerProperties = [
    {first = "acks", second = "all"},
  ]

Not the best for humans... but I prefer it to having to build custom parsing components.

Read Hocon config as a Map[String, String] with key in dot notation and value

3 Answers3