Scala changing parquet path in config (typesafe)

Question

Currently I have a configuration file like this:

project {
  inputs {
    baseFile {
      paths = ["project/src/test/resources/inputs/parquet1/date=2020-11-01/"]
      type = parquet
      applyConversions = false
    }
  }
}

And I want to change the date "2020-11-01" to another one during run time. I read I need a new config object since it's immutable, I'm trying this but I'm not quite sure how to edit paths since it's a list and not a String and it definitely needs to be a list or else it's going to say I haven't configured a path for the parquet.

val newConfig = config.withValue("project.inputs.baseFile.paths"(0),
        ConfigValueFactory.fromAnyRef("project/src/test/resources/inputs/parquet1/date=2020-10-01/"))

But I'm getting a:

Error com.typesafe.config.ConfigException$BadPath: path parameter: Invalid path 'project.inputs.baseFile.': path has a leading, trailing, or two adjacent period '.' (use quoted "" empty string if you want an empty element)

What's the correct way to set the new path?

Tomer Shetah · Accepted Answer · 2020-12-09T16:10:42.740

One option you have, is to override the entire array:

import scala.collection.JavaConverters._
val mergedConfig = config.withValue("project.inputs.baseFile.paths",
                      ConfigValueFactory.fromAnyRef(Seq("project/src/test/resources/inputs/parquet1/date=2020-10-01/").asJava))

But a more elegant way to do this (IMHO), is to create a new config, and to use the existing as a fallback.

For example, we can create a new config:

val newJsonString = """project {
                   |inputs {
                   |baseFile {
                   |  paths = ["project/src/test/resources/inputs/parquet1/date=2020-10-01/"]
                   |}}}""".stripMargin

val newConfig = ConfigFactory.parseString(newJsonString)

And now to merge them:

val mergedConfig = newConfig.withFallback(config)

The output of:

println(mergedConfig.getList("project.inputs.baseFile.paths"))
println(mergedConfig.getString("project.inputs.baseFile.type"))

is:

SimpleConfigList(["project/src/test/resources/inputs/parquet1/date=2020-10-01/"])
parquet

As expected.

You can read more about Merging config trees. Code run at Scastie.

I didn't find any way to replace one element of the array with withValue.

Scala changing parquet path in config (typesafe)

1 Answers1