0

I have a jar that I am running through spark-submit and earlier i was using Argot to parse a HOCON config file which was parsed through ConfigFactory and then I'd read from there.

 * spark/bin/spark-submit --class ConsumerApp \
 *                        --master local[2] \
 *                        some-consumer-jar-0.1.0.jar \
 *                        --config config.hocon

Unfortunately Argot is a dead project now and to upgrade to current version of Scala I have to start using Scopt , but I have trouble understanding how can I parse the same config file using Scopt and load in the ConfigFactory , without making too many changes.

Tried reading through documentation but it couldn't find much.

current implementation 

def main(args: Array[String]) {

    // General bumf for our app
    val parser = new ArgotParser(
      programName = "generated",
      compactUsage = true,
      preUsage = Some("%s: Version %s, %s.".format(
        generated.Settings.name,
        generated.Settings.version,
        generated.Settings.organization)
      )
    )

    // Optional config argument
    val config = parser.option[Config](List("config"),
      "filename",
      "Configuration file.") {
      (c, opt) =>

        val file = new File(c)
        if (file.exists) {
          ConfigFactory.parseFile(file)
        } else {
          parser.usage("Configuration file \"%s\" does not exist".format(c))
          ConfigFactory.empty()
        }
    }
    parser.parse(args)

    // read the config file if --config parameter is provided else fail
    val conf = config.value.getOrElse(throw new RuntimeException("--config argument must be provided"))

and then read it like this 

arg1 = conf.getConfig("somelevel").getString("arg1"),
Alexey Novakov
  • 744
  • 6
  • 20
Egyptian
  • 63
  • 1
  • 3
  • 9
  • May I ask you, why do you need to parse `HOCON` with parser libraries, if Typesafe-Config is already doing that once you provide a file path in the `parseFile` function? – Alexey Novakov Apr 09 '19 at 15:47
  • --config is the optional argument here , hence the need to check if the argument is passed with spark submit , if yes the parse it through ConfigFactory . – Egyptian Apr 09 '19 at 16:13
  • Again, I am not sure why to use a parser library, if one can pass the HOCON file via command line option and read this file via `ConfigFactory.parseFile`. Similar question is here: https://stackoverflow.com/questions/42835971/spark-submit-config-through-file – Alexey Novakov Apr 10 '19 at 13:06

0 Answers0