I've the JSON file as shown below, my scala program has to read this JSON file as an args(0)
and for args(1)
i will be passing CUSTOMER_VEHICLE as an argument. Based on the args(1)
argument passed, program has to fetch the corresponding staginDir path, rawLayerTbl(table name)
, fileDelimiter as a three separate variables.
Created a case class and tried creating JSON object matching the case class but no luck, is there a simple way to achieve this using scala. I will need to execute this on the Databricks runtime version 6.3(includes Apache Spark 2.4.4, Scala 2.11).
rawSource:
DEALER_MASTER :
staginDir :
- "ADLS BLOB LOCATION"
rawLayerTbl:
- "TABLE_NAME"
fileDelimiter :
- "|"
CUSTOMER_VEHICLE :
staginDir :
- "ADLS BLOB LOCATION"
rawLayerTbl:
- "TABLE_NAME"
fileDelimiter :
- "|"
EMPLOYEE_MASTER :
staginDir :
- "ADLS BLOB LOCATION"
rawLayerTbl:
- "TABLE_NAME"
fileDelimiter :
- "|"
CUSTOMERCREDIT_MASTER :
staginDir :
- "ADLS BLOB LOCATION"
rawLayerTbl:
- "TABLE_NAME"
fileDelimiter :
- "|"
CONTINUOUS_CAN_MASTER :
staginDir :
- "ADLS BLOB LOCATION"
rawLayerTbl:
- "TABLE_NAME"
fileDelimiter :
- "|"
LOGIN_TABLE:
staginDir:
- "ADLS BLOB LOCATION"
rawLayerTbl:
- "TABLE_NAME"
fileDelimiter:
- "|"