0

I using Analytics Engine on IBM Cloud and trying to pass Ambari configuration Like below in Advanced provisioning options. 

{
    "ambari_config": {
        "hardware_config": "default",
        "software_package": "ae-1.2-hive-spark",
        "num_compute_nodes": 1,
        "advanced_options": {
            "ambari_config": {
                "spark2-defaults": {
                    "spark.dynamicAllocation.minExecutors": 1,
                    "spark.shuffle.service.enabled": true,
                    "spark.dynamicAllocation.maxExecutors": 2,
                    "spark.dynamicAllocation.enabled": true
                }
            }
        }
    }
}

I am following this documentation to pass the above configuration

https://cloud.ibm.com/docs/services/AnalyticsEngine?topic=AnalyticsEngine-advanced-provisioning-options

After multiple retires i see that each time my cluster request is failing.

Rachit Arora
  • 1
  • 1

2 Answers2

0

After reviewing my request, I figured out that I am passing ambari_config attribute twice for my request which i not accepted

Valid json which worked for me looks like this 

{
    "hardware_config": "default",
    "software_package": "ae-1.2-hive-spark",
    "num_compute_nodes": 1,
    "advanced_options": {
        "ambari_config": {
            "spark2-defaults": {
                "spark.dynamicAllocation.minExecutors": 1,
                "spark.shuffle.service.enabled": true,
                "spark.dynamicAllocation.maxExecutors": 2,
                "spark.dynamicAllocation.enabled": true
            }
        }
    }
}
Rachit Arora
  • 1
  • 1
0

one more scenario where cluster creation can fail is like InvalidTopologyException: The following config types are not defined in the stack: [spar2-hive-site-override]

Above issue was because of TYPO to define config property file where user want to add or modify properties.

Shobhit Shukla
  • 1