0

I would like to include --optional-components ANACONDA,JUPYTER in my CICD deployment using Deployment Manager

I have tried to place it in python template configuration under metadata secion as well as directly in properties, I checked schemas, existing templates, however I cant find proper spot for it nor anything related in documentation(maybe I missed something, but I only reached gcloud CLI part not DM equvalent)

My expected result would have --optional-components ANACONDA,JUPYTER inside : `

resources = []

# cluster X
resources.append({
    'name': 'X',
    'type': 'dataproc.py',
    'subnetwork': 'default',
    'properties': {
        'zone': ZONE,
        'region': REGION,
        'serviceAccountEmail': 'X',
        'softwareConfig': {
            'imageVersion': 'X',
            'properties': {
                'dataproc:dataproc.conscrypt.provider.enable' : 'False'
            }
        },
        'master': {
            'numInstances': 1,
            'machineType': 'n1-standard-1',
            'diskSizeGb': 50,
            'diskType': 'pd-standard',
            'numLocalSsds': 0
        }, 
        'worker': {
            'numInstances': 2,
            'machineType': 'n1-standard-1',
            'diskType': 'pd-standard',
            'diskSizeGb': 50,
            'numLocalSsds': 0
        },
        'initializationActions':[{
            'executableFile': 'X'
        }],
        'metadata': {
            'PIP_PACKAGES':'requests_toolbelt==0.9.1 google-auth==1.6.31'
        },
        'labels': {
            'environment': 'dev',
            'data_type': 'X'
        }
    }
})`
Organic Platypus
  • 61
  • 7

1 Answers1

0

Deployment manager only uses the APIs and only supports certain APIs. For your case to write --optional-components ANACONDA,JUPYTER , please follow this link as you almost close.To have details about the cloud dataproc components refer to the dataproc API reference.

Patrick W
  • 4,603
  • 1
  • 12
  • 26
Shafiq I
  • 406
  • 2
  • 7