Python - Convert List Of Dictionaries & Key Values To String

Question

Take the following list of dictionaries and key value pairs:

              [{'name': 'test-project',
                'properties': {'name': 'test-project',
                               'parent': {'id': '', 'type': 'folder'},
                               'projectId': 'test-project'},
                'type': 'cloudresourcemanager.v1.project'},
               {'metadata': {'dependsOn': ['test-project']},
                'name': 'billing_test-project',
                'properties': {'billingAccountName': 'billingAccountName',
                               'name': 'projects/test-project'},
                'type': 'deploymentmanager.v2.virtual.projectBillingInfo'},
               {'name': 'apis',
                'properties': {'apis': ['compute.googleapis.com'],
                               'billing': 'billing_test-project',
                               'concurrent_api_activation': True,
                               'project': 'test-project'},
                'type': 'apis.py'},
               {'name': 'service-accounts',
                'properties': {'project': 'test-project',
                               'service-accounts': ''},
                'type': 'service-accounts.py'},
               {'action': 'gcp-types/compute-v1:compute.projects.setUsageExportBucket',
                'metadata': {'dependsOn': ['test-project',
                                           'test-project-compute.googleapis.com']},
                'name': 'set-export-bucket',
                'properties': {'bucketName': 'gs://usage-exports',
                               'project': 'test-project',
                               'reportNamePrefix': 'usage_gce_'}}]}

I need to convert this into the following syntax:

resources:\n- name: test-project\n properties:\n name: test-project\n parent:\n id:\n type: folder\n

I thought perhaps something like the following would work following my brief Google search:

'\n'.join(d for d in resources)

Unfortunately, that then gives me the error: "TypeError: sequence item 0: expected str instance, dict found"

Any help with this would be greatly appreciated.

(As a side note, Google's only example of the config content being a string is for creating a VM: https://cloud.google.com/deployment-manager/docs/deployments#api; I notice that the spaces seem to increase with each key value pair, but I'm not entirely sure if that's actually required here or not).

EDIT: Apologies, I meant to say I need the key value pairs in a similar format to the below, as a string:

resource = "resources:\n- name: vm-created-by-cloud-config\n  type: compute.v1.instance\n  properties:\n    zone: us-central1-a\n    machineType: https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-a/machineTypes/n1-standard-1\n    disks:\n    - deviceName: boot\n      type: PERSISTENT\n      boot: true\n      autoDelete: true\n      initializeParams:\n        diskName: disk-created-by-cloud-config\n        sourceImage: https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/debian-7-wheezy-v20151104\n    networkInterfaces:\n    - network: https://www.googleapis.com/compute/v1/projects/myproject/global/networks/default\n"

So, when printed, it looks like this:

resources:
- name: test-project
  properties:
    name: test-project
    parent:
      id:
      type: folder
    projectId: test-project
  type: cloudresourcemanager.v1.project
- metadata:
    dependsOn: test-project
  name: billing_test-project
  properties:
    billingAccountName: billingAccountName
    name: projects/test-project
  type: deploymentmanager.v2.virtual.projectBillingInfo
- name: apis
  properties:
    apis: compute.googleapis.com
    billing: billing_test-project
    concurrent_api_activation: True
    project: test-project
  type: apis.py
- name: service-accounts
  properties:
    project: test-project
    service-accounts:
  type: service-accounts.py
- action: gcp-types/compute-v1:compute.projects.setUsageExportBucket
  metadata:
    dependsOn: test-project,test-project-compute.googleapis.com
  name: set-export-bucket
  properties:
    bucketName: gs://usage-exports
    project: test-project
    reportNamePrefix: usage_gce_

revliscano · Accepted Answer · 2020-05-19T14:42:26.397

Try applying the following recursive function. This should work for your specific use case:

resources = {'resources': [{'name': 'test-project',
                'properties': {'name': 'test-project',
                               'parent': {'id': '', 'type': 'folder'},
                               'projectId': 'test-project'},
                'type': 'cloudresourcemanager.v1.project'},
               {'metadata': {'dependsOn': ['test-project']},
                'name': 'billing_test-project',
                'properties': {'billingAccountName': 'billingAccountName',
                               'name': 'projects/test-project'},
                'type': 'deploymentmanager.v2.virtual.projectBillingInfo'},
               {'name': 'apis',
                'properties': {'apis': ['compute.googleapis.com'],
                               'billing': 'billing_test-project',
                               'concurrent_api_activation': True,
                               'project': 'test-project'},
                'type': 'apis.py'},
               {'name': 'service-accounts',
                'properties': {'project': 'test-project',
                               'service-accounts': ''},
                'type': 'service-accounts.py'},
               {'action': 'gcp-types/compute-v1:compute.projects.setUsageExportBucket',
                'metadata': {'dependsOn': ['test-project',
                                           'test-project-compute.googleapis.com']},
                'name': 'set-export-bucket',
                'properties': {'bucketName': 'gs://usage-exports',
                               'project': 'test-project',
                               'reportNamePrefix': 'usage_gce_'}}]}


def unpack_dict(d, spaces=0):
    try:
        s = ' ' * spaces
        spaces += 2
        return  ' '.join([f'\n{s}{k}: {unpack_dict(v, spaces)}' for k, v in d.items()])
    except AttributeError:
        if isinstance(d, list):
            return ''.join([unpack_dict(item) for item in d])
        else:
            return d


result = unpack_dict(resources).strip()

output for print(result)

resources: 
name: test-project 
properties: 
  name: test-project 
  parent: 
    id:  
    type: folder 
  projectId: test-project 
type: cloudresourcemanager.v1.project
metadata: 
  dependsOn: test-project 
name: billing_test-project 
properties: 
  billingAccountName: billingAccountName 
  name: projects/test-project 
type: deploymentmanager.v2.virtual.projectBillingInfo
name: apis 
properties: 
  apis: compute.googleapis.com 
  billing: billing_test-project 
  concurrent_api_activation: True 
  project: test-project 
type: apis.py
name: service-accounts 
properties: 
  project: test-project 
  service-accounts:  
type: service-accounts.py
action: gcp-types/compute-v1:compute.projects.setUsageExportBucket 
metadata: 
  dependsOn: test-projecttest-project-compute.googleapis.com 
name: set-export-bucket 
properties: 
  bucketName: gs://usage-exports 
  project: test-project 
  reportNamePrefix: usage_gce_

Please note:

If you can't use F-strings due to your Python version (< 3.6), you can use the format() function of the str class.
If you need those hyphens, like the one prepending "name" (which I'm not clear enough where else they should go in the output), you could create a temporal dictionary with those included in the keys where they are supposed to be, and apply the unpack_dict() function I provided, passing that temporal dictionary instead of the original resources.

Thanks very much @revliscano! That's really useful! Do you know how I would go about only adding the '\n' line after the key value? So the result would be something like: 'resources:\n name: test-project\n properties:\n name: test-project\n' etc. — RobTheRobot16, May 19 '20 at 11:19
Hello, I'm glad it helped. I have edited my answer. Try calling the `print()` function with `result` now. — revliscano, May 19 '20 at 13:29
Thanks again, @revliscano! That definitely works now. Rather annoyingly however, it looks like Google needs the spaces (I've edited my original post and an example of what they're after). I'll play about with that you've given me, but if you do have any suggestions for how to tackle this it would be greatly appreciated again. My initial line of thinking is to somehow iterate through each one and check to see what "level" the key resides in. If it matches the second level, I'll have to add two spaces, level three with 4 spaces, etc. — RobTheRobot16, May 19 '20 at 14:18
No problem, I have edited my answer once again. If this time it satisfies your requirements, accepting the answer as correct would be great ;-) — revliscano, May 19 '20 at 14:43
Thanks very much! I've definitely learn a lot with your help today! :) — RobTheRobot16, May 19 '20 at 14:54

Python - Convert List Of Dictionaries & Key Values To String

1 Answers1