Glom spec for list of dictionaries inside a list of dictionaries

Question

I am currently using glom to parse through a JSON API response, which returns, among other things, a list of dictionaries, with a list of dictionaries inside it. The problem I'm having is getting glom to access the correct dictionary entry.

Example JSON:

{'answeredAt': '2019-08-23T21:11:04Z',
  'direction': 'Inbound',
  'disposition': 'Answered',
  'duration': 110867,
  'endedAt': '2019-08-23T21:12:55Z',
  'from': {'connectedAt': '2019-08-23T21:11:04Z',
   'departmentName': None,
   'deviceType': None,
   'disconnectedAt': '2019-08-23T21:12:55Z',
   'name': 'blah',
   'number': '1234567890',
   'number_e164': '1234567890',
   'serviceId': None,
   'userId': None},
  'initialQueueName': 'blah',
  'joinedLinkedIds': [],
  'legs': [{'departmentName': 'default',
    'deviceType': 'Unknown',
    'legType': 'Dial',
    'menuName': None,
    'menuOption': None,
    'menuPrompt': None,
    'number': '1234567890',
    'optionAction': None,
    'optionArg': None,
    'queueName': None,
    'serviceId': 327727,
    'timestamp': '2019-08-23T21:11:04Z',
    'userId': None},
   {'departmentName': 'default',
    'deviceType': 'Unknown',
    'legType': 'Answer',
    'menuName': None,
    'menuOption': None,
    'menuPrompt': None,
    'number': '1234567890',
    'optionAction': None,
    'optionArg': None,
    'queueName': None,
    'serviceId': 327727,
    'timestamp': '2019-08-23T21:11:04Z',
    'userId': None},
   {'departmentName': None,
    'deviceType': None,
    'legType': 'EnterIVR',
    'menuName': 'blah',
    'menuOption': None,
    'menuPrompt': None,
    'number': None,
    'optionAction': None,
    'optionArg': None,
    'queueName': None,
    'serviceId': None,
    'timestamp': '2019-08-23T21:11:05Z',
    'userId': None},
   {'departmentName': None,
    'deviceType': None,
    'legType': 'IVRSchedule',
    'menuName': 'Day',
    'menuOption': None,
    'menuPrompt': None,
    'number': None,
    'optionAction': None,
    'optionArg': None,
    'queueName': None,
    'serviceId': None,
    'timestamp': '2019-08-23T21:11:06Z',
    'userId': None},
   {'departmentName': None,
    'deviceType': None,
    'legType': 'EnterQueue',
    'menuName': None,
    'menuOption': None,
    'menuPrompt': None,
    'number': None,
    'optionAction': None,
    'optionArg': None,
    'queueName': 'blah',
    'serviceId': None,
    'timestamp': '2019-08-23T21:11:15Z',
    'userId': None},
   {'departmentName': None,
    'deviceType': None,
    'legType': 'Hangup',
    'menuName': None,
    'menuOption': None,
    'menuPrompt': None,
    'number': 'blah',
    'optionAction': None,
    'optionArg': None,
    'queueName': None,
    'serviceId': None,
    'timestamp': '2019-08-23T21:12:55Z',
    'userId': None}],
  'linkedId': 'some unique key',
  'startedAt': '2019-08-23T21:11:04Z',
  'to': {'connectedAt': '2019-08-23T21:11:04Z',
   'departmentName': 'default',
   'deviceType': 'Unknown',
   'disconnectedAt': '2019-08-23T21:12:55Z',
   'name': None,
   'number': '1234567890',
   'number_e164': '1234567890',
   'serviceId': 327727,
   'userId': None},
  'version': {'label': None, 'major': 4, 'minor': 2, 'point': 1}},

The information I'm trying to get at is in 'legs', where 'legType' == 'Dial' or 'EnterIVR'. I need 'number' from the 'Dial' leg, and 'menuName' from the 'EnterIVR' leg. I can get it, for instance, to list back all the different legTypes, but not the data specifically from those.

This is where I'm at currently:

with open('callstest.csv',mode='w') as calls:
    data_writer = csv.writer(calls, delimiter = ',')
    data_writer.writerow(['LinkedID','Number','Queue','Client'])
    target = response_json['calls']
    glomtemp = {}
    for item in target:
        spec = {
            'Linked ID':'linkedId',
            #this returns the number I need only in certain cases, 
            #so I need 'number' from the 'Dial' legType
            'Number': ('to', 'number') 
            'Queue': 'initialQueueName',
            'Client': #need help here, should be 'menuName' from
                      #'EnterIVR' legType
                }
        glomtemp = glom(item,spec)
        #print(glomtemp)
        data_writer.writerow([glomtemp['Linked ID'],glomtemp['Number'],glomtemp['Queue']])

Right now I can get them to fall back with Coalesce to "None", but that's not what I'm looking for.

Any suggestions on how I should spec this to get the info out of those 2 legs for 'Number' and 'Client'?

score 1 · Answer 1 · answered Aug 29 '19 at 20:30

If I understand correctly, you want to filter out certain entries that don't fit the supported legType. You're definitely onto something with the Coalesce, and I think the key here is glom's Check specifier type, combined with the SKIP singleton. I had to tweak your current spec a bit to match the example data, but this runs:

from glom import glom, Check, Coalesce, SKIP

LEG_SPEC = {'Client': Coalesce('menuName', default=''),
            'Number': Coalesce('to.number', default=''),
            'Linked ID': 'serviceId',
            'Queue': 'queueName'}
entries_spec = ('legs', 
                [Check('legType', one_of=('Dial', 'EnterIVR'), default=SKIP)],
                [LEG_SPEC])
pprint(glom(target, entries_spec))
# prints:
# [{'Client': None, 'Linked ID': 327727, 'Number': '', 'Queue': None},
#  {'Client': 'blah', 'Linked ID': None, 'Number': '', 'Queue': None}]

Not sure if that was exactly what you were hoping to see, but the pattern is there. I think you want Nones (or '') for those other fields because the csv you're writing is going to want to put something in those columns.

There are other ways of doing filtered iteration using glom, too. The snippets page has a short section, complete with examples.

This was originally what I ended up with, but the point was to skip over the legTypes that don't match 'Dial' or 'EnterIVR', because the keys are the same in all the legs, just with different values. I ended up going oldschool and just iterating over the dictionary to get the value I wanted and popping it in a variable, then writing that variable to the CSV. `for item in target: menuname = "" for leg in item['legs']: if leg['legType'] == 'EnterIVR': menuname = leg['menuName'] ])` — David Brown, Aug 29 '19 at 21:17
@DavidBrown This does skip over legTypes that don't match 'Dial' or 'EnterIVR'. Notice your original example had 6 legs in it, and the output of this is two, one from a Dial leg and the other from an EnterIVR leg. — Mahmoud Hashemi, Aug 29 '19 at 22:32

Glom spec for list of dictionaries inside a list of dictionaries

1 Answers1