Python nested update cursor with nested dictionary. There's got to be an easier way

Question

I've created a nested dictionary calling values from a table, and I need to update the attribute table for a feature class using that data. I have it working with two hard-coded fields as a test, but I need to figure out how to automate getting the length of featFields and using that to indicate the index position for each field to be updated. So, instead of hard-coding row[1], row[2], etc. and 'LOCDESC' and 'RIMELEV', I'd be using a variable to step through index positions for each one.

I am working in Python. End goal is a toolbox for use in ArcMap 10.2 or 10.3.

import arcpy
arcpy.env.workspace = r"C:/SARP10/MACP_Tool"

#Define fields to update and the field to use as join field

Table = "Test2.csv"
Input = "Test.gdb/MHs"
csvFields = ['Location_Details', 'Elevation']
featFields = ['LOCDESC', 'RIMELEV']
csvKey = "Manhole_Number"
featKey = "FACILITYID"
csvFields.insert(0, csvKey)
featFields.insert(0, featKey)
print csvFields
#Create dictionary to store values from the update table
UpdateDict = {}

#Iterates through the values in the table and stores them in UpdateDict

with arcpy.da.SearchCursor(Table, csvFields) as cursor:
    for row in cursor:
        UpdateDict[row[0]] = dict(zip(featFields[1:], row[1:]))
    print UpdateDict

MHNum = len(UpdateDict) # gets # of MHs to be updated
MHKeys = UpdateDict.keys() # gets key values, i.e. MH numbers

print "You are updating fields for the following {} manholes: {}".format(MHNum, MHKeys)

#Iterates through feature class attribute table and updates desired attributes
with arcpy.da.UpdateCursor(Input, featFields) as cursor:
    i = 0
    z = 0

    for row in cursor:
        i += 1
        for f in UpdateDict.keys():
            if f == row[0]:
                row[1] = UpdateDict.values()[z]['LOCDESC']#uses counter and subdict key to call correct value
                row[2] = UpdateDict.values()[z]['RIMELEV']#uses counter and subdict key to call correct value
                cursor.updateRow(row)
                z +=1 #counter keeps track of rows and provides index location for dictionary
                print "Updating {} of {} manholes in this submittal: {}.".format(z, MHNum, f)
            else:
                pass
print "Updated {} of {} rows.".format(MHNum, i)
print "Script completed."

score 0 · Answer 1 · answered Jan 02 '16 at 15:20

0

Since the (currently hard-coded) iteration of row[n] steps through the values of featFields, you can set up an for loop that iterates through them both, something like:

if f == row[0]:
    # loop set by length of featFields list
    for j in range(0, len(featFields) - 1):
        row[j + 1] = UpdateDict.values()[z][featFields[j]]
        cursor.updateRow(row)
        # etc.

Note the "offset" -- row[1] should be using featFields[0] and so on -- that needs to be accounted for.

answered Jan 02 '16 at 15:20

Erica

2,399
5
26
34

I tried a loop like this last night, but it throws a RuntimeError: "The value type is incompatible with the field type". The hard coded version works fine, so I'm wondering if there's an issue with the nested logic. I ran your code exactly except for one change: I started with j in range(1, len(featFields) -1) to exclude the key 'FACILITYID' added to index position 0. Getting that error again... can anyone tell me why? – Ellie Maclin Jan 02 '16 at 18:14
The _value type incompatible_ error sounds like it is trying to put a string value into a numeric field (i.e., `RIMELEV` may be a string `'3'` instead of an integer `3`). – Erica Jan 02 '16 at 18:23
My instinct says the same thing... but then why does the hard coded version work with no errors? What's different in the looped version? That's what I need to ID and fix, and I'm stumped. – Ellie Maclin Jan 02 '16 at 19:36
Something to do with the values not matching up correctly? Have it print out the value of `j` and `featFields[j]` each time you're going through the loop, and make sure that they're what you expect to see. Use `type()` to show you what data type the variable is (e.g. of `featFields[j]`) to verify that is the problem. – Erica Jan 02 '16 at 19:42
It likely has to do with the nested dictionary order. This is the dictionary: {u'WN010136': {'LOCDESC': u'IN STREET', 'RIMELEV': 237.7352603}, u'WN010134': {'LOCDESC': u'IN STREET', 'RIMELEV': 234.0856613}, u'WN010135': {'LOCDESC': u'IN STREET', 'RIMELEV': 232.3809403}, u'WN010133': {'LOCDESC': u'IN STREET AT BRIDGE', 'RIMELEV': 238.6215228}, u'WN010130': {'LOCDESC': u'IN STREET', 'RIMELEV': 250.7409211}, u'WN010131': {'LOCDESC': u'IN STREET AT BRIDGE', 'RIMELEV': 239.0950564}} – Ellie Maclin Jan 02 '16 at 19:51
I'm running out of ideas, unfortunately :( – Erica Jan 02 '16 at 20:18

score 0 · Accepted Answer · answered Jan 05 '16 at 16:08

The problem was with accessing the right fields in the data dictionary. Final code accesses a list of outer keys and a list of inner key: value pairs with a variable (z) set to keep the index number equal in both lists. Thanks for your help, @Erica!

Here's what works:

import arcpy
arcpy.env.workspace = r"C:/SARP10/MACP_Tool"

#Defines fields to update and the field to use as join field
Table = "Test2.csv"
Input = "Test.gdb/MHs"
csvFields = ['Location_Details', 'Elevation', 'Rim_to_Invert', 'Rim_to_Grade', 'Cover_Size', 'Wall_Material', 'Wall_Diam', 'Wall_Lining_Interior', 'Photo2_Link', 'MH_InspectReportLink'] #update table fields
featFields = ['LOCDESC', 'RIMELEV', 'RIMTOINVERT', 'RIMTOGRADE','COVERSIZE','WALLMAT','DIAMETER','LINERTYPE','HYPERLINK_PHOTO2','HYPERLINK_RPT']#fc field names
csvKey = "Manhole_Number"
featKey = "FACILITYID"
csvFields.insert(0, csvKey)
featFields.insert(0, featKey)
print "Your table contains the following fields to be updated: {}\n".format(str(csvFields))

#Process: Create dictionary to store values from the update table, iterate through values and store in UpdateDict
UpdateDict = {}

with arcpy.da.SearchCursor(Table, csvFields) as cursor:
   for row in cursor:
      UpdateDict[row[0]] = dict(zip(featFields[1:], row[1:]))
## debug print "You have created update dictionary 'UpdateDict': \n{}\n\n".format(UpdateDict)

MHNum = len(UpdateDict) # gets # of MHs to be updatedMHKeys = sorted(UpdateDict.keys()) # gets key values, i.e. MH numbers
MHKeys = UpdateDict.keys() #calls outer keys (MH numbers, which are join values) into a list of keys
MHVals = UpdateDict.values()#calls inner nested key:value pairs to a list

##debug print "Dictionary keys: {}\n\n Dictionary values: {}\n\n".format(str(MHKeys),str(MHVals))
print "You are updating fields for the following {} manholes: {}".format(MHNum, str(MHKeys))

#Process: Iterates through feature class attribute table and updates desired attributes
with arcpy.da.UpdateCursor(Input, featFields) as curs:
    i = 0 #attribute table row counter
    for row in curs:
        i += 1
        for f in MHKeys:
            if f == row[0]:
                z = MHKeys.index(f)#get index location in MHKeys
                for y in range(0,len(featFields)-1):
                    row[y+1] = MHVals[z][featFields[y+1]]#use z to pull corresponding value in MHVals to correct key in MHKeys
                print "Current MH: {} \nUpdating Values: {} \n\n".format(f, UpdateDict.values()[z])
                curs.updateRow(row)
            else:
                pass

print "Updated {} of {} rows.".format(MHNum, i)
print "Script completed."

Python nested update cursor with nested dictionary. There's got to be an easier way

2 Answers2