List Multiples of the Same Variable only Once

Question

I have a python script that looks at a json file and lists variables to a CSV. The problem I am having is latitude and logitude are listed twice. Therefore, when I write the row, it looks at those variables and creates an output with duplicate values.

import csv, json, sys

def find_deep_value(d, key):
# Modified from https://stackoverflow.com/questions/48568649/convert-json-to-csv-using-python/48569129#48569129

    if key in d:
        return d[key]
    for k in d.keys():
        if isinstance(d[k], dict):
            for j in find_deep_value(d[k], key):
                return j

inputFile = open("pywu.cache.json", 'r')  # open json file
outputFile = open("CurrentObs.csv", 'w')  # load csv file
data = json.load(inputFile)  # load json content
inputFile.close()  # close the input file
output = csv.writer(outputFile)  # create a csv.write

# Gives you latitude coordinates from within the json
lat = find_deep_value(data, "latitude")

# Gives you longitude coordinates from within the json
lon = find_deep_value(data, "longitude")

# Gives you a list of weather from within the json
weather = find_deep_value(data, "weather")

# Gives you a list of temperature_strings from within the json
temp = find_deep_value(data, "temperature_string")

output.writerow([lat, lon, weather, temp])

outputFile.close()

Is there a way to only list them once?

Could you add a sample of the data you're loading too, please? Basically it would be sweet if we had everything we need to just run it and see your problem on our computers. — Adam Barnes, Feb 02 '18 at 17:59
I assume your `key` will always be in `d` so your problem is very likely in `if key in d: yield d[key]` since you always return the same value. — Stefan Falk, Feb 02 '18 at 18:05
Do you know about [set](https://docs.python.org/2/library/sets.html) ? — IMCoins, Feb 02 '18 at 18:30

score 0 · Answer 1 · answered Feb 02 '18 at 19:19

0

You need to use return rather than yield. Yield is for generators. Once you fix that, you'll also need to change

list(find_deep_value(data, "latitude"))

to

find_deep_value(data, "latitude")

for each of those lines. And finally, change

output.writerow(lat + lon + weather + temp)

to

output.writerow([lat, lon, weather, temp])

What's happening (you might want to read up on generators first) is when a key is not in the top-level dictionary, you start looping through them, and when the first 'latitude' is reached, the yield keyword returns a generator object. You have that generator wrapped in list() which immediately unpacks the entire generator into a list. So if you have more than one sub-dictionary with the given key in it, you're going to end up looking through and finding every single one.

answered Feb 02 '18 at 19:19

Dan Cusher

121
5

Thanks Dan for your response. I understand what you're saying: list will parse the entire file and return all values matching keys. I made the changes you suggested. However, when running the code, I now get a "TypeError: 'NoneType' object is not iterable" on the lat variable. – user5834454 Feb 02 '18 at 19:36
1

If you get `NoneType object is not iterable`, you might want to ask your question on **Google** to find how to fix this. – IMCoins Feb 02 '18 at 19:47
@user5834454 It's hard to say without seeing your updated code, but consider this: if your `data` doesn't contain `key` in any of its dictionaries...what will `find_deep_value` return? What will the rest of the code do after that? Without seeing the updated code, I would suspect that's part of the problem. – Dan Cusher Feb 02 '18 at 20:10
I updated the code, above with the changes you suggested. – user5834454 Feb 02 '18 at 20:49

List Multiples of the Same Variable only Once

1 Answers1