adding values to defaultdict set breaking on 128

Question

Something always breaks once my address reaches value 128, and it stops putting the new value at the end of the nested dict. And instead starts ordering it from the beginning again. Why is it doing this?

result.setdefault(fan_definitions_name, set()).add(int(address))

The structure of the data I'm looping through looks like this ->

list_address_of_selected_fans = [[101, 'standard fan definition1', 'JVEC-SP'],[102, 'standard fan definition1', 'JVEC-SP']]

EDIT: here is a runnable example where I provide the input. You can run this example directly (I'm using python 3.8.2)

list_address_of_selected_fans = [[101, 'standard fan definition1', 'JVEC-SP'],
                                 [102, 'standard fan definition1', 'JVEC-SP'],
                                 [103, 'standard fan definition1', 'JVEC-SP'],
                                 [104, 'standard fan definition1', 'JVEC-SP'],
                                 [105, 'standard fan definition1', 'JVEC-SP'],
                                 [106, 'standard fan definition1', 'JVEC-SP'],
                                 [107, 'standard fan definition1', 'JVEC-SP'],
                                 [108, 'standard fan definition1', 'JVEC-SP'],
                                 [109, 'standard fan definition1', 'JVEC-SP'],
                                 [110, 'standard fan definition1', 'JVEC-SP'],
                                 [111, 'standard fan definition1', 'JVEC-SP'],
                                 [112, 'standard fan definition1', 'JVEC-SP'],
                                 [113, 'standard fan definition1', 'JVEC-SP'],
                                 [114, 'standard fan definition1', 'JVEC-SP'],
                                 [115, 'standard fan definition1', 'JVEC-SP'],
                                 [116, 'standard fan definition1', 'JVEC-SP'],
                                 [117, 'standard fan definition1', 'JVEC-SP'],
                                 [118, 'standard fan definition1', 'JVEC-SP'],
                                 [119, 'standard fan definition1', 'JVEC-SP'],
                                 [120, 'standard fan definition1', 'JVEC-SP'],
                                 [121, 'standard fan definition1', 'JVEC-SP'],
                                 [122, 'standard fan definition1', 'JVEC-SP'],
                                 [123, 'standard fan definition1', 'JVEC-SP'],
                                 [124, 'standard fan definition1', 'JVEC-SP'],
                                 [125, 'standard fan definition1', 'JVEC-SP'],
                                 [126, 'standard fan definition1', 'JVEC-SP'],
                                 [127, 'standard fan definition1', 'JVEC-SP'],
                                 [128, 'standard fan definition1', 'JVEC-SP'],
                                 [129, 'standard fan definition1', 'JVEC-SP'],
                                 [130, 'standard fan definition1', 'JVEC-SP'],
                                 [131, 'standard fan definition1', 'JVEC-SP'],
                                 [132, 'standard fan definition1', 'JVEC-SP'],
                                 [133, 'standard fan definition1', 'JVEC-SP'],
                                 [134, 'standard fan definition1', 'JVEC-SP'],
                                 [135, 'standard fan definition1', 'JVEC-SP'],
                                 [136, 'standard fan definition1', 'JVEC-SP'],
                                 [137, 'standard fan definition1', 'JVEC-SP'],
                                 [138, 'standard fan definition1', 'JVEC-SP'],
                                 [139, 'standard fan definition1', 'JVEC-SP'],
                                 [140, 'standard fan definition1', 'JVEC-SP'],
                                 [141, 'standard fan definition1', 'JVEC-SP'],
                                 [142, 'standard fan definition1', 'JVEC-SP'],
                                 [143, 'standard fan definition1', 'JVEC-SP'],
                                 [144, 'standard fan definition1', 'JVEC-SP'],
                                 [145, 'standard fan definition1', 'JVEC-SP'],
                                 [146, 'standard fan definition1', 'JVEC-SP'],
                                 [147, 'standard fan definition1', 'JVEC-SP'],
                                 [148, 'standard fan definition1', 'JVEC-SP'],
                                 [149, 'standard fan definition1', 'JVEC-SP'],
                                 [150, 'standard fan definition1', 'JVEC-SP']]
result = {}

for address, fan_definitions_name, fan_type in list_address_of_selected_fans:
    result.setdefault(fan_definitions_name, set()).add(int(address))

print(result)

result ends up being -> {'standard fan definition1': {128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127}}

Why is it doing this? The numbers should be in order such as they are initially in the list_address_of_selected_fans variable.

EDIT2: here is live code showing the issue https://repl.it/talk/share/adding-values-to-defaultdict-set-breaking-on-128/122013

[Please do not upload images of code when asking a question.](//meta.stackoverflow.com/q/285551) Include your code / variables as a [formatted code block](//stackoverflow.com/help/formatting) instead of an image. — Pranav Hosangadi, Feb 12 '21 at 22:38
@PranavHosangadi Thanks for the tip, I'm not experienced with creating many post on this site. From my prescriptive, I did include formatted blocks of code to ask my question, and the image just provides additional information that may or may not be helpful. — Andrew Clark, Feb 12 '21 at 22:57
Andrew, the link to the meta post tells you why we discourage screenshots of text. You can almost always share the same information by pasting the text. — Pranav Hosangadi, Feb 14 '21 at 14:20
@martineau I have provided a runnable code sample that shows my problem. Thank you for your help. I'm not sure what is causing this. — Andrew Clark, Feb 15 '21 at 19:05
Andrew: I can't reproduce the problem. When I run your code, `result` is a dictionary containing one key, value pair. The key is `'standard fan definition1'` and its value is `{101, 102, ..., 150}` that includes _all_ the values between `101` and `150`. This means you haven't provided a [mre]. — martineau, Feb 15 '21 at 19:19
@martineau here I put the code live on replit and it is still creating the same issue. I want the integer values to stay in order, in the same order I'm looping through them. But once I get to 128, it stops putting it at the end and instead starts at beginning again. https://repl.it/talk/share/adding-values-to-defaultdict-set-breaking-on-128/122013 — Andrew Clark, Feb 15 '21 at 19:58

score 2 · Accepted Answer · answered Feb 15 '21 at 20:00

2

The set() object is an unordered collection (see https://docs.python.org/3.8/library/stdtypes.html#set-types-set-frozenset)

For your purposes, if you want them in the order in which they originally appeared and you are going to allow repeats, you could swap out your set() and add() for list() and append():

for address, fan_definitions_name, fan_type in list_address_of_selected_fans:
    result.setdefault(fan_definitions_name, list()).append(int(address))

If you want to ignore repeats, then you'd have to check for those by creating sets in addition to the lists...:

result = {}
result_sets = {}

for address, fan_definitions_name, fan_type in list_address_of_selected_fans:
    iaddress = int(address)
    prev_set = result_sets.get(fan_definitions_name, set())
    if (iaddress not in prev_set):
        result.setdefault(fan_definitions_name, list()).append(iaddress)
        result_sets.setdefault(fan_definitions_name, set()).add(iaddress)

Or, if you want to have it all in one variable and less code, you can accomplish that with OrderedDict like so:

from collections import OrderedDict
result = {}
for address, fan_definitions_name, fan_type in list_address_of_selected_fans:
    result.setdefault(fan_definitions_name, OrderedDict())[int(address)] = None

print(result['standard fan definition1'].keys())

answered Feb 15 '21 at 20:00

Mark H

4,246
3
12

1

PERFECT! THANK YOU MARK! This was exactly what I was looking for. Your first proposed solution seems to be the most appropriate for me and works but I appreciate these other answers too and will learn about them and understand the differences. Thanks again! It says I'll need to wait 23 more hours before I can release bounty, but then it's yours. Thank you – Andrew Clark Feb 15 '21 at 20:06
1

I should clarify the claim I made in the second approach. If you want to ignore repeats and want to avoid searching through a list everytime (i.e. you want the speed benefit of a set), then you have to do set in addition to list. If you didn't care about speed, like when dealing with a small amount of data, you could just do an "in" check to see if the value was already in the list. – Mark H Feb 15 '21 at 20:07
1

Awesome thanks! Glad it works. The quickest way to resolve your confusion might be to look up the difference between hash-based collections and non-hash-based collections. (It's way easier to understand than it sounds) – Mark H Feb 15 '21 at 20:08
1

My data set will never be larger than 50 values, so I think a list will work fine for me. I will look into this hash-based collections vs non-hash so that I'll be knowledgeable in the future! Thanks again, cheers. – Andrew Clark Feb 15 '21 at 20:19

adding values to defaultdict set breaking on 128

1 Answers1