How to assert lists with pytest in a stable way?

Question

After migrating a Python application from 2.6 to 2.7 I have learned from my pytest results that the order of a list has changed. The content of the list is the result of a third party openLDAP library.

This is the list in Python 2.6:

assert ['1', '2', '8'] == ['1', '2', '8']

But with Python 2.7 the order of the list has changed, which results in a AssertionError:

assert ['1', '8', '2'] == ['1', '2', '8']

What is the best advice, to change the implementation to get ordered lists, or change the test i.e. by converting all lists to a set for comparing the assert result in a stable way?

score 15 · Accepted Answer · answered Nov 23 '18 at 09:37

You have a couple of options, depending on the nature of your data. I assume you require your test to pass if the elements are the same without regard to order.

If you can guarantee all items in your list are unique, use set:

assert set(['1', '8', '2']) == set(['1', '2', '8'])

If you cannot guarantee there are no duplicates, use sorted:

assert sorted(['1', '8', '2']) == sorted(['1', '2', '8'])

score 1 · Answer 2 · answered Feb 24 '22 at 21:27

There's a pytest plugin called pytest-unordered that's designed to not only handle this basic use case, but can also be applied at various levels of your nested object hierarchy.

pip install pytest-unordered

from pytest_unordered import unordered

assert ['1', '8', '2'] == unordered(['1', '2', '8'])

If you have an object hierarchy made up of dictionaries and lists that you're trying to compare, you can even use a recursive method like this to make all the lists unordered:

def unordered_deep(obj):
    if isinstance(obj, dict):
        return dict((k, unordered_deep(v)) for k, v in obj.items())
    if isinstance(obj, list):
        return unordered((unordered_deep(x) for x in obj))
    return obj

assert actual_results == unordered_deep(expected_results)

How to assert lists with pytest in a stable way?

2 Answers2