Sort list while pushing None values to the end

Question

I have a homogeneous list of objects with None, but it can contain any type of values. Example:

>>> l = [1, 3, 2, 5, 4, None, 7]
>>> sorted(l)
[None, 1, 2, 3, 4, 5, 7]
>>> sorted(l, reverse=True)
[7, 5, 4, 3, 2, 1, None]

Is there a way without reinventing the wheel to get the list sorted the usual python way, but with None values at the end of the list, like that:

[1, 2, 3, 4, 5, 7, None]

I feel like here can be some trick with "key" parameter

Are the values guaranteed to be `int`s or `None`, or do you need `None` to sort after all arbitrary objects? — abarnert, Aug 23 '13 at 20:49
No, it can be string or None, float and None. Generally, it's homogeneous list with None elements — Nikolai Golub, Aug 23 '13 at 20:54
Does the list always contains a `none`, can it be `np.inf` or `np.nan`? — CT Zhu, Aug 23 '13 at 20:55
@NikolayGolub: For future reference, it's better to make information like that clear in the question—after all, SethMMorton's answer fits what you asked, yet doesn't fit what you actually wanted, so you wouldn't have had a good solution if F.J hadn't happened by. — abarnert, Aug 23 '13 at 20:57
@abarnert I was going by the first sentence, "I have homogeneous list of objects with None", so I assumed that the OP's list was *not* heterogeneous. — SethMMorton, Aug 23 '13 at 22:50
@SethMMorton: Yes, but a list of strings is also homogenous, and won't work properly with `float('inf')`. That's what I mean—your answer was perfect for the question as written, but it wasn't what he actually needed, and there's no way you could have guessed that. — abarnert, Aug 23 '13 at 23:42
@abarnert I see what you are saying. I guess this goes back to the XY discussion we had from a few days ago. I now understand why it's important to ask "Why?". — SethMMorton, Aug 24 '13 at 00:13
@SethMMorton: Well, it never hurts to guess at the "Why?" and provide an answer. If you're right, you save everyone time. And even if you guess wrong, your answer could easily be exactly what someone else who searches for this problem in the future needs. — abarnert, Aug 24 '13 at 00:26
@abarnert: Seth's solution doesn't work for the original question… — Neil G, Aug 25 '13 at 11:42
@NeilG The original question was for "A list like this :`[1, 3, 2, 5, 4, None, 7]`", which is all integers and `None`. It wasn't till the edit that it was specified to be a "homogeneous list of objects with None, but it can contain any type of values". So my answer works for the pre-edit question, but not post edit question. — SethMMorton, Aug 25 '13 at 20:14
This won't work in Py3 due to None comparison issues, right? — Pavel Šimerda, Mar 08 '16 at 12:51
@PavelŠimerda Both answers work because there is no comparison of `None` to other types - it is transformed first. — SethMMorton, Mar 08 '16 at 14:07
@SethMMorton You are only partially right. It will indeed work but for a different reason. You are stating the obvious that `None` will be transformed. But you ignore that `None` will remain as the second tuple item in its transformed form `(True, None)`. The actual mechanism is that `(True, None)` will only ever be compared to `(True, None)`. Obviously you need to include `None` more than once to test it. — Pavel Šimerda, Mar 08 '16 at 16:46
@PavelŠimerda No, I did not ignore that `None` remains the second item in the tuple. Since the first item is `True` for `None` and only `None`, no objects will ever be compared to `None` because the first element protects us from the comparison. This is the transformation that I spoke of. However, we shouldn't forget that neither answer protects us from truly heterogeneous lists (i.e. containing both strings and numbers) raising `TypeErrors` on Python 3. — SethMMorton, Mar 08 '16 at 17:08
@SethMMorton Sure but that's another story. One would have to create a more comprehensive transformation, e.g. `lambda x: (type_order[type(x)], x)` with `type_order = { int: 0, str: 1, type(None): 2 }`. — Pavel Šimerda, Mar 08 '16 at 22:20

score 206 · Accepted Answer · answered Aug 23 '13 at 20:48

206

>>> l = [1, 3, 2, 5, 4, None, 7]
>>> sorted(l, key=lambda x: (x is None, x))
[1, 2, 3, 4, 5, 7, None]

This constructs a tuple for each element in the list, if the value is None the tuple with be (True, None), if the value is anything else it will be (False, x) (where x is the value). Since tuples are sorted item by item, this means that all non-None elements will come first (since False < True), and then be sorted by value.

answered Aug 23 '13 at 20:48

Andrew Clark

202,379
35
273
306

3

I like this solution, because it fits for all type of homogeneous litst – Nikolai Golub Aug 23 '13 at 20:52
@NikolayGolub: It even supports heterogeneous lists (so long as your heterogeneous types are all comparable in the first place, of course). – abarnert Aug 23 '13 at 20:58
27

If using `reverse=True` with this solution, use `x is not None` in the key, otherwise all None values are returned first. – tutuDajuju Nov 26 '17 at 08:17
2

@tutuDajuju You don't need tuples with True and False in the first place if you want to sort by descending value and None last. You can simply do `sorted(l, reverse=True)` for `[7, 5, 4, 3, 2, 1, None]` as seen in the original question. – CodeManX Apr 25 '18 at 14:46
3

@tutuDajuju is right if sorting something like datetimes. – David Feb 18 '20 at 18:47
this would not work however if there were multiple "None"s – Akin Hwan Dec 11 '20 at 14:53
1

It does handle the cases of multiple `None`s in the list. All the Nones get pushed to the end. – codeforester Feb 12 '21 at 04:11
`sorted(l, key=lambda x: (x is not None, x), reverse=True)` sorts it like this `[7, 5, 4, 3, 2, 1, None]` – Avishkar Yonjan Tamang Apr 04 '22 at 06:45
In 3.9, @CodeManX, I get `TypeError: '<' not supported between instances of 'int' and 'NoneType'` for your case. – Dave Liu Apr 28 '22 at 08:50
1

@DaveLiu It works in Python 2.x (tested with 2.7.18) but apparently not in 3.x (tested with 3.2..3.9). – CodeManX May 02 '22 at 21:14
For list of dicts you can set the key like this: result = sorted(items, key=lambda x: (x['some_key'] is not None, x['some_key'])) (descending order) or result = sorted(items, key=lambda x: (x['some_key'] is None, x['some_key'])) (ascending order) – Robert Nagtegaal May 17 '22 at 11:07

score 21 · Answer 2 · answered Aug 23 '13 at 20:48

21

Try this:

sorted(l, key=lambda x: float('inf') if x is None else x)

Since infinity is larger than all integers, None will always be placed last.

answered Aug 23 '13 at 20:48

SethMMorton

45,752
12
65
86

1

WOW! I didn't know about this "inf" awesome! – Aug 23 '13 at 20:56
4

This doesn't work if the list contains infinities or if the list is a list of strings. – Neil G Aug 25 '13 at 11:41
@NeilG Yes, we covered that in the comments to the question itself. This answer was for the question as it was originally posted, but will still work for integers only so I didn't delete it. – SethMMorton Aug 25 '13 at 19:54

score -1 · Answer 3 · answered Jun 06 '18 at 12:47

I created a function that expands on the answer by Andrew Clark and the comment by tutuDajuju.

def sort(myList, reverse = False, sortNone = False):
    """Sorts a list that may or may not contain None.
    Special thanks to Andrew Clark and tutuDajuju for how to sort None on https://stackoverflow.com/questions/18411560/python-sort-list-with-none-at-the-end

    reverse (bool) - Determines if the list is sorted in ascending or descending order

    sortNone (bool) - Determines how None is sorted
        - If True: Will place None at the beginning of the list
        - If False: Will place None at the end of the list
        - If None: Will remove all instances of None from the list

    Example Input: sort([1, 3, 2, 5, 4, None, 7])
    Example Input: sort([1, 3, 2, 5, 4, None, 7], reverse = True)
    Example Input: sort([1, 3, 2, 5, 4, None, 7], reverse = True, sortNone = True)
    Example Input: sort([1, 3, 2, 5, 4, None, 7], sortNone = None)
    """

    return sorted(filter(lambda item: True if (sortNone != None) else (item != None), myList), 
        key = lambda item: (((item is None)     if (reverse) else (item is not None)) if (sortNone) else
                            ((item is not None) if (reverse) else (item is None)), item), 
        reverse = reverse)

Here is an example of how you can run it:

myList = [1, 3, 2, 5, 4, None, 7]
print(sort(myList))
print(sort(myList, reverse = True))
print(sort(myList, sortNone = True))
print(sort(myList, reverse = True, sortNone = True))
print(sort(myList, sortNone = None))
print(sort(myList, reverse = True, sortNone = None))

When your function gets that large, why not break it out into a proper function definition? Leaving it as a lambda just makes it hard to read. — Harabeck, Jun 13 '19 at 14:01

Sort list while pushing None values to the end

3 Answers3

Linked

Related