Where is official documentation for tilde (~) in Pandas?

Question

I am pretty sure that ~ in Pandas is boolean not. I found a couple of StackOverflow questions / answers, but no pointer to official documentation.

Sanity Check

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import pandas as pd


df = pd.DataFrame([(1, 2, 1),
                   (1, 2, 2),
                   (1, 2, 3),
                   (4, 1, 612),
                   (4, 1, 612),
                   (4, 1, 1),
                   (3, 2, 1),
                   ],
                  columns=['groupid', 'a', 'b'],
                  index=['India', 'France', 'England', 'Germany', 'UK', 'USA',
                         'Indonesia'])

print(df)
filtered = df[~(df['a'] == 2)]
print(filtered)

The df is

           groupid  a    b
India            1  2    1
France           1  2    2
England          1  2    3
Germany          4  1  612
UK               4  1  612
USA              4  1    1
Indonesia        3  2    1

and filtered is

         groupid  a    b
Germany        4  1  612
UK             4  1  612
USA            4  1    1

So I'm pretty sure it is boolean not.

cs95 · Accepted Answer · 2018-03-12T14:14:23.867

11

The ~ is the operator equivalent of the __invert__ dunder which has been overridden explicitly for the purpose performing vectorized logical inversions on pd.DataFrame/pd.Series objects.

s = pd.Series([True, False])

~s

0    False
1     True
dtype: bool

s.__invert__()

0    False
1     True
dtype: bool

Note: Dunder methods must not be used directly in code, always prefer the use of the operators.

Also, since you've asked, the section on Boolean Indexing describes its use.

Another common operation is the use of boolean vectors to filter the data. The operators are: | for or, & for and, and ~ for not. These must be grouped by using parentheses.

Bold emphasis mine.

edited Mar 12 '18 at 14:14

answered Mar 12 '18 at 13:55

cs95

379,657
97
704
746

Nice explanation. However I believe this operator is overridden from standard python [__invert__](https://docs.python.org/3/reference/datamodel.html#object.__invert__) to support vectorised behaviour. Similar to the rest of pandas operators. – Sohaib Farooqi Mar 12 '18 at 14:13
Thanks, that terminology needed fixing. – cs95 Mar 12 '18 at 14:14

score 1 · Answer 2 · edited Mar 13 '18 at 05:51

1

I found it referenced on this page. It's about halfway down, I'd navigate to it with ctrl+F. You're correct though, it's the not operator.

edited Mar 13 '18 at 05:51

cs95

379,657
97
704
746

answered Mar 12 '18 at 13:54

M Dillon

164
1
8

Please include all relevant portions of cited links in your answer. – cs95 Mar 12 '18 at 14:02

score 1 · Answer 3 · answered Mar 12 '18 at 13:58

1

Here they define explicitly:

Another common operation is the use of boolean vectors to filter the data. The operators are: | for or, & for and, and ~ for not. These must be grouped by using parentheses.

answered Mar 12 '18 at 13:58

zipa

27,316
6
40
58

score 0 · Answer 4 · answered Mar 12 '18 at 13:56

0

If you go to: https://docs.python.org/3/library/operator.html, it says:

Bitwise Inversion   ~ a invert(a)

answered Mar 12 '18 at 13:56

TYZ

8,466
5
29
60

But does Pandas overwrite `~` with a magic method? I guess that is possible with [`__neg__`](https://docs.python.org/3/reference/datamodel.html#object.__neg__)? – Martin Thoma Mar 12 '18 at 14:06

Where is official documentation for tilde (~) in Pandas?

Sanity Check

4 Answers4