Dask equivalent to Pandas replace?

Question

Something I use regularly in pandas is the .replace operation. I am struggling to see how one readily performs this same operation on a dask dataframe?

df.replace('PASS', '0', inplace=True)
df.replace('FAIL', '1', inplace=True)

That should be quite simple to implement in dask if you raise an issue on the tracker: https://github.com/dask/dask — TomAugspurger, Nov 30 '16 at 22:10

score 15 · Accepted Answer · answered Nov 30 '16 at 22:33

15

You can use mask:

df = df.mask(df == 'PASS', '0')
df = df.mask(df == 'FAIL', '1')

Or equivalently chaining the mask calls:

df = df.mask(df == 'PASS', '0').mask(df == 'FAIL', '1')

answered Nov 30 '16 at 22:33

root

32,715
6
74
87

Perfect! Thank you! I kept playing around with `assign` and `where` but to no avail. I obviously didn't quite make the connection to the functionality `mask` provides. – docross Nov 30 '16 at 23:05

score 3 · Answer 2 · answered Oct 05 '18 at 15:08

3

If anyone would like to know how to replace certain values in a specific column, here's how to do this:

def replace(x: pd.DataFrame) -> pd.DataFrame:
    return x.replace(
      {'a_feature': ['PASS', 'FAIL']},
      {'a_feature': ['0', '1']}
    )
df = df.map_partitions(replace)

Since we operate on a pandas' DataFrame here, please refer to the documentation for further information

answered Oct 05 '18 at 15:08

LiquidSpirit

31
1

In the [map_partition](https://docs.dask.org/en/latest/dataframe-api.html#dask.dataframe.DataFrame.map_partitions) you may want to add the `meta` argument – skibee May 07 '19 at 14:10

Dask equivalent to Pandas replace?

2 Answers2

Linked