Check if a value in one Dask dataframe is in another Dask dataframe

Question

I found the answer in this post for pandas dataframe

But when I try to apply it to a Dask dataframe I get a “not implemented” error.

Answer 1

You should load the data into memory, cause dask is lazy loading, it's data not in memory until you execute compute(). and function isin in Dask requires a list-like parameter, not a Dask-object. Official website has instructions as:

Parameters: values:set or list-like

So we can change the example of for pandas dataframe to:

Df1.name.isin(Df2.IDs.compute()).astype(int).compute()

0    1
1    1
2    0
3    0
Name: name, dtype: int64

Refer: Dask.compute