Stack Overflow
Stack Overflow

Questions tagged [isin]

isin is a concept of checking if some value is contained in a list; That concept is used in Python with [pandas] and [numpy]

isin is a concept of checking if some value is contained in a list; That concept is used in Python with [pandas] and [numpy]

147 questions
2
votes
1 answer

Column Values still shown after .isin()

As requested, here is a minimal reproducable example that will generate the issue of .isin() not dropping the values not in .isin() but just setting them to zero: import os import pandas as pd df_example = pd.DataFrame({'Requesting as': {0:…
Hurzinger
  • 93
  • 1
  • 8
2
votes
1 answer

Pandas dataframe select Columns based on other dataframe contains column value in it

I have two dataframes. Here is dwpjp.head(): jp_number 0 25146315052147720191 1 57225427599900052634 2 86076681691411639833 3 50491824499499656478 4 95588382889227620465 and…
DKBOSS
  • 113
  • 1
  • 7
2
votes
3 answers

Is there method faster than np.isin for large array?

For large array(n>1e8), is there any faster way than np.isin for checking whether there are same elements? I have tried several method like pandas isin, cython but all of those takes more time than np.isin example: (Test whether each element of a…
SY Jeon
  • 29
  • 4
2
votes
1 answer

pandas isin function on a for loop

1.csv cut price depth carat table 0 Good 327 57.9 0.23 65.0 1 Good 335 63.3 0.31 58.0 2 Very Good 336 62.8 0.24 57.0 3 Very Good 336 62.3 0.24 57.0 4 Very Good 337 61.9 0.26 55.0 5 Premium 326 …
Vishal Vijayan
  • 57
  • 1
  • 2
  • 12
1
vote
0 answers

PySpark: join using isin to find if a column in one dataframe is substring of another column of another dataframe

I have tried searching if someone has asked this question about PySpark but I had no success. I have a DataFrame of messy names, called df1 (as indicated in the image) and I prepared a DataFrame of clean names, called df2 (see the image). How can I…
jota_ele_a
  • 11
  • 3
1
vote
2 answers

Pandas matching column isin another (list) column (broadcasting `.isin`)

In a workflow matching up a spec against some allowed values, I wish to find which rows (index) are matching a spec. This is different from Pandas, isin, column of lists, as I an not matching each row against a (static) list. I can do it with…
Helge Jensen
  • 133
  • 10
1
vote
1 answer

Exclude/Filter values from dataframe with function .isin() in Pandas

I'm working on a Pandas dataframe with transactional data (customer purchases) and want to exclude rows with certain customer numbers contained in a column 'CUSTOMER_ID'. To achieve this, I created a list with the customer numbers to be…
codesign
  • 15
  • 2
1
vote
1 answer

Retrieve rows from dataframe don't exit in another dataframe pandas

I need rows from df1 doesn't exist in df2 based on 3 columns [Time1, ID1, Order1]. I need df3 has rows of df1 don't exist on df2 Note: Time1 is in datetime format, Example input df1 Time1 ID1 Order1 12/14/2022 6:10:32 PM X A 9/15/2022 …
asmaa mahmoud
  • 73
  • 5
1
vote
1 answer

Flag column values that are not present in another dataframe

I have a benchmark df_1: Col_1 insight_id Col_2 Col_n 24249 ABC123 656 AAA 24249 ABC123 670 AXA 22549 ABC124 656 AAC 24249 ABC124 656 ADA 24236 ABC125 656 …
johnnydoe
  • 382
  • 2
  • 12
1
vote
4 answers

Pandas: check a sequence in one column for each unique value in another column

I have a table that looks like this: Date Unique id Indicator 2018 1 1 2019 1 0 2020 1 0 2020 2 1 2018 2 0 2019 2 1 2020 2 1 2021 2 1 For each value in "Unique id" I want to check whether "Indicator" match a special…
Nauan Sarbasov
  • 13
  • 3
1
vote
1 answer

Find last available date if date does not exist in other DataFrame

Suppose that you have two data frames which can be created using code below: df1 = pd.DataFrame(data={'start_date': ['2021-07-02', '2021-07-09', '2021-07-16', '2021-07-23', …
Lopez
  • 461
  • 5
  • 19
1
vote
0 answers

groupby ID in a dataframe1 then use isin function in the grouped by data frame, with another dataframe2, and output a data frame as well

I have two data frames: first one is this : which has 3 columns (ID , Name, Salary) and the seconed data frame is this: which has 2 columns (ID , Name) I want to groupby ID in the first data frame (def1) after that I want to check if this ID…
sunshine
  • 15
  • 5
1
vote
2 answers

Change values in a column to np.nan based upon row index

I want to selectively change column values to np.nan. I have a column with a lot of zero (0) values. I am getting the row indices of a subset of the total. I place the indices into a variable (s0). I then use this to set the column value to np.nan…
MarkS
  • 1,455
  • 2
  • 21
  • 36
1
vote
2 answers

Delete row indices based on common columns in a Dataframe

I have following two dataframes df1 and df2 final raw st abc 12 10 abc 17 15 abc 14 17 and final raw abc 12 abc 14 My expected output is final raw st abc 17 15 I would like to delete rows based on…
Manglu
  • 258
  • 2
  • 10
1
vote
0 answers

Remove all rows with specific ID if other column condition is met

I have a dataframe: id country 1 usa 1 mex 1 de 2 br 2 mex 3 usa I want to remove all Ids that country == usa Desired output: id country 2 br 2 mex
vando
  • 37
  • 6
1
2
3
…
9 10