52

I want to create a modified dataframe with the specified columns. I tried the following but throws the error "Passing list-likes to .loc or [] with any missing labels is no longer supported"

# columns to keep
filtered_columns = ['text', 'agreeCount', 'disagreeCount', 'id', 'user.firstName', 'user.lastName', 'user.gender', 'user.id']
tips_filtered = tips_df.loc[:, filtered_columns]

# display tips
tips_filtered

Thank you

swasthik nayak
  • 605
  • 1
  • 5
  • 11
  • Check for typos or leading/trailing spaces in the `df.columns` too , if necessary you can `str.strip()` them off. Some/any of the columns is not present in the actual dataframe – anky Apr 18 '20 at 15:44

5 Answers5

52

It looks like Pandas has deprecated this method of indexing. According to their docs:

This behavior is deprecated and will show a warning message pointing to this section. The recommended alternative is to use .reindex()

Using the new recommended method, you can filter your columns using:

tips_filtered = tips_df.reindex(columns = filtered_columns).

NB: To reindex rows, you would use reindex(index = ...) (More information here).

Joel Oduro-Afriyie
  • 1,563
  • 14
  • 13
16

Some of the columns in the list are not included in the dataframe , if you do want do that , let us try reindex

tips_filtered = tips_df.reindex(columns=filtered_columns)
Code42
  • 2,292
  • 1
  • 17
  • 22
BENY
  • 317,841
  • 20
  • 164
  • 234
15

I encountered the same error with missing row index labels rather than columns.
For example, I would have a dataset of products with the following ids: ['a','b','c','d']. I store those products in a dataframe with indices ['a','b','c','d']:

df=pd.DataFrame(['product a','product b','product c', 'product d'],index=['a','b','c','d'])

Now let's assume I have an updated product index: row_indices=['b','c','d','e'] in which 'e' corresponds to a new product: 'product e'. Note that 'e' was not present in my original index ['a','b','c','d'].

If I try to pass this updated index to my df dataframe: df.loc[row_indices,:],

I'll get this nasty error message:

KeyError: "Passing list-likes to .loc or [] with any missing labels is no longer supported. The following labels were missing: Index(['e'], dtype='object').

To avoid this error I need to do intersection of my updated index with the original index:

df.loc[df.index.intersection(row_indices),:]

this is in line with recommendation of what pandas docs

denis_smyslov
  • 741
  • 8
  • 8
  • 1
    I encountered this problem today, and you save my life. Thank you so much @Kuffner – Hamdi Tarek Jan 09 '21 at 18:23
  • Thank you for this - I needed a quick method for reindexing stock price data such that there are ffilled prices on non-market days (just reindex with ffill and a date range, no prob), but I needed to also indicate which were the market days - I saved my original index, and then went to use .loc with the original index to set MarketDay=True - none of the normal methods worked - this method was the only thing that worked. So I thank you. – James_SO Mar 12 '21 at 19:10
1

This error pops up if indexing on something which is not present - reset_index() worked for me as I was indexing on a subset of the actual dataframe with actual indices, in this case the column may not be present in the dataframe.

Prerna
  • 11
  • 1
  • 1
    Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-ask). – Community Sep 22 '21 at 08:51
0

I had the same issue while trying to create new columns along with existing ones :

df = pd.DataFrame([[1,2,3]], columns=["a","b","c"])
def foobar(a,b):
  return a,b
df[["c","d"]] = df.apply(lambda row: foobar(row["a"], row["b"]), axis=1)

The solution was to add result_type="expand" as an argument of apply() :

df[["c","d"]] = df.apply(lambda row: foobar(row["a"], row["b"]), axis=1, result_type="expand")
Skippy le Grand Gourou
  • 6,976
  • 4
  • 60
  • 76