pandas.DataFrame.fillna - TypeError: only integer scalar arrays can be converted to a scalar index

Question

I am trying to do this from the official pandas documentation. pandas.DataFrame.fillna So Basicly filling up the NaN values in the df dataframe's "myc" column with values of 1.

DATA dataframe

df
   myc    B   C  D
0  NaN  2.0 NaN  0
1  0.2  4.0 NaN  1
2  NaN  NaN NaN  5
3  NaN  3.0 NaN  4

CODE 1

values = {'myc': 1}
df.fillna(value=values)

Results Goal 1

    myc B   C   D
0   1.0 2.0 NaN 0
1   0.2 4.0 NaN 1
2   1.0 NaN NaN 5
3   1.0 3.0 NaN 4

ERROR MESAGE 1

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-21-6a9e5a691bca> in <module>
      1 values = {'myc': 1}
----> 2 df.fillna(value=values)

~/anaconda3/lib/python3.8/site-packages/pandas/core/frame.py in fillna(self, value, method, axis, inplace, limit, downcast)
   4315         downcast=None,
   4316     ) -> Optional["DataFrame"]:
-> 4317         return super().fillna(
   4318             value=value,
   4319             method=method,

~/anaconda3/lib/python3.8/site-packages/pandas/core/generic.py in fillna(self, value, method, axis, inplace, limit, downcast)
   6071                     if k not in result:
   6072                         continue
-> 6073                     obj = result[k]
   6074                     obj.fillna(v, limit=limit, inplace=True, downcast=downcast)
   6075                 return result if not inplace else None

~/anaconda3/lib/python3.8/site-packages/pandas/core/frame.py in __getitem__(self, key)
   2876                 if self.columns.nlevels > 1:
   2877                     return self._getitem_multilevel(key)
-> 2878                 return self._get_item_cache(key)
   2879 
   2880         # Do we have a slicer (on rows)?

~/anaconda3/lib/python3.8/site-packages/pandas/core/generic.py in _get_item_cache(self, item)
   3539 
   3540             loc = self.columns.get_loc(item)
-> 3541             values = self._mgr.iget(loc)
   3542             res = self._box_col_values(values, loc)
   3543 

~/anaconda3/lib/python3.8/site-packages/pandas/core/internals/managers.py in iget(self, i)
    986         Return the data as a SingleBlockManager.
    987         """
--> 988         block = self.blocks[self.blknos[i]]
    989         values = block.iget(self.blklocs[i])
    990 

TypeError: only integer scalar arrays can be converted to a scalar index

CODE 2 I have also later on tried to list out the unique features for the any_feature column df['any_feature'].unique()

ERROR 2

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-39-934988075beb> in <module>
----> 1 df['any_feature'].unique()

~/anaconda3/lib/python3.8/site-packages/pandas/core/frame.py in __getitem__(self, key)
   2876                 if self.columns.nlevels > 1:
   2877                     return self._getitem_multilevel(key)
-> 2878                 return self._get_item_cache(key)
   2879 
   2880         # Do we have a slicer (on rows)?

~/anaconda3/lib/python3.8/site-packages/pandas/core/generic.py in _get_item_cache(self, item)
   3539 
   3540             loc = self.columns.get_loc(item)
-> 3541             values = self._mgr.iget(loc)
   3542             res = self._box_col_values(values, loc)
   3543 

~/anaconda3/lib/python3.8/site-packages/pandas/core/internals/managers.py in iget(self, i)
    986         Return the data as a SingleBlockManager.
    987         """
--> 988         block = self.blocks[self.blknos[i]]
    989         values = block.iget(self.blklocs[i])
    990 

TypeError: only integer scalar arrays can be converted to a scalar index

Tried Solutions

Not dictionary - Pandas: Getting "TypeError: only integer scalar arrays can be converted to a scalar index" while trying to merge data frames
Not dictionary - Only integer scalar arrays can be converted to a scalar index how to resolve
not answered - How to resolve Python TypeError: "only integer scalar arrays can be converted to a scalar index"
Not dictionary - TypeError: only integer scalar arrays can be converted to a scalar index with 1D numpy indices array
Not dictionary - numpy array TypeError: only integer scalar arrays can be converted to a scalar index
Not dictionary - TypeError: only integer scalar arrays can be converted to a scalar index when use Pandas Fillna
I don't want to convert it - How to convert index of a pandas dataframe into a column?
I have tried running the following code to test if that object is a dataframe and it said true so it is - https://stackoverflow.com/a/14809149/10270590
- INPUT
```
   isinstance(df, pd.DataFrame)
```
- OUTPUT True

What is panda version you have ? Your code works fine on my side — BENY, Oct 13 '20 at 14:30
I checked it both with pandas 1.1.1 and 1.1.3 there is no problem with pandas. I think you have a problem somewhere in your DF. Maybe numpy version? — Mehdi Golzadeh, Oct 13 '20 at 14:43
The only thing that I have doen before is just to load in the code ```df = pd.read_csv('hello.csv')``` — sogu, Oct 13 '20 at 14:46
Can you share hello .csv somewhere? if it works on another computer the problem would be some of your installed packages. — Mehdi Golzadeh, Oct 13 '20 at 14:50
I am not allowed: but the column that we are looking at "myc" is build out of values between 0-1. the other columns are Numeric and Objects mixed. — sogu, Oct 13 '20 at 14:51
How about ```df['myc'] = df['myc'].fillna(1)```? – Quang Hoang -> ```TypeError: only integer scalar arrays can be converted to a scalar index``` same thing — sogu, Oct 13 '20 at 14:57

score 1 · Answer 1 · answered Oct 13 '20 at 15:07

Something weird is going on in your code, because:

the replacement of NaN should occur only in myc column,
but your result contains replaced values also e.g. in C column and NaN are replaced there with 2.

Run just the below code (separated from your code):

import pandas as pd
import io

txt = '''myc,B,C,D
NaN,2.0,NaN,0
3.0,4.0,NaN,1
NaN,NaN,NaN,5
NaN,3.0,NaN,4'''

df = pd.read_csv(io.StringIO(txt))
result = df.fillna(value={'myc': 1})

The result should be:

   myc    B   C  D
0  1.0  2.0 NaN  0
1  3.0  4.0 NaN  1
2  1.0  NaN NaN  5
3  1.0  3.0 NaN  4

If you get the same result, then apparently there is something wrong with your code, but in some other place (outside the piece of code that you presented).

Another detail to change is that values is an attribute of Pandas and you should not use variables with the same names.

I have made corrections, sorry I have typed in here my DF so I have randomly put there the values. — sogu, Oct 13 '20 at 15:10

score 0 · Accepted Answer · answered Oct 14 '20 at 14:05

0

Simple Solution

Export dataframe to csv: df.to_csv(r'somefilename.csv', index=False)
Load back the saem data to a DataFrame: df1 = pd.read_csv("r'somefilename.csv")

answered Oct 14 '20 at 14:05

sogu

2,738
5
31
90

pandas.DataFrame.fillna - TypeError: only integer scalar arrays can be converted to a scalar index

2 Answers2

Linked