1

Pandas is doing something weird with dtypes here and I am trying to figure out why...

Here is an example pd.DataFrame.

df = pd.DataFrame([[1, 2, 3], [1.1, 2.2, 3.3]]).T.convert_dtypes()

It's df.dtypes are:

0      Int64
1    float64
dtype: object

I need to verify if the columns are the correct datatypes, so I do the following:

df[1].dtype == float

I get True. When I do this for the 0 (int) column:

df[0].dtype == int

I get False

The only way to "verify" the int type it seems to be if I do this: df[0].dtype == pd.core.arrays.integer.Int64Dtype()

Question: Why the inconsistency?

Newskooler
  • 3,973
  • 7
  • 46
  • 84

1 Answers1

2

Pandas has is_integer_dtype just for this:

df.dtypes

0      Int64
1    float64
dtype: object

# Both of these work. You can either pass a column or dtype 
pd.api.types.is_integer_dtype(df[0])
pd.api.types.is_integer_dtype(df.dtypes[0])
# True

Checking floats would be done with is_float_dtype. There are similar function for testing other dtypes, peruse the documentation for more.

cs95
  • 379,657
  • 97
  • 704
  • 746
  • Can I in the same way test if a datetime object is of timezone utc? I see this exists `is_datetime64tz_dtype` but this does not specify the tz. – Newskooler Jul 04 '20 at 21:22
  • @Newskooler Not familiar with any function to do that, although couldn't you just extract the timezone info from your datetime objects and compare them somehow? Maybe [this link](https://stackoverflow.com/questions/6706499/checking-if-date-is-in-utc-format) could help. – cs95 Jul 04 '20 at 21:26
  • Current I do it like so: `df['time'].dtype == pd.core.dtypes.dtypes.DatetimeTZDtype(tz=timezone.utc)` but what you are suggesting may be more robust. I will try. – Newskooler Jul 04 '20 at 21:27
  • 1
    @Newskooler did you check [this link](https://stackoverflow.com/questions/6706499/checking-if-date-is-in-utc-format) as well? If nothing works, please feel free to open a new question! Did you have any other questions about the code posted above? – cs95 Jul 04 '20 at 21:29
  • Thanks for all. That help! – Newskooler Jul 04 '20 at 21:30