0

This is my DataFrame df:

col1       col2
-0.441406  2.523047
-0.321105  1.555589 
-0.412857  2.223047
-0.356610  2.513048

When I check df, I see that there are some infinite values.

np.any(np.isnan(df))
np.all(np.isfinite(df))

False 
True

What is the difference between NaN and infinite? Also, how can I delete all infinite values to get True in np.all(np.isfinite(X))?

This is what I tried:

df = df.replace([np.inf, -np.inf], np.nan).dropna(how="all")

But still the check of infinite gives me True.

enter image description here

Moreover, .apply(lambda s: s[np.isfinite(s)].dropna()).count() gives me the same number of rows of all columns as simply df.shape, which indicates the lack of infinite values. But in this case why np.all(np.isfinite(df)) returns True?

ScalaBoy
  • 3,254
  • 13
  • 46
  • 84

1 Answers1

1

Your question is similar to dropping infinite values from dataframes in pandas?, did you try:

df.replace([np.inf, -np.inf], np.nan).dropna(subset=["col1", "col2"], how="all")

np.nan is not considered as finite, you may replace np.nan by any finite number see that code for example:

import pandas as pd
import numpy as np

df = pd.DataFrame(columns=list('ABC'))
df.loc[0] = [1,np.inf,-np.inf]
print df

print np.all(np.isfinite(df))

df_nan = df.replace([np.inf, -np.inf], np.nan).dropna(subset=df.columns, how="all")
print df_nan

print np.all(np.isfinite(df_nan))

df_0 = df.replace([np.inf, -np.inf], 0).dropna(subset=df.columns, how="all")
print df_0

print np.all(np.isfinite(df_0))

Result:

     A    B    C
0  1.0  inf -inf
False
     A   B   C
0  1.0 NaN NaN
False
     A    B    C
0  1.0  0.0  0.0
True
A. STEFANI
  • 6,707
  • 1
  • 23
  • 48