How to change inf values in numpy array for the previous non inf value?

Question

I have a numpy array that may contain inf values.

The numpy array is a 1D vector of numbers.

Is there a way to change the inf values of the array for the previous value of the array (which is not inf)?

So if the 1000th index of the array is an inf it should replace it by the 999th index which is not inf.

Heres an example of what I want

vals = np.random.random(10000)
vals[vals<0.1] = np.inf

indexes = np.asarray(vals==np.inf).nonzero()

for i in indexes:
    vals[i] = vals[i-1]

if np.isinf(vals).any():
    print("It doesnt work")
else:
    print("It works")

what do you mean by the previous value of the array , is it elements of any array ? — rainu, Dec 23 '20 at 18:28
Ive edited the question to make it more clear. Does that answer the question? — Daniel Casasampera, Dec 23 '20 at 18:51

Rajnish kumar · Answer 1 · 2020-12-23T19:16:13.197

1

def pandas_fill(arr):
    df = pd.DataFrame(arr)
    df.fillna(method='ffill', axis=1, inplace=True)
    out = df.as_matrix()
    return out

def numpy_fill(arr):
    mask = np.isnan(arr)
    idx = np.where(~mask,np.arange(mask.shape[1]),0)
    np.maximum.accumulate(idx,axis=1, out=idx)
    out = arr[np.arange(idx.shape[0])[:,None], idx]
    return out

inf and -inf will be loaded as nan. So, this should be handled with that.

Try out this updated one.

import numpy as np

Data = np.array([np.nan,1.3,np.nan,1.4,np.nan,np.nan])

nansIndx = np.where(np.isnan(Data))[0]
isanIndx = np.where(~np.isnan(Data))[0]
for nan in nansIndx:
    replacementCandidates = np.where(isanIndx>nan)[0]
    if replacementCandidates.size != 0:
        replacement = Data[isanIndx[replacementCandidates[0]]]
    else:
        replacement = Data[isanIndx[np.where(isanIndx<nan)[0][-1]]]
    Data[nan] = replacement
print(Data)

edited Dec 23 '20 at 19:16

answered Dec 23 '20 at 18:37

Rajnish kumar

186
1
14

can you post the error please. I hope you have imported pandas or numpy whichever you are using – Rajnish kumar Dec 23 '20 at 18:51
for the pandas: AttributeError: 'DataFrame' object has no attribute 'as_matrix'. – Daniel Casasampera Dec 23 '20 at 18:55
for the numpy: IndexError: tuple index out of range. At the line idx = np.where(~mask,np.arange(mask.shape[1]),0) – Daniel Casasampera Dec 23 '20 at 18:56
I dont know if it matters but my array has a len of 1.000.000 – Daniel Casasampera Dec 23 '20 at 19:00
try out the updated script that i have posted – Rajnish kumar Dec 23 '20 at 19:16
What if there are multiple inf's in a row? Do you want the closest previous value that isn't inf? If the first element is inf, what should it be? – tdelaney Dec 23 '20 at 19:37
1

Yes, I want the closest previous thats not inf. The first element shouldnt be an inf. This solution works fine i think. But ive repplaced the isnan for isinf – Daniel Casasampera Dec 23 '20 at 20:19

score 1 · Accepted Answer · answered Dec 23 '20 at 19:11

1

why do you not use the simplest way?

for i in range (0,len(a)):
    if a[i]==inf: a[i]=a[i-1]

I have never work with inf. maybe you the type of it is str and so you should write a[i]=='inf'

answered Dec 23 '20 at 19:11

Reihaneh Kouhi

489
1
7
23

inf should be np.inf. Also is there a better way? My arrays are very big. – Daniel Casasampera Dec 23 '20 at 19:56

How to change inf values in numpy array for the previous non inf value?

2 Answers2