Changing individual elements of numpy array based on its value

Question

I'm trying to iterate over every element of an numpy array and change its value in a semi-random manner. The reason for that is, that I'll apply that method to different arrays: I want them to be changed but I don't want them to be changed the same way.

Here is what I tried so far:

with np.nditer(smatrix, op_flags=['readwrite']) as it:
    for element in it:
        if element < 0:
            element = element - uniform(0.1,0.2)
        if 0 <= element < 0.05:
            element = uniform(0.15,0.3)
        elif 0.05 <= element < 1:
            element = 0
        elif 1 == element:
            element = 1

Another possibility:

for element in np.nditer(smatrix, op_flags=['readwrite']):
    if element < 0:
        element = element - uniform(0.1,0.2)
    if 0 <= element < 0.05:
        element = uniform(0.15,0.3)
    elif 0.05 <= element < 1:
        element = 0
    elif 1 == element:
        element = 1

However, the resulting array looks exactly the same as the inital array...

I'm relatively new to programming and stuck regarding this matter for quite a while. It would be awesome if someone could give me a hint on how to solve it and ideally a short explanation. Thank you!!

Daweo · Answer 1 · 2019-02-27T13:46:04.010

If you want to apply transformation to every cell of numpy.array then numpy.vectorize is right tool for that. Firstly you need function, which will use content of single cell as input, which itself will be feed to vectorize and which return desired value, see code:

import numpy as np
import random
def f(x):
    if x < 0:
        return x-random.uniform(0.1,0.2)
    if 0 <= x < 0.05:
        return random.uniform(0.15,0.3)
    if 0.05 <= x < 1:
        return 0.0
    if x == 1:
        return 1.0
    return x
vf = np.vectorize(f)

data = np.array([-4.3,0.03,0.9,1.0,1.9])
altered_data = vf(data)

Output (might be bit different for you as it is partially random):

[-4.48922564  0.19609667  0.          1.          1.9       ]

As you can check, it worked as intended for all values. Note that I added "do nothing to element" behavior (return x) as safety measure, in case element will be greater than 1.

For more info regarding numpy.vectorize read documentation.

EDIT: Fixed issue reported in comment, problem was that f sometimes returned int and sometimes float, while it should always return float.

I tried your solution with different 2D arrays, and discovered an issue if the first value = 1 Examples: data = np.array([[0.02,0,0.1,1],[1,0.05,0,0.01]]) [[0.17426108 0.25596603 0. 1. ] [1. 0. 0.29732222 0.23448857]] data = np.array([[1,0,0.1,1],[1,0.05,0,0.01]]) [[1 0 0 1] [1 0 0 0]] — Julia, Feb 27 '19 at 13:25
`vectorize` performs a trial calculation to determine the return dtype. To skip that define a `otypes` parameter. `vectorize` can be handy if you must use a scalar function on a multidimensional array, but it is slow - slower than a more explicit Python iteration. — hpaulj, Feb 27 '19 at 18:18

hpaulj · Answer 2 · 2019-02-27T18:16:47.447

I think nditer needs a stronger disclaimer. It isn't a good iteration tool for beginners. It is tricky to use right, and isn't particularly fast. This Python version is really a stepping stone toward using it in compiled code, as illustrated with the cython example at the end of the main nditer page:

https://docs.scipy.org/doc/numpy/reference/arrays.nditer.html

There are faster ways of doing this kind of array calculation, but I'll try to explain why your nditer doesn't work.

The problem is with basic Python iteration. In loop like:

alist = [0,0,0]
for element in alist:
    element = 1

alist is not modified. The element=1 just reassigns the element variable, but does not modify the iteration variable itself. Instead you have mutate element itself, if that's possible.

With nditer the iteration variable is an ndarray itself, and is mutable:

In [69]: smatrix = np.zeros((2,2),int)                                          
In [70]: with np.nditer(smatrix, op_flags=['readwrite']) as it: 
    ...:     for element in it: 
    ...:         print(element, type(element)) 
    ...:         element[...] = 1 
    ...:                                                                        
0 <class 'numpy.ndarray'>
0 <class 'numpy.ndarray'>
0 <class 'numpy.ndarray'>
0 <class 'numpy.ndarray'>
In [71]: smatrix                                                                
Out[71]: 
array([[1, 1],
       [1, 1]])

If this isn't clear, look more carefully at the examples in https://docs.scipy.org/doc/numpy/reference/generated/numpy.nditer.html. Notice the use the out parameter, and expressions like x[:] = [-1, -2, -3].

If this talk of mutating variables is confusing, it's a good indication that nditer is too advanced a tool at this point.

Changing individual elements of numpy array based on its value

2 Answers2