Python advanced Numpy slicing and indexing to vectorize method

Question

I am trying to vectorize this method that I am using for image augmentation in ML:

def random_erase_from_image(images, random_erasing, image_size):
#could probably be vectorized to speed up
to_return = images
for t in range(images.shape[0]):
    if np.random.randint(0, 2) == 0:#do random erasing
        x_erase_size = np.random.randint(0, random_erasing)
        y_erase_size = np.random.randint(0, random_erasing)

        x_erase_start = np.random.randint(0, image_size-x_erase_size)
        y_erase_start = np.random.randint(0, image_size-y_erase_size)

        shape = to_return[t, y_erase_start:y_erase_start+y_erase_size, x_erase_start:x_erase_start+x_erase_size, :].shape

        print(shape)

        to_return[t, y_erase_start:y_erase_start+y_erase_size, x_erase_start:x_erase_start+x_erase_size, :] = (np.random.random(shape) * 255).astype('uint8')

return images

This is as far as I have gotten, but don't know how to slice properly.

def random_erase_vec(images, random_erasing, image_size):
    #could probably be vectorized to speed up
    to_return = images
    mask = np.random.choice(a=[False, True], size=images.shape[0], p=[.5, .5])  
    x_erase_size = np.random.randint(0, random_erasing, size=images.shape[0])
    y_erase_size = np.random.randint(0, random_erasing, size=images.shape[0])

    x_erase_start = np.random.randint(0, image_size-x_erase_size, size=images.shape[0])
    y_erase_start = np.random.randint(0, image_size-y_erase_size, size=images.shape[0])

    random_values = (np.random.random((images.shape))* 255).astype('uint8')

    to_return[:, [y_erase_start[:]]:[y_erase_start[:]+y_erase_size[:]], [x_erase_start[:]]:[x_erase_start[:]+x_erase_size[:]], :] = random_values[:, [y_erase_start[:]]:[y_erase_start[:]+y_erase_size[:]], [x_erase_start[:]]:[x_erase_start[:]+x_erase_size[:]], :]

    return images

I am trying to avoid reshaping, but if that is what is needed, I guess it will do. Let me know any ways you can think of to speed up the original method.

I am getting this error on the slicing line: "slice indices must be integers or None or have an index method"

I also want to mask so not all images are randomly erased, but I want to do that after I get the slicing part completed.

Thank you for your help.

Edit: Example inputs:

images: numpy array with dimensions [# of images, height (32), width (32), channels (3)

random_erasing: poorly names, but the max size of the image in either dimension to be erased. Currently set to 20

image_size: Could have gotten from the images array now that I think about it, but cleaning up hasn't been a priority yet

Could you provide examples for the inputs? Or at least their dimensions? — scleronomic, Jul 01 '20 at 13:24
If you feel like reshaping can help you slice, here's what you can do: Make a new variable that's reshaped, i.e. `image2=images.reshape(...)`. When `numpy` uses reshape, it actually doesn't rewrite a new array, and it still points to the same place in memory. So altering individual elements of `image2` through slicing will still affect the original `images` array. This way, you can reshape into a somewhat easier-to-slice shape while still effectively changing your original array. — M Z, Jul 01 '20 at 13:46
On further though, reshaping is likely needed, since the random_erasing size changes, so the output size would be different for different slices — user7321498, Jul 01 '20 at 14:09

scleronomic · Answer 1 · 2020-07-03T12:15:00.450

I cleaned up your function a little and tried to vectorize it partly, but as you want changing sizes for the random patches it is a bit complicated.

import numpy as np

def random_erase(images, random_erasing):
    n, *image_size, n_channels = images.shape
    to_return = images.copy()
    
    for t in range(n):
        x_erase_size = np.random.randint(1, random_erasing)
        y_erase_size = np.random.randint(1, random_erasing)

        x_erase_start = np.random.randint(1, image_size[0]-x_erase_size)
        y_erase_start = np.random.randint(1, image_size[1]-y_erase_size)

        x_erase_end = x_erase_start + x_erase_size
        y_erase_end = y_erase_start + y_erase_size
        
        shape = (x_erase_size, y_erase_size, n_channels)
        random_image = np.random.randint(0, 255, size=shape, dtype=np.uint8)
        to_return[t, x_erase_start:x_erase_end, y_erase_start:y_erase_end, :] = random_image
        
    return to_return


def random_erase_vec(images, random_erasing):
    n, *image_size, n_channels = images.shape

    to_return = images.copy()
    x_erase_size = np.random.randint(1, random_erasing, size=n)
    y_erase_size = np.random.randint(1, random_erasing, size=n)

    x_erase_start = np.random.randint(1, image_size[0]-x_erase_size, size=n)
    y_erase_start = np.random.randint(1, image_size[1]-y_erase_size, size=n)

    x_erase_end = x_erase_start + x_erase_size
    y_erase_end = y_erase_start + y_erase_size

    shapes = np.vstack((x_erase_size, y_erase_size))
    sizes = np.prod(shapes, axis=0)
    sizes_cs = np.cumsum(np.concatenate([[0], sizes]))
    total_size = np.sum(sizes)

    idx = np.empty((total_size, 3), dtype=int)
    for i in range(n):
        idx_x, idx_y = np.meshgrid(np.arange(x_erase_start[i], x_erase_end[i]), 
                                   np.arange(y_erase_start[i], y_erase_end[i]))
        idx[sizes_cs[i]:sizes_cs[i+1], 0] = i
        idx[sizes_cs[i]:sizes_cs[i+1], 1] = idx_x.flatten()
        idx[sizes_cs[i]:sizes_cs[i+1], 2] = idx_y.flatten()

    random_values = np.random.randint(0, 255, size=(total_size, n_channels), dtype=np.uint8)
    to_return[idx[:, 0], idx[:, 1], idx[:, 2], :] = random_values

    return to_return

# images = np.random.random((1000, 100, 100, 1))
# random_erasing = 32
a = random_erase(images, random_erasing)
b = random_erase_vec(images, random_erasing)
# a: 0.059 s
# b: 0.049 s

The speed up is not amazing (roughly 20%) and as this is for preprocessing in ML probably your best bet is to use more workers to prepare the data, so that you can fully utilize your GPU.

EDIT: Yes, I use .copy() to make sure that the argument is not mutated outside the routine. So you can ignore this out if you like.

I use the term workers as in the [tensorflow documentation]:(https://www.tensorflow.org/api_docs/python/tf/keras/Model)

workers Used for generator or keras.utils.Sequence input only. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1. If 0, will execute the generator on the main thread.

use_multiprocessing Used for generator or keras.utils.Sequence input only. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can't be passed easily to children processes.

Thank you for looking at this. This is called inside a generator is why I am concerned about the speed of it. Is the copy needed or is this just so the argument is not mutated inside the routine? Also, I don't understand the "more workers" comment — user7321498, Jul 01 '20 at 14:51

Python advanced Numpy slicing and indexing to vectorize method

1 Answers1