Data augmentation in python throws an error "int() argument must be a string, a bytes-like object or a number, not 'dict'"

Question

I am trying to load all the images present in a folder, augment each of them and save it in a different repository

I am able to augment an by hard coding the path and image name however, when I am trying to store the path of all the images and then processing the loop, it is not working and throwing an error mentioned in the title (also : int() argument must be a string, a bytes-like object or a number, not 'dict' ). In the later part after augmenting the image, I am storing the outputs in a different folder. The code is also as follows:

It would be really helpful if anyone can provide a solution to this problem.

    import imgaug as ia
    from imgaug import augmenters as iaa
    import numpy as np
    import pandas as pd # data processing, CSV file I/O (e.g. d.read_csv)
    import os
    from glob import glob
    import imageio

    ia.seed(20)
    #os.chdir('C:/Users/Madhav/Desktop/RIC/data/images_001/images')

    img = {os.path.basename(x): x for x in glob(os.path.join('C:/Users/Madhav/Desktop/FinalSem/Data_Images/','images*','*', '*.png'))}

    #img = imageio.imread("00000001_000.png") #read you image
    images = np.array(
    [img for _ in range(32)], dtype=np.uint8)  # 32 means creat 32 enhanced images using following methods.

    seq = iaa.Sequential(
        [
            iaa.Fliplr(0.5),  
            iaa.Crop(percent=(0, 0.1)),            
            iaa.Sometimes(0.5, iaa.GaussianBlur(sigma=(0, 0.5))),        
            iaa.ContrastNormalization((0.75, 1.5)),         
            iaa.AdditiveGaussianNoise(loc=0, scale=(0.0, 0.05 * 255), per_channel=0.5),    
            iaa.Multiply((0.8, 1.2), per_channel=0.2),
            iaa.Affine(
                scale={
                    "x": (0.8, 1.2),
                    "y": (0.8, 1.2)
                },
                translate_percent={
                    "x": (-0.2, 0.2),
                    "y": (-0.2, 0.2)
                },
                rotate=(-25, 25),
                shear=(-8, 8))
        ],
        random_order=True)  # apply augmenters in random order
    images_aug = seq.augment_images(images)
    for i in range(32):
          imageio.imwrite(str(i)+'C:/Users/Madhav/Desktop/FinalSem/Augmented_Generated/Aug.png', images_aug[i])
#write all changed images

    TypeError
Traceback (most recent call last)
    <ipython-input-11-9a5de30a5337> in <module>
         18 #img = imageio.imread("00000001_000.png") #read you image
         19 images = np.array(
    ---> 20     [img for _ in range(32)], dtype=np.uint8)  # 32 means creat 32         enhanced images using following methods.
         21 
         22 seq = iaa.Sequential(

    TypeError: int() argument must be a string, a bytes-like object or a number, not 'dict'

The constructor `np.array()` with a `dtype` of `np.uint8` needs a list of 8-bit integers as input, but you passed it a list of dictionaries instead. Why did you do that? — jjramsey, Aug 02 '19 at 15:21
I was trying if passing the path of the images can help here. Is there any alternative? or a way to solve this problem? — madhav lavania, Aug 02 '19 at 15:33
@bastiengirschig Judging from the [documentation for `augment_images`](https://imgaug.readthedocs.io/en/latest/source/api_augmenters_meta.html?highlight=augment_images#imgaug.augmenters.meta.Augmenter.augment_images), `images` can just be a list of numpy arrays. — jjramsey, Aug 02 '19 at 16:29
Yes, that's what I expected. I've still provided a solution that creates a single numpy array. This could be helpful in some cases (cropping, masking, arithmetic, etc... on all images) — bastien girschig, Aug 02 '19 at 16:32

bastien girschig · Accepted Answer · 2019-08-02T19:46:32.230

2

I'm not familiar with imgaug, but I think this should work:

from os import path
from glob import glob
from scipy.ndimage import imread
import numpy as np

# Here i'm actually opening each image, and putting its pixel data in
# numpy arrays
images = [ imread(imgpath) for imgpath in glob(path.join('images', '*.png')) ]
# 'images' doesn't need to be a numpy array, it can be a regular
# python list (array). In fact, it can't be if the images are of
# different sizes

From then on, you can continue with your original code.

Note that if you have a lot of images, you might run into memory problems. In that case, you'll need to break up your list into smaller batches (kind of like what you did with your 'range(32)'). Add a comment if you need help with that.

edited Aug 02 '19 at 19:46

answered Aug 02 '19 at 16:20

bastien girschig

663
6
26

Thankyou very much for the help. This resolved the issue. I passed them in the bunch and it worked fine. Thanks again – madhav lavania Aug 02 '19 at 17:28
Hi Bastien, it is working in one but not in another, it could be because of some library issue. I have installed Scipy and other libraries that could be necessary. But it still says: " ImportError: cannot import name 'imread' " – madhav lavania Aug 02 '19 at 18:02
I think the one you had commented should work in the same way: `imageio.imread` – bastien girschig Aug 02 '19 at 18:06
Weird. How are you installing your libraries ? are you using pip directly ? virtualenv ? conda ? – bastien girschig Aug 02 '19 at 18:19
I am using conda – madhav lavania Aug 02 '19 at 18:21
By the way, what do you mean by "in one but not in another". another folder ? – bastien girschig Aug 02 '19 at 18:21
I'm not super familiar with conda. If I tell you to install conda the `pillow` package into "the other one", can you do it ? – bastien girschig Aug 02 '19 at 18:24
No, it went into a loop without throwing an error in one laptop but then after 30 mins it didn't work. I thought it worked. Also, when I tried in another laptop by installing same libraries, it gave me the same error instantly. – madhav lavania Aug 02 '19 at 18:25
Ya installed and imported pillow and opencv. But nothing is helping here :( – madhav lavania Aug 02 '19 at 18:27
You got an ImportError after 30min ? That's even weirder. Can you verify that a script with only this: `from PIL import Image` does not crash ? – bastien girschig Aug 02 '19 at 18:29
Now it shows :( "AssertionError: Expected 3d/4d array of form (N, height, width) or (N, height, width, channels), got shape (99,)." Is this because of different size of images? – madhav lavania Aug 02 '19 at 18:34
Shared below as an answer. – madhav lavania Aug 02 '19 at 18:52

score 0 · Answer 2 · answered Aug 02 '19 at 18:49

0

Below is the code, sorry if it is a very silly mistake from my end itself

from PIL import Image
import imgaug as ia
from imgaug import augmenters as iaa

# Here i'm actually opening each image, and putting its pixel data in
# numpy arrays
images = [imageio.imread(imgpath) for imgpath in     glob(path.join('C:/Users/Madhav/Desktop/Final Sem/Data_Images/', '*.png')) ]
# This converts the list from before into a numpy array. If all images have the
# same size, 'images' will be a 'true' numpy array. Otherwise, it's going to be
# a numpy 'collection' (I don't know the real name)
images = np.array(images)

#print (images.shape)


seq = iaa.Sequential(
    [
        iaa.Fliplr(0.5),  
        iaa.Crop(percent=(0, 0.1)),            
        iaa.Sometimes(0.5, iaa.GaussianBlur(sigma=(0, 0.5))),        
        iaa.ContrastNormalization((0.75, 1.5)),         
        iaa.AdditiveGaussianNoise(
            loc=0, scale=(0.0, 0.05 * 255), per_channel=0.5),    
        iaa.Multiply((0.8, 1.2), per_channel=0.2),
        iaa.Affine(
            scale={
                "x": (0.8, 1.2),
                "y": (0.8, 1.2)
            },
            translate_percent={
                "x": (-0.2, 0.2),
                "y": (-0.2, 0.2)
            },
            rotate=(-25, 25),
            shear=(-8, 8))
    ],
random_order=True)  # apply augmenters in random order

images_aug = seq.augment_images(images)
for i in range(32):
imageio.imwrite(str(i)+'C:/Users/Madhav/Desktop/Final     Sem/Augmented_Generated/*.png', images_aug[i])

answered Aug 02 '19 at 18:49

madhav lavania

13
5

Can you add this before the 'iaa.Sequential' line ?: `print images[0].shape` – bastien girschig Aug 02 '19 at 18:52
I've added a full working version to my answer. Can you check it out ? – bastien girschig Aug 02 '19 at 19:05
Still not working, it is actually throwing an error on line "images_aug = seq.augment_images(images) " where i guess it is not able to augment it properly after converting it into an array. The error again says "AssertionError: Expected 3d/4d array of form (N, height, width) or (N, height, width, channels), got shape (99,). " – madhav lavania Aug 02 '19 at 19:15
Could you share a few of the images ? And also tell me the shape of the 'images' array ? – bastien girschig Aug 02 '19 at 19:20
aha! So your images are not all of the same size, right ? – bastien girschig Aug 02 '19 at 19:32
try removing or commenting the `images = np.array(images)` line – bastien girschig Aug 02 '19 at 19:33
yes.... they are of different size. This could be a reason why I am getting these errors. – madhav lavania Aug 02 '19 at 19:35
Finally............. Thank you very much and sorry I took so much of your time. This resolved the issue, I am able to get the augmented images in the set folder. – madhav lavania Aug 02 '19 at 19:39
Great! I'm curious: Are you working on some sort of machine learning model for medical imagery ? – bastien girschig Aug 02 '19 at 19:44
Hi Bastien, Yes I am, in classifying pulmonary diseases in a multi-class and multi-stream fashion. – madhav lavania Aug 02 '19 at 20:38

Data augmentation in python throws an error "int() argument must be a string, a bytes-like object or a number, not 'dict'"

2 Answers2

Linked