Increasing dataset size using imgaug

Question

I am merging two different datasets containing images into one dataset. One of the datasets contains 600 images in the training set. The other dataset contains only 90-100 images. I want to increase the size of the latter dataset by using the imgaug library. The images are stored in folders under the name of their class. So the path for a "cake" image in the training set would be ..//images//Cake//cake_0001. I'm trying to use this code to augment the images in this dataset:

path = 'C:\\Users\\User\\Documents\\Dataset\\freiburg_groceries_dataset\\images'

ia.seed(6)

seq = iaa.Sequential([
    iaa.Fliplr(0.5),
    iaa.Crop(percent=(0, 0.1)),
    iaa.Affine(rotate=(-25,25))
], random_order=True)

for folder in os.listdir(path):
    try:
        for i in os.listdir(folder):
            img = imageio.imread(i)
            img_aug = seq(images=img)
            iaa.imshow(img_aug)
            print(img_aug)
    except:
        pass

Right now there's not output, even if I put print(img) or imshow(img) or anything. How do I ensure that I got more images for this dataset? Also, what is the best spot to augment images? Where do the augmented images get stored, and how do I see how many new images were generated?

_Right now there’s not output..._ Is the program printing nothing, or is it never printing? — AMC, Jan 09 '20 at 04:28
It never prints within the `for i in os.listdir(folder):` loop. Even if I just put `print('something')`, it doesn't print. I used the same syntax to rename all the files in another part of my code and it reformatted them all correctly, so I'm not sure why it's failing here — Sanglang, Jan 09 '20 at 04:39
What is that try-except for? Have you tried checking whether or not it never reaches the printing because it’s throwing an exception? In fact, are you even sure that the for loop is iterating correctly? — AMC, Jan 09 '20 at 04:43
Since the images are stored in the folder of their class, the try-except makes sure that it doesn't throw an error when it encounters a new folder instead of an image. When I use a for loop instead of a try loop, I get an error from the folder. When I use the same try-except loop in another part of my code, it iterates through it correctly. I'm not sure why it wouldn't work here? — Sanglang, Jan 09 '20 at 05:00
@SharathKumarGajavelli Yes it looks correct -- as a test run, I am using a small set to try to augment the images. The `images` dir in my `path` variable has a few folders containing images. — Sanglang, Jan 09 '20 at 05:05

Sharath Kumar Gajavelli · Answer 1 · 2020-01-09T05:11:32.577

1

It's because folder is not the path to the directory you are looking for. You should change for i in os.listdir(folder): to for i in os.listdir(path+'\\'+folder):. Then it looks inside the path\folder directory for files.

edited Jan 09 '20 at 05:11

answered Jan 09 '20 at 05:06

Sharath Kumar Gajavelli

140
7

That may be part of the problem, I changed it to that syntax. Now it will print output only if I put a print statement immediately after the `for` loop. Also, if replace `pass` with a `print('something')` statement after `except:` it will also print that. So it seems to be passing through the try and except loop, and raising an exception every time. – Sanglang Jan 09 '20 at 05:13
@Sanglang What’s the exception? I forgot to mention this earlier, but bare except statements are bad practice. – AMC Jan 09 '20 at 05:51
@AMC the exception for when the loop encounters a folder instead of an image. I'm trying to get it to iterate through 25 folders, each folder containing approx 100 images. I don't want it to throw an error when it iterates over a folder instead of an image, if that makes sense. I have heard that bare except statements are bad practice, but I can't find a better way to get my code to iterate through multiple folders. – Sanglang Jan 09 '20 at 21:02
@Sanglang Can you describe the file structure here? If you're iterating over folders, and then through the contents of each folder, you can't "accidentally" treat a folder as an image. – AMC Jan 09 '20 at 21:24
@AMC I guess the error was coming from the `for i in os.listdir(folder):` for loop, as per @Sharath's rec, changing the path to `for i in os.listdir(path + '\\' + folder):` makes it iterate over all the folders correctly. I could eliminate the try/except loop and all the images are being augmented now :) – Sanglang Jan 09 '20 at 22:31

score 1 · Accepted Answer · answered Jan 09 '20 at 10:56

The Question was not clear. So, for the issue2: error in saving file and not able to visualize using imshow().

First: In the second loop code block

img = imageio.imread(i)
img_aug = seq(images=img)
iaa.imshow(img_aug)
print(img_aug)

1st error is: i is not the file path. To solve this replace imageio.imread(i) with imageio.imread(path+'/'+folder+'/'+i).

2nd error is: iaa doesn't have the property imshow(). To fix this replace iaa.imshow(img_aug) with iaa.imgaug.imshow(img_aug). This fixes the error of visualizing and finishing the loop execution.

Second: If you have any issue in saving images, then use PIL. i.e.,

from PIL import Image
im = Image.fromarray(img_aug)
im.save('img_aug.png')`

Yep, changing the path in the `for i in os.listdir(folder)` path definitely was the issue, thanks! And thank you for the additional tips! — Sanglang, Jan 09 '20 at 22:32

Increasing dataset size using imgaug

2 Answers2