Can I create an image classification algorithm model with online image links instead of downloaded images

Question

Is it possible to create an image classification algorithm model if I dont have the training data downloaded. However, I have all the image links from which I can extract the images. I want to avoid downloading all the images since there is lots of data. Using image links will be easier to maintain. I am currently planning to use Resnet. However, I tried to search online and couldn't find a way to use online image links for training dataset. Help here would be appreciated.

Answer 1

For training your model you need the actual images. You have two choices:

1. Download the images once and store them locally. This does not use much network capacity, but you need storage for the images. Processing will be much faster, as accessing a local disk is orders of magnitude faster than reading a file over a network.

2. Download each file every time you need it. This takes up a lot less space, but is vastly slower, as reading the images takes a long time. It also puts a lot of strain on the network, and especially the server the images are stored on, as you will need to download them repeatedly.

It is clearly a very bad idea to download the images every time, both from the time required, and the network usage, let alone the effect it has on the servers where the images are stored.

So, there is no real choice: if you cannot store the images, you cannot do this.

Answer 2

The data needs to be where the processing takes place, so if you are trying to train on a local machine then it can't happen.

You can use Google Colab to train online and upload the files onto your Google Drive using this.