-2

The images are like this: fake image generated with no label real image with label

I have 12000 fake images, that I generated based on bright spots on an image with no label. I have 1200 real images, that have annotations and true labels. I want the output to be labeled generated images. I want to know how should I proceed.

I want my Variational Autoencoder to generate real images or close to real with labels. I thought of two options: The first option is to use the fake images and train the 12000 images and test with the 1200 real images, since as you can see from the example, some of them match. The second option is to downsample the 12000 and train semisupervisely, with unlabeled and labeled samples.

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Savoyevatel
  • 47
  • 4

1 Answers1

0

Autoencoders generally do not form labels but rather attempt to recreate what you train on. You would need a grouping mechanism to create 'labels' for your data. To do this, simply perform the following:

  1. Train autoencoder on all/labeled data (perform your splits as normal). Really depends on what you are trying to obtain. I think for your case you really want to use labeled data. And 'generate' new images from your fake images that are close to your real images. As such use labeled data only for training.

  2. Take the encoder output for all your labeled data and train a grouping algorithm such as kmeans or another network (perform your train/validation split here as well).

2b) You could also run all your data through the encoding and do kmeans here. Maybe there is an additional group?

  1. Label your unlabeled data but passing it through encoder-> grouping
Jason Chia
  • 1,144
  • 1
  • 5
  • 18
  • when you said that I need to take the encoder output, that means the values of mu, z, logvar?. What are the values I should be looking at to train a grouping algorithm? – Savoyevatel Dec 10 '22 at 20:09
  • 1
    I am unclear to what you are referring to. Your encoder section should output some number of features based on the model. I.e. 3x3 filters -> N groups of 3x3 features. These features should correspond to your labels. Which you can then train on some grouping algorithm (kemans/SVM/NN) to finally label your unlabelled data. (Labelled data-> train autoencoder) -> (Encoder output of labelled data-> train grouping algorithm) -> (unlabeled data -> encoder-> grouping algo -> 'labeled data') – Jason Chia Dec 12 '22 at 08:15
  • I have tried on my own, and I think I am here so far. I trained my encoder on labeled data ----> then I got my model "model.layers[5].get_weights()[0].shape # 3 * 3 * 128" ------>then I got the features "features = model.predict(x)". However, so when I train the clustering algorithm this will be transfer learning for the unlabeled part? or do I just have to fit the clustering into whatever data I have as unlabeled? – Savoyevatel Dec 20 '22 at 13:26
  • It depends. Cluster A: Trained on labeled data-> cluster your unlabeled data to closest fit. Cluster B: Train on all data -> cluster based on features only which may lead to additional clusters. Depending on your requirements you might want to explore both methods. – Jason Chia Dec 21 '22 at 07:35
  • I have clustered successfully my unlabeled data and I was wondering how could I use these clusters as a classification task since I have labeled data (1 to 5). I would like to also know, should I use my generated images from the VAE? – Savoyevatel Jan 17 '23 at 17:46
  • Again perhaps you are misunderstanding my explanation. 1) image -> VAE -> image, 2) VAE(features) -> clustering -> cluster (1-5) if you trained on your labeled data 3) unlabaled -> VAE -> VAE(features) -> clustering -> cluster(1-5) Classification task is done. Generated images do not help your classification problem. But it can help identify anomalies. img1->VAE->img2, img2-img1 = anomaly. In this way you can also possibly extend it to 5 VAE models for class 1-5 and if the anomaly score is too big its not part of that class. – Jason Chia Jan 18 '23 at 09:24
  • thanks for your help, its more clear now, do you mind sharing a book or something that can help me understand these terms more? I am self taught, and I am struggling to understand all of this, specially the latent space. So for a classification task, images lets say: [20000,64,64,3] image shape (20000 being the number of images) are trained in a neural network and then a prediction is made. However, with the latent space [20000, 22] being 20000 the number of data points and 22 the data point shape. Once classified, how am I supposed to know how this latent point looked like? – Savoyevatel Jan 19 '23 at 00:46
  • You can use something like T-sne on the 22 data points to visualize the latent space of your 20000 images. Sorry no books to recommend. Try google search and maybe check out your local library – Jason Chia Jan 23 '23 at 10:37