-1

I am implementing an autoencoder, used to rebuild color images. The loss function I want to use requires a reduced color set (max ~100 different colors) but I am struggling to find a suitable differentiable algorithm.

Another doubt I have is the following: is it better to apply such quantization directly in the loss function, or can I implement it in a custom non-trainable layer? In the second case, need the algorithm to be differentiable?

My first idea approaching this problem was to quantize the images before feeding them to the network, but I don`t know how to "force" the network to produce only the quantized colors as output.

Any suggestion is greatly appreciated, I do not need code, just some ideas or new perspectives. Being pretty new to Tensorflow I am probably missing something.

Simone Berasi
  • 19
  • 3

1 Answers1

0

If you want to compress the image, it seems you want to find discrete color set for image compression. In that case auto-encoder is not suitable approach for image compression.

The general auto-encoder compress tensor of images(B x C x H x W) to latent code of each images(B x D, typically D = 512). The beauty of this approach is that the optimal latent space is found 'automatically'.

Nevertheless if you want to utilize convex optimization tool of tensorflow, some continuous relaxation technique like interpolation could be helpul.

In the following paper, they utilize continuous relaxation for discrete path selection of neural network.

Liu, H., Simonyan, K., & Yang, Y. (2018). Darts: Differentiable architecture search. ICLR. enter image description here

In the following paper, they utilize interpolation to learn quantized kernel bank on look-up table.

Jo, Y., & Kim, S. J. (2021). Practical single-image super-resolution using look-up table. CVPR. enter image description here

Both of them provide codes.

kingsj0405
  • 149
  • 14