1

Probably lots of people already saw this article by Google research:

http://googleresearch.blogspot.ru/2015/06/inceptionism-going-deeper-into-neural.html

It describes how Google team have made neural networks to actually draw pictures, like an artificial artist :)

I wanted to do something similar just to see how it works and maybe use it in future to better understand what makes my network to fail. The question is - how to achieve it with nolearn\lasagne (or maybe pybrain - it will also work but I prefer nolearn).

To be more specific, guys from Google have trained an ANN with some architecture to classify images (for example, to classify which fish is on a photo). Fine, suppose I have an ANN constructed in nolearn with some architecture and I have trained to some degree. But... What to do next? I don't get it from their article. It doesn't seem that they just visualize the weights of some specific layers. It seems to me (maybe I am wrong) like they do one of 2 things:

1) Feed some existing image or purely a random noise to the trained network and visualize the activation of one of the neuron layers. But - looks like it is not fully true, since if they used convolution neural network the dimensionality of the layers might be lower then the dimensionality of original image

2) Or they feed random noise to the trained ANN, get its intermediate output from one of the middlelayers and feed it back into the network - to get some kind of a loop and inspect what neural networks layers think might be out there in the random noise. But again, I might be wrong due to the same dimensionality issue as in #1

So... Any thoughts on that? How we could do the similar stuff as Google did in original article using nolearn or pybrain?

Maksim Khaitovich
  • 4,742
  • 7
  • 39
  • 70

2 Answers2

3

From their ipython notebook on github:

Making the "dream" images is very simple. Essentially it is just a gradient ascent process that tries to maximize the L2 norm of activations of a particular DNN layer. Here are a few simple tricks that we found useful for getting good images:

  • offset image by a random jitter
  • normalize the magnitude of gradient
  • ascent steps apply ascent across multiple scales (octaves)
Thomas Jungblut
  • 20,854
  • 6
  • 68
  • 91
1

It is done using a convolutional neural network, which you are correct that the dimensions of the activations will be smaller than the original image, but this isn't a problem.

You change the image with iterations of forward/backward propagation just how you would normally train a network. On the forward pass, you only need to go until you reach the particular layer you want to work with. Then on the backward pass, you are propagating back to the inputs of the network instead of the weights.

So instead of finding the gradients to the weights with respect to a loss function, you are finding gradients to inputs with respect to the l2 Normalization of a certain set of activations.

Frobot
  • 1,224
  • 3
  • 16
  • 33