Keras model output a model with a specific size by using resize

Question

In a generator model of GANs, I am trying to generate image of a specific size. My target size is 28x280x3. I actually, so far I was creating a generator output of 28x28x3. Therefore, I am trying by using UpSampling2D to increase the size of the model. I am able to make the model output of size 28x224x3 after three UpSampling2D layers. However, my target is 28x280x3. How can I gap that divergence dimensions? I noticed that there is this approach that targets at resizing layers. How can it work in my case? My code is looking like the following:

def build_generator_face(latent_dim, channels, face_sequence):

  model = Sequential()
  model.add(Dense(128 * 7 * 7, activation="relu", input_shape=(None, latent_dim))) 
  model.add(Reshape((7, 7, 128)))
  model.add(UpSampling2D())
  model.add(Conv2D(128, kernel_size=4, padding="same"))
  model.add(BatchNormalization(momentum=0.8))
  model.add(Activation("relu"))
  model.add(UpSampling2D())
  model.add(Conv2D(64, kernel_size=4, padding="same"))
  model.add(BatchNormalization(momentum=0.8))
  model.add(Activation("relu"))

  if face_sequence == False:
    #model.add(UpSampling2D(size=(2, 2)))
    model.add(Conv2D(64, kernel_size=4, padding="same"))
    model.add(BatchNormalization(momentum=0.8))
    model.add(Activation("relu"))

    #model.add(UpSampling2D(size=(2, 2)))
    model.add(Conv2D(64, kernel_size=4, padding="same"))
    model.add(BatchNormalization(momentum=0.8))
    model.add(Activation("relu"))

  else:
    model.add(UpSampling2D(size=(1, 2)))
    model.add(Conv2D(64, kernel_size=4, padding="same"))
    model.add(BatchNormalization(momentum=0.8))
    model.add(Activation("relu"))

    model.add(UpSampling2D(size=(1, 2)))
    model.add(Conv2D(64, kernel_size=4, padding="same"))
    model.add(BatchNormalization(momentum=0.8))
    model.add(Activation("relu"))

    model.add(UpSampling2D(size=(1, 2)))
    model.add(Conv2D(64, kernel_size=4, padding="same"))
    model.add(BatchNormalization(momentum=0.8))
    model.add(Activation("relu"))

    pdb.set_trace()
    model.add(Reshape((-1,3), input_shape=(28,224,3)))
    model.add(Lambda(lambda x: x[:7840])) # throw away some, so that #data = 224^2
    model.add(Reshape(28,280,3)) # this line gives me an error but am not sure if it is necessary or not the code is found in here: https://stackoverflow.com/questions/41903928/add-a-resizing-layer-to-a-keras-sequential-model

  model.add(Conv2D(channels, kernel_size=4, padding="same"))
  model.add(Activation("tanh"))

  model.summary()
  noise = Input(shape=(latent_dim,))
  img = model(noise)

  mdl = Model(noise, output = img)

  return mdl

If face_sequence is False the model is generating an output of 28x28x3. I want when the boolean variable is True to generate an output of size 28x280x3. How this can be done?

Will this solve your problem? https://stackoverflow.com/questions/41903928/add-a-resizing-layer-to-a-keras-sequential-model — TYZ, Apr 29 '20 at 15:29
I tried it already, it did not work. I am not sure that reshape is also a good solution. Actually, the first row model.add(Reshape(-1,3), input_shape=(160,320,3)) was already problematic. — Jose Ramon, Apr 29 '20 at 15:31
Yes, so I am trying to figure out if that can work. Maybe this approach is too coarse, and I need to perform it in an array. Not sure. — Jose Ramon, Apr 29 '20 at 15:34
I updated using the solution provided by @TYZ and the received error. — Jose Ramon, Apr 29 '20 at 15:37
And you specifically want to use a single model to have 10 faces next to each other instead of having 10 predictions from your 28x28x3 network? — Lowry, Apr 29 '20 at 15:40
I want to generate a sequence of frames and not single frames. I am trying to figure output whether generated just a single image can work for me (or alternatively I can generate a matrix of images). — Jose Ramon, Apr 29 '20 at 15:42

Lowry · Accepted Answer · 2020-05-01T16:22:52.297

2

You were using only the first channel with 7840 and then trying to reshape into the wanted shape. For that you would need 23520 elements (28*280*3), but you only had 18816 (28*224*3).

This code resizes earlier in the process and uses one more UpSampling->Conv2D resulting in the wanted shape.

def build_generator_face(latent_dim, channels, face_sequence):

  model = Sequential()

  model.add(Dense(128 * 7 * 7, activation="relu", input_shape=(None, latent_dim))) 
  model.add(Reshape((7, 7, 128)))
  model.add(UpSampling2D())
  model.add(Conv2D(128, kernel_size=4, padding="same"))
  model.add(BatchNormalization(momentum=0.8))
  model.add(Activation("relu"))

  model.add(UpSampling2D())
  model.add(Conv2D(64, kernel_size=4, padding="same"))
  model.add(BatchNormalization(momentum=0.8))
  model.add(Activation("relu"))

  if face_sequence == False:
    model.add(Conv2D(64, kernel_size=4, padding="same"))
    model.add(BatchNormalization(momentum=0.8))
    model.add(Activation("relu"))

    model.add(Conv2D(64, kernel_size=4, padding="same"))
    model.add(BatchNormalization(momentum=0.8))
    model.add(Activation("relu"))

  else:
    model.add(UpSampling2D(size=(1, 2)))
    model.add(Conv2D(64, kernel_size=4, padding="same"))
    model.add(BatchNormalization(momentum=0.8))
    model.add(Activation("relu"))

    # go from 56 to 35 and continue upsampling

    model.add(Lambda(lambda x: x[:,:,:35,:]))



    model.add(UpSampling2D(size=(1, 2)))
    model.add(Conv2D(64, kernel_size=4, padding="same"))
    model.add(BatchNormalization(momentum=0.8))
    model.add(Activation("relu"))

    model.add(UpSampling2D(size=(1, 2)))
    model.add(Conv2D(64, kernel_size=4, padding="same"))
    model.add(BatchNormalization(momentum=0.8))
    model.add(Activation("relu"))

    model.add(UpSampling2D(size=(1, 2)))
    model.add(Conv2D(64, kernel_size=4, padding="same"))
    model.add(BatchNormalization(momentum=0.8))
    model.add(Activation("relu"))

    pdb.set_trace()


  model.add(Conv2D(channels, kernel_size=4, padding="same"))
  model.add(Activation("tanh"))

  model.summary()


  return model

edited May 01 '20 at 16:22

answered Apr 29 '20 at 16:14

Lowry

428
5
12

maybe this is better otherwise there is explode in the dimensionality – Jose Ramon Apr 29 '20 at 16:30
The first covolution in else should have 3 kernels I guess. By changing that I am receiving the following error: *** ValueError: total size of new array must be unchanged in this line: model.add(Reshape((28,35,3))) – Jose Ramon Apr 29 '20 at 16:36
With same padding the output shape is same as the input shape, so changing kernel to 3 would not change anything in the output size. This in Colab works perfectly with final shape of (None, 28, 280, 3): https://pastebin.com/06YpnwKG – Lowry Apr 29 '20 at 16:36
In debug mode, I can run until here: model.add(Lambda(lambda x: x[:2940])) but then while this line works it seems that reshape cannot proceed: total size of new array must be unchanged – Jose Ramon May 01 '20 at 15:55
and also I need to kernel size to the last convolution since it is 64 while the input_shape = (28,56,3). – Jose Ramon May 01 '20 at 16:02
Does that code from pastebin work for you? It does run in Google Colab for me. However what happens after model.summary() is not working. – Lowry May 01 '20 at 16:07
How exactly this line works: model.add(Reshape((-1,3), input_shape=(28,56,3)))? the previous layer has this size (None, 28, 56, 64), I got an mismatch error. – Jose Ramon May 01 '20 at 16:07
I got the point of the code but it does not work for me. And I am not sure about the first of the three lines model.add(Reshape((-1,3), input_shape=(28,56,3))), in order to work I need to change the previous convolution filter to 3 instead of 64, and then the last line: model.add(Reshape((28,35,3))) does not work. – Jose Ramon May 01 '20 at 16:09
That was a mistake on my part. I edited the code to now take (28,35,64) – Lowry May 01 '20 at 16:12
If i change it to 64 that make sense, i got an error as well: ValueError: total size of new array must be unchanged – Jose Ramon May 01 '20 at 16:18
But if i change the convolution number of filters to 3 instead of 64 it works :(, and then I got an error later in odel.add(Reshape((28,35,3))) . – Jose Ramon May 01 '20 at 16:19
How this line works exactly: model.add(Reshape((-1,3), input_shape=(28,56,64))), I guess I can do sth similar with Flatten but is too costly. – Jose Ramon May 01 '20 at 16:20
Yes this is true. But with Flatten the issue is the insane big size of parameters. Anyway Modify it in your code and I will accept the answer. – Jose Ramon May 01 '20 at 16:21
Ok, it was possible to do without Flatten, now only Lambda. Hope this works now! – Lowry May 01 '20 at 16:23
Ok I will try the Ladmba version otherwise with Flatten I need to do add I guess a Dense layer with 2940 and then to proceed with Reshape. – Jose Ramon May 01 '20 at 16:31
No need to add Dense layer. You can just reshape that flattened output vector into the shape you want. But this solution with only Lambda should work without the need to flatten and reshape. – Lowry May 01 '20 at 16:34

Keras model output a model with a specific size by using resize

1 Answers1