0

I have searched and found similar problems, but none that seem to be the same issue as what I'm facing. I'm trying to implement a neural network with Keras using Theano backend (both up to date), which involves a Lambda layer that takes the 1-dimensional output of a layer, and converts it into an n-dimensional vector with the 1-d output repeated n times.

The problem that I seem to be running into is that at the Lambda layer Keras seems to be expecting that the input has the same dimension as the output shape I'm specifying:

x=Input(shape=(2,))
V1=Dense(1)(x)
V2=Lambda(lambda B : B[0,0]*K.ones((3,)),output_shape=(3,))(V1)
model=Model(inputs=x,outputs=V2)
rms = RMSprop()
model.compile(loss='mse', optimizer=rms)
model.predict(np.array([1,2]).reshape((1,2)))

which gives this error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-7-40a7e91d5963> in <module>()
----> 1 model.predict(np.array([1,2]).reshape((1,2)))

/Users/user/anaconda/envs/py35/lib/python3.5/site-packages/keras/engine    /training.py in predict(self, x, batch_size, verbose)
   1504         f = self.predict_function
   1505         return self._predict_loop(f, ins,
-> 1506                                   batch_size=batch_size, verbose=verbose)
   1507 
   1508     def train_on_batch(self, x, y,

/Users/user/anaconda/envs/py35/lib/python3.5/site-packages/keras/engine/training.py in _predict_loop(self, f, ins, batch_size, verbose)
   1137 
   1138             for i, batch_out in enumerate(batch_outs):
-> 1139                 outs[i][batch_start:batch_end] = batch_out
   1140             if verbose == 1:
   1141                 progbar.update(batch_end)

ValueError: could not broadcast input array from shape (3) into shape (1)

I know there are other ways to try to do this (K.repeat_elements) but this has also given me error messages about broadcasting. Note, the problem persists even if I remove the B[0,0]* (so that the Lambda layer doesn't depend on B at all). If I change the (3,) in the K.ones and output_shape to (1,) then it seems to work.

From what I understand, the Lambda layers should be able to handle input/output pairs of differing dimension, is that correct?

hughes
  • 27
  • 3

1 Answers1

0

In output_shape, you don't consider the batch size. So this is correct: (3,)

But in tensors, the batch size is not ignored. You need at least two dimensions in the result of your expression: (Batch_size,3).

Also, don't use elements of the tensors, use the entire tensors. I haven't found a case where it would be important or useful to use separate elements (since you're supposed to do exactly the same operation to the entire batch)

I suggest you use K.repeat_elements(B, rep=3, axis=-1)

Daniel Möller
  • 84,878
  • 18
  • 192
  • 214
  • Very helpful, thanks. I guess the point of confusion for me was that in `output_shape` the batch size occupies the second spot in the tuple, while in defining the `K.ones` tensor in the lambda function the batch size goes in the first slot. Can you give me an idea why this is? With that in mind, both your suggestions work (after reshaping the `K.repeat_elements` tensor). Thanks! – hughes Jun 29 '17 at 19:18
  • No, the batch size always takes the first position. I don't know why there is a comma in `(3,)`, but it doesn't mean anything useful. You just must put that comma when you have only one dimension. If you have (None, 3), you define it as (3,). If you have (None, 3, 4), you define it as (3,4). -- It's probably just a notation issue, maybe it's necessary to have at least one comma to create a tuple? – Daniel Möller Jun 29 '17 at 19:53
  • When the answer is ok to you, consider marking it as answered :) -- That helps other users when searching for an answer. – Daniel Möller Jun 29 '17 at 19:53