14

My network has two time-series inputs. One of the input has a fixed vector repeating for every time step. Is there an elegant way to load this fixed vector into the model just once and use it for computation?

Hooked
  • 84,485
  • 43
  • 192
  • 261
Yakku
  • 453
  • 1
  • 7
  • 14
  • 1
    If it is always the same, why do you need to input it at all? Or do you mean it is fix for a given sequence but will be different for different sequences? – sietschie May 12 '17 at 08:57
  • @sietschie Yes, you got it right. It will be different for different sequences. So instead of creating a repeated vector, I was wondering if there is a smart way of initializing the network with this input. – Yakku May 12 '17 at 09:22
  • Are you thinking about something like this [RepeatVector](https://keras.io/layers/core/#repeatvector)? – sietschie May 12 '17 at 09:25
  • But if I am using RepeatVector I still need to give the vector every epoch. I was wondering if there is some method which can be done while initializing the network itself. – Yakku May 12 '17 at 09:29
  • Related: [Creating constant value in Keras](https://stackoverflow.com/q/46465813/3744182). – dbc Apr 27 '19 at 23:19
  • 1
    @dbc Even though this post is older an has higher ranked answers (which do not work), the actual solution is in the linked thread. So even though the other thread is the duplicate, it contains the solution. I don't know how Stackoverflow handles such cases. – azureai Apr 28 '19 at 00:31

3 Answers3

9

You can create a static input using the tensor argument as described by jdehesa, however the tensor should be a Keras (not tensorflow) variable. You can create this as follows:

from keras.layers import Input
from keras import backend as K

constants = [1,2,3]
k_constants = K.variable(constants)
fixed_input = Input(tensor=k_constants)
Alexander Measure
  • 894
  • 1
  • 14
  • 18
  • 4
    How would I make and fit a model using these constants? Do I also have to provide the constants at fit time? – Zach Sep 19 '18 at 19:37
  • For an actual solution see https://stackoverflow.com/questions/46465813/creating-constant-value-in-keras – azureai Apr 27 '19 at 23:14
  • @azureai it works for me. It's actually described in the documentation as well – Edu Aug 20 '19 at 10:01
2

EDIT: Apparently the answer below does not work (nowadays anyway). See Creating constant value in Keras for a related answer.

Looking at the source (I haven't been able to find a reference in the docs), it looks like you can just use Input and pass it a constant Theano/TensorFlow tensor.

from keras.layers import Input
import tensorflow as tf

fixed_input = Input(tensor=tf.constant([1, 2, 3, 4]))

This will "wrap" the tensor (actually more like "extend" it with metadata) so you can use it with any Keras layer.

jdehesa
  • 58,456
  • 7
  • 77
  • 121
  • How would I make and fit a model using these constants? Do I also have to provide the constants at fit time? – Zach Sep 19 '18 at 19:36
  • 2
    This does not work. But you gave a working solution in another thread: https://stackoverflow.com/questions/46465813/creating-constant-value-in-keras – azureai Apr 28 '19 at 00:40
1

Something to add: When you come to compile the model you need to give the constant input as an input otherwise the graph disconnects

#your input
inputs = Input(shape = (input_shape,))

# an array of ones
constants = [1] * input_shape

# make the array a variable
k_constants = K.variable(constants, name = "ones_variable") 

# make the variable a tensor
ones_tensor = Input(tensor=k_constants, name = "ones_tensor")

# do some layers
inputs = (Some_Layers())(inputs)

# get the complementary of the outputs
output = Subtract()([ones_tensor,inputs])

model = Model([inputs, ones_tensor],output)
model.complie(some_params)

when you train you can just feed in the data you have, you don't need the constant layer anymore.

I have found that no matter what you try it's usually easier to just use a custom layer and take advantage of the power of numpy:

class Complementry(Layer):

    def __init__(self, **kwargs):
        super(Complementry, self).__init__(**kwargs)

    def build(self, input_shape):
        super(Complementry, self).build(input_shape)  # Be sure to call this at the end

    def call(self, x):
        return 1-x  # here use MyArray + x
Andrew Louw
  • 679
  • 5
  • 10