0

I am trying to build up a Keras Model using a keras.layers.Embedding layer with embeddings_initializer=keras.initializers.Identity.

This should let me implement one-hot-encodings directly in the Keras Model, while feeding the model with embeddings representation of word lists (i.e. From word keys to sparse binary vector)

Input Example for each Sample:

[ [5,22,3], # Statement encoding
  [6,9,1,76], # Context encoding
]

I am trying to implement what is described in: Keras One Hot Encoding Memory Management - best Possible way out

(Look at Daniel's Answer)

Expected output from the Embedding Layer:

[ [ [B11], [B12], [B13] ], # Statement encoding for lstm_emb_phrase 
  [[B21], [B22], [B23], [B24] ], # Context encoding for lstm_emb_cont
]

where #Bij# should be a binary array for the word j of the phrase i.

This is the code I currently wrote:

DEFAULT_INNER_ACTIVATION = 'relu'
DEFAULT_OUTPUT_ACTIVATION = 'softplus'

    def __init__(self, sentence_max_lenght, ctx_max_len, dense_features_dim, vocab_size):

        lstm_input_phrase = keras.layers.Input(shape=(sentence_max_lenght,), name='L0_STC_MyApp')
        lstm_input_cont = keras.layers.Input(shape=(ctx_max_len,), name='L0_CTX_MyApp')

        # The following line is #56
        lstm_emb_phrase = keras.layers.Embedding(vocab_size, vocab_size, embeddings_initializer=keras.initializers.Identity,
                                                 input_length=sentence_max_lenght, name='L0E_STC_MyApp')(lstm_input_phrase)

        lstm_emb_phrase = keras.layers.LSTM(DEFAULT_MODEL_L1_STC_DIM, name='L1_STC_MyApp')(lstm_emb_phrase)
        lstm_emb_phrase = keras.layers.Dense(DEFAULT_MODEL_L2_STC_DIM, name='L2_STC_MyApp', activation=DEFAULT_INNER_ACTIVATION)(lstm_emb_phrase)


        lstm_emb_cont = keras.layers.Embedding(vocab_size, vocab_size,
                                               embeddings_initializer=keras.initializers.Identity,
                                               input_length=ctx_max_len, name='L0E_CTX_MyApp')(lstm_input_cont)

        lstm_emb_cont = keras.layers.LSTM(DEFAULT_MODEL_L1_CTX_DIM, name='L1_CTX_MyApp')(lstm_emb_cont)
        lstm_emb_cont = keras.layers.Dense(DEFAULT_MODEL_L2_CTX_DIM, name='L2_CTX_MyApp', activation=DEFAULT_INNER_ACTIVATION)(lstm_emb_cont)


        x = keras.layers.concatenate([lstm_emb_phrase, lstm_emb_cont])
        x = keras.layers.Dense(DEFAULT_MODEL_L3_DIM, activation=DEFAULT_INNER_ACTIVATION)(x)
        x = keras.layers.Dense(DEFAULT_MODEL_L4_DIM, activation=DEFAULT_INNER_ACTIVATION)(x)

        main_output = keras.layers.Dense(DEFAULT_OUTPUT_DIM, activation=DEFAULT_OUTPUT_ACTIVATION)(x)

        self.model = keras.models.Model(inputs=[lstm_input_phrase, lstm_input_cont],
                                        outputs=main_output)

        self.model.compile(loss='binary_crossentropy', metrics=['accuracy'])

This is the error I get when I try to run my script:

WARNING:tensorflow:From [PATH_TO_MyApp]\venv\lib\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (
from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
Traceback (most recent call last):
  File "my_script.py", line 307, in <module>
    main()
  File "my_script.py", line 60, in main
    flag = flag or train_r_skills(argsparser)
  File "my_script.py", line 230, in train_r_skills
    sk_extr = MyKerasModel(stm_max_len, ctx_max_len, segm_max_len, vocab_size)
  File "[PATH_TO_MyApp]\MyApp\parser\MyKerasModel_002.py", line 56, in __init__
    input_length=ctx_features_dim, name='L0E_CTX_MyApp')(lstm_input_cont)
  File "[PATH_TO_MyApp]\venv\lib\site-packages\keras\engine\base_layer.py", line 431, in __call__
    self.build(unpack_singleton(input_shapes))
  File "[PATH_TO_MyApp]\venv\lib\site-packages\keras\layers\embeddings.py", line 109, in build
    dtype=self.dtype)
  File "[PATH_TO_MyApp]\venv\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "[PATH_TO_MyApp]\venv\lib\site-packages\keras\engine\base_layer.py", line 252, in add_weight
    constraint=constraint)
  File "[PATH_TO_MyApp]\venv\lib\site-packages\keras\backend\tensorflow_backend.py", line 402, in variable
    v = tf.Variable(value, dtype=tf.as_dtype(dtype), name=name)
  File "[PATH_TO_MyApp]\venv\lib\site-packages\tensorflow\python\ops\variables.py", line 213, in __call__
    return cls._variable_v1_call(*args, **kwargs)
  File "[PATH_TO_MyApp]\venv\lib\site-packages\tensorflow\python\ops\variables.py", line 176, in _variable_v1_call
    aggregation=aggregation)
  File "[PATH_TO_MyApp]\venv\lib\site-packages\tensorflow\python\ops\variables.py", line 155, in <lambda>
    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
  File "[PATH_TO_MyApp]\venv\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 2495, in default_variable_creator
    expected_shape=expected_shape, import_scope=import_scope)
  File "[PATH_TO_MyApp]\venv\lib\site-packages\tensorflow\python\ops\variables.py", line 217, in __call__
    return super(VariableMetaclass, cls).__call__(*args, **kwargs)
  File "[PATH_TO_MyApp]\venv\lib\site-packages\tensorflow\python\ops\variables.py", line 1395, in __init__
    constraint=constraint)
  File "[PATH_TO_MyApp]\venv\lib\site-packages\tensorflow\python\ops\variables.py", line 1503, in _init_from_args
    initial_value(), name="initial_value", dtype=dtype)
TypeError: __call__() missing 1 required positional argument: 'shape'

Can you help me to understand what is going on?

EDIT: I checked also Convert code from Keras 1 to Keras 2: TypeError: __call__() missing 1 required positional argument: 'shape' , as C. Lightfoot suggested, however, at the best of my understanding seems to not match my issue (I am not converting code from Keras 1 to Keras 2). Please give me more hints if there is anything I am missing about this article.

Cheers, /H

Hermes
  • 19
  • 5
  • 1
    Possible duplicate of [Convert code from Keras 1 to Keras 2: TypeError: \_\_call\_\_() missing 1 required positional argument: 'shape'](https://stackoverflow.com/questions/48533216/convert-code-from-keras-1-to-keras-2-typeerror-call-missing-1-required-p) – C. Lightfoot Mar 29 '19 at 10:58
  • 1
    I checked https://stackoverflow.com/questions/48533216/convert-code-from-keras-1-to-keras-2-typeerror-call-missing-1-required-p, as C. Lightfoot suggested, however, at the best of my understanding seems to not match my issue (I am not converting code from Keras 1 to Keras 2). Please give me more hints if there is anything I am missing about this article. – Hermes Mar 29 '19 at 11:15
  • 1
    I put in place a temporary workaround, by manipulating the input before feeding the NN. Therefore I am not using an Embedding Layer for now. Please let me know if there is a solution to this issue. – Hermes Apr 01 '19 at 20:19

0 Answers0