keras add features after embedding

Question

I want to add part of speech features into my word vector after embedding in Keras. I would like to add them as one hot and concat them after embedding. But the part of speech of a word is dynamic so I can't use another embedding layer for part of speech one hot look up and combine two embedding layers.

so in your case you have feature x and you need [x, x1] where x1 = f(x). This can be done as preprocessing step, before feeding to model. — sathyz, Jul 19 '18 at 07:07

score 1 · Accepted Answer · answered Jul 19 '18 at 06:47

1

Here is a simple approach, I'm assuming input as a word and its POS tag.

word = Input(...)
pos = Input(...)
emb = Embedding( ... ) (word)
layer = Concatenate()([emb, pos])
outputs = .... # your processing
model = Model(inputs=[word,pos], outputs=outputs)

answered Jul 19 '18 at 06:47

sathyz

1,401
10
12

Thanks a lot. I'll give it a try. – Zedom Jul 19 '18 at 07:10
@sathyz can you please explain where the `layer` variable is used? – KLaz Aug 14 '19 at 18:08
Refer [Keras functional API](https://keras.io/getting-started/functional-api-guide/). In this case, you could just do `outputs = Dense(10, activation="softmax") (layer)` – sathyz Aug 30 '19 at 06:50

score 0 · Answer 2 · answered Nov 27 '19 at 00:43

SathyZ just want to know we dont have to use embedding layer for pos tag

word = Input(...)
pos = Input(...)
emb_word = Embedding( ... ) (word)
emb_pos = Embedding( ... ) (pos)
layer = Concatenate()([emb, pos])
outputs = .... # your processing
model = Model(inputs=[word,pos], outputs=outputs)

keras add features after embedding

2 Answers2