-1

I am studying Syntaxnet of Tensoflow.

Howerver, I'm confusing about the way to use --word_embeddings option. Could you show me a example? Thanks a lot.

Mistalis
  • 17,793
  • 13
  • 73
  • 97
Yang W
  • 1

1 Answers1

0

Not too sure what syntaxnet is but here is how I use word embeddings in my sequence to sequence model.

Here I declare a variable which will store the embedding. It is a matrix with rows equal to the number of words in the vocabulary (NWORDS) and each column is the size of the word vector for that word (WORD_VEC_SIZE). It is randomly initialized and is a trainable parameter of the model.

  word_embedding = tf.get_variable('word_embedding', shape = (NWORDS, WORD_VEC_SIZE), initializer = tf.truncated_normal_initializer(0.0, 1,0))

The input to my model is a list of integers which represent the indexs of the words in the embedding. It is of dimension BATCH_SIZE x TIMESTEPS.

source_input = tf.placeholder(tf.int32, (BATCH_SIZE, TIME_STEPS), 'source_input')

Using the embedding and the input I can perform a lookup to convert the integer representations of the words into vectors. 

y = tf.nn.embedding_lookup([word_embedding], source_input)

Now y has the shape (BATCH_SIZE, TIME_STEPS, WORD_VEC_SIZE) and can be fed into the model for further processing.

chasep255
  • 11,745
  • 8
  • 58
  • 115
  • Thanks for your answer, but it's not exactly solution to my question. When I load pretrained word embeddings in syntaxnet, the error list below:raise type(e)(node_def, op, message) tensorflow.python.framework.errors.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [5,8] rhs shape= [31157,128] [[Node: embedding_matrix_0/init = Assign[T=DT_FLOAT, _class=["loc:@embedding_matrix_0"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/cpu:0"](embedding_matrix_0, embedding_matrix_0/init/WordEmbeddingInitializer)]] – Yang W Jul 22 '16 at 00:54