Questions tagged [elmo]

A tag for ELMo, a way to model words into a deep contextualized representation. It is part of and developed by AllenNLP.

ELMo is a deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). These word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus. They can be easily added to existing models and significantly improve the state of the art across a broad range of challenging NLP problems, including question answering, textual entailment and sentiment analysis.

75 questions
2
votes
2 answers

error when try to import the Elmo module from TensorFlow Hub?

I'm not able to import the elmo module from TensorFlow Hub. I am able to import other modules and use them successfully. I'm running TF2.0 on a GCP Jupyterlab instance with GPUs. When I try this: import tensorflow as tf import tensorflow_hub as…
Alex Kinman
  • 2,437
  • 8
  • 32
  • 51
2
votes
1 answer

LSTM - Elmo from TF-Hub with TF2.0 pure implementation

I have the following problem mainly due to my lack of expertise with TF in general and TF2.0 in particular. I try to use Elmo embedding with a keras LSTM implementation. The code that I am inspired from is using the TF-Hub Elmo module. Which appears…
2
votes
0 answers

tensorflow_hub to pull BERT embedding on windows machine

I would like to get BERT embedding using tensorflow hub. I found it very easy to get ELMO embedding and my steps are below. Could anyone explain how to get BERT embedding on a windows machine? I found this but couldn't get it work on windows…
user2543622
  • 5,760
  • 25
  • 91
  • 159
2
votes
1 answer

confuse about parameter 'tokens_length' of elmo model in tensorflow hub

I'm looking the ELMo model in tensorflow hub and, I'm not very clear about what does tokens_length = [6, 5] means in the flow example use: (https://tfhub.dev/google/elmo/2) elmo = hub.Module("https://tfhub.dev/google/elmo/2",…
xiao
  • 542
  • 1
  • 9
  • 16
2
votes
0 answers

How can we fix following error " InternalError: Unable to get element as bytes"?

I am a student, and I am trying to implement text classification using elmo on keras. I imported elmo layer from tensorflow-hub. def ELMoEmbedding(x): return embed(inputs={ "tokens": tf.squeeze(tf.cast(x, tf.string)), "sequence_len":…
hR 312
  • 824
  • 1
  • 9
  • 22
2
votes
0 answers

What is the difference between word2vec, glove, and elmo?

What is the difference between word2vec, glove, and elmo? According to my understanding all of them are used for training word embedding, am I correct?
hR 312
  • 824
  • 1
  • 9
  • 22
2
votes
2 answers

keras pad_sequence for string data type

I have a list of sentences. I want to add padding to them; but when I use keras pad_sequence like this: from keras.preprocessing.sequence import pad_sequences s = [["this", "is", "a", "book"], ["this", "is", "not"]] g = pad_sequences(s, dtype='str',…
1
vote
0 answers

tf hub.module can't load elmo. I hope it works normally

When I run the following code with Jupyter Notebook, it originally worked normally, but suddenly an error pops up. import tensorflow.compat.v1 as tf tf.disable_v2_behavior() tf.disable_eager_execution() # Load pre trained ELMo model elmo =…
1
vote
1 answer

Allennlp: How to load a pretrained ELMo as the embedding of allennlp model?

I am new in allennlp. I trained an elmo model to apply it to other allennlp models as the embedding but failed. It seems that my model is not compatible to the interface the config gives. What can I do? My elmo is trained by allennlp with the…
IshiKura-a
  • 13
  • 2
1
vote
0 answers

Error with ELMO embeddings - TensorArray has size zero, but element shape [?,256] is not fully defined. Currently only static shapes are supported

I Am trying to implement ELMO embeddings via tensorflow in a neural network. Here is a code snippet of my network : def get_elmo_embeds_model(): input_text = tf.keras.layers.Input(shape=(1,), dtype=tf.string) embedding =…
1
vote
1 answer

Keras: How to load a model with pretrained ELMO Layer

I have trained a Deep Learning network that has a pretrained ELMO layer. I've saved the model and weights using the code below. model.save("model.h5") model.save_weights("weights.h5") I now need to load the load but I'm not sure whats the right…
webber
  • 1,834
  • 5
  • 24
  • 56
1
vote
1 answer

Tensorflow urllib.error.URLError:

Recently, I tried to use the elmo in tensorflow, but I meet some mistakes, if you can help me, I would be really appreciate. this is my test_code: import tensorflow as tf import tensorflow_hub as hub import numpy as np import urllib.request if…
杨泽鹏
  • 11
  • 2
1
vote
1 answer

Comparison among ELMo, BERT, and GloVe

What are the differences among ELMo, BERT, and GloVe in word representation? How differently do they perform word embedding tasks? Which one is better and what advantages and disadvantages does each have in comparison with others?
user14251114
1
vote
0 answers

Flair Elmo Embeddings use in Pytorch model

I am trying to generate the elmo embeddings on a PyTorch model, on every batch iteration, like: for batch in iterator: optimizer.zero_grad() embeddings = get_elmo_embeddings(batch.dataset.examples) predictions =…
Nick
  • 11
  • 2
1
vote
1 answer

Training data preparation for Training ELMO Embedding from scratch

I am trying to build my own custom chemical domain ELMO embedding. I am following the instruction from https://github.com/allenai/bilm-tf How do I prepare the training data if I have many multi word token in domain like chemistry. For example: 1.…