Questions tagged [elmo]

A tag for ELMo, a way to model words into a deep contextualized representation. It is part of and developed by AllenNLP.

ELMo is a deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). These word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus. They can be easily added to existing models and significantly improve the state of the art across a broad range of challenging NLP problems, including question answering, textual entailment and sentiment analysis.

75 questions
0
votes
0 answers

how to improving the pre-trained embedding

I am currently using the pre-trained elmo model provided by tensorflow_hub. I want to express words such as technical terms and abbreviations well through Elmo embedding. Is there a way to improve the pre-trained elmo by additionally learning new…
0
votes
0 answers

Convert 3d array (num_doc, length, 1024) of ELMo embedding into 2d array for PCA

I did an experiment for build topic modeling using ELMo embedding. I used Elmo for many language here and get 3 dimensional array as the embedding output like this (13294, 200, 1024). I want to do the dimensionality reduction using the PCA; but the…
0
votes
0 answers

How to convert csv to ConLLu format?

I tried to train my data using ELMo pre-trained model in another language beside English https://github.com/HIT-SCIR/ELMoForManyLangs. The input required conllu format while my data is csv format. My question is how to convert csv to conllu…
0
votes
0 answers

TensorFlow error: 'hub.KerasLayer.trainable = True is unsupported when loading from the TF1 Hub format' error when trying to create an EMLo layer

Hello I am trying to embed an ELMo layer into the model below, but am getting an error stating Setting hub.KerasLayer.trainable = True is unsupported when loading from the TF1 Hub format. How to I resolve this? I am using tensorflow==2.10.0 and…
Dieu94
  • 371
  • 1
  • 11
0
votes
1 answer

Error training ELMo - RuntimeError: The size of tensor a (5158) must match the size of tensor b (5000) at non-singleton dimension 1

I am trying to train my own custom ELMo model on AllenNLP. The following bug RuntimeError: The size of tensor a (5158) must match the size of tensor b (5000) at non-singleton dimension 1 arises when training the model. There are instances where the…
Goh Jia Yi
  • 329
  • 3
  • 16
0
votes
1 answer

RuntimeError: variable_scope module_1/ was unused but the corresponding name_scope was already taken. How can i fix it

i got this problem with elmo and tensorflow and i wanna fix it without downgrade. what should i do `**CODE** import tensorflow_hub as hub import tensorflow as tf #Elmo elmo = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True) # Provide…
sarah
  • 1
  • 1
0
votes
0 answers

Saving and loading Keras model with ELMO embedding layer

I'm training a Keras model for token classification with an ELMO layer. I will need to save the model for future use, I've tried with model.save_weights("model_weights.h5"), but then if I load them into a new model that I build, and then I call…
user17796629
0
votes
1 answer

Why my ELMo-CNN model gives worse performance than Word2vec?

I want to compare the performance between ELMo and word2vec as word embedding using the CNN model by classifying 4000 tweet data on five class labels, but the results show that ELMo gives worse performance than word2vec. I used ELMoformanylangs for…
0
votes
1 answer

How to incorporate ELMo into the simple classification of AllenNLP Guide

I am a beginner and not a native English speaker, so I may ask poor questions.Sorry! I recently finished the official AllenNLP tutorial(https://guide.allennlp.org/training-and-prediction) and want to change the simple classifier's word embedding to…
0
votes
1 answer

Keras ELMO fails during training: Unsupported object type int

I've got this network thats using TF Hub's Elmo layer for a classification task. Oddly it starts the training but fails during the process with the error: Unsupported object type int import tensorflow_hub as hub import tensorflow as tf elmo =…
webber
  • 1,834
  • 5
  • 24
  • 56
0
votes
2 answers

Keras ELMO: Error while reading resource variable

I'm trying to get started with using Elmo with Keras & Tensorflow. On running the code below I'm getting an error that seems like the weights aren't initialized. I have also tried defining ElmoLayer class as it's defined here (link), but still the…
webber
  • 1,834
  • 5
  • 24
  • 56
0
votes
0 answers

indices[117,181] = -1 is not in [0, 250) using Elmo and LSTM

I have been using Elmo for getting a matrix embedding on the IMDB dataset on 250 sentences. After I applied Elmo I got an array of dimensions (250, 1024). The code for generating the Elmo matrix is the following: def getElmo(elmo,x): …
Little
  • 3,363
  • 10
  • 45
  • 74
0
votes
1 answer

Elmo embeddings errors

i'm using ELMO to create embeddings for my dataset, i always get this error: embed=elmo(data,signature="default",as_dict=True)["elmo"] TypeError: 'AutoTrackable' object is not callable my code is as simple as this: import tensorflow_hub as…
0
votes
1 answer

How do I produce ELMo embeddings for tokenised strings without getting "Function call stack: pruned"?

I am trying to produce ELMo embeddings for batches of tokenised strings. However I keep receiving the following error: Traceback (most recent call last): File "/home/lorcan/.local/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line…
Lorcán
  • 555
  • 3
  • 15
0
votes
1 answer

ValueError: Input 0 of layer bidirectional is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: (None, 120, 1024, 1024)

I am trying for named entity recognition and here are the details of my x_train and x_test: Shape X_train: (3555, 120, 1024) Shape X_test: (887, 120, 1024) input = Input(shape=(120,)) word_embedding_size = 1024 model = Embedding(input_dim=n_words,…
Sarada
  • 61
  • 1
  • 9