Questions tagged [language-model]
266 questions
0
votes
1 answer
Tensorflow hub-NNLM word embedding using sentiment140 data gives input shape error
I am using tensorflow hub "https://tfhub.dev/google/nnlm-en-dim128/2" word embedding for the sentiment analysis of Kaggle "sentiment140" dataset.
Data set : Kaggle("sentiment140") https://www.kaggle.com/kazanova/sentiment140
Tensorflow-Hub :…
Sekar
- 37
- 8
0
votes
1 answer
Does adding a list of Word2Vec embeddings give a meaningful represenation?
I'm using a pre-trained word2vec model (word2vec-google-news-300) to get the embeddings for a given list of words. Please note that this is NOT a list of words that we get after tokenizing a sentence, it is just a list of words that describe a…
Padmaja Bhagwat
- 135
- 1
- 1
- 6
0
votes
1 answer
How to incoporate mask into negative likelihood loss (torch.nn.functional.nll_loss)
Hello I am implementing a lstm for language modelling
for homework and I am at the loss implementation phase. Our instructor told us to use F.nll_loss but the sequences are padded and we have to take into account a mask that is given which tells us…
SolipsistElvis
- 71
- 8
0
votes
1 answer
loading a FastText model in MATLAB
I have trained a FastText model in Python and saved the files into a folder. These are the contents of the…
user14251114
0
votes
3 answers
Set up kenlm for Windows
The official website makes it pretty clear that there is no support for kenlm in Windows. There is a Windows tag at the github repository but it seems to be maintained by few random contributors then and there.
How to set up kenlm for Windows then?
Bendemann
- 735
- 11
- 31
0
votes
1 answer
Assigning weights during testing the bert model
I have a basic conceptual doubt. When i train a bert model on sentence say:
Train: "went to get loan from bank"
Test :"received education loan from bank"
How does the test sentence assigns the weights for each token because i however dont pass…
prog
- 1,073
- 5
- 17
0
votes
1 answer
How to get text from XML via Python?
I'm training language model. My input are subtitles in XML format. I need to get just plain text from it and save to to a text file so I can work with it.
Input
0
votes
1 answer
How to work with n-grams for classification tasks?
I'm going to train a classifier on a sample dataset using n-gram. I searched for related content and wrote the code below. As I'm a beginner in python, I have two questions.
1- Why should the dictionary have this 'True' structure (marked with…
ECub Devs
- 165
- 3
- 10
0
votes
1 answer
How can i use kenlm to check word alignment in a sentence?
I have seen many blogs saying language models can be used for numerous tasks but I cannot find any good implementations other than just the text generation.
My query is,
How can i use a language model like kenlm to correct my sentences for…
0
votes
1 answer
fine tune causal language model using transformers and pytorch
I have some questions about fine-tuning causal language model using transformers and PyTorch.
My main goal is to fine-tune XLNet. However, I found the most of posts online was targeting at text classification, like this post. I was wondering, is…
BigD
- 571
- 7
- 24
0
votes
1 answer
LM in elastic search
how can I improve recall for this condition ?any suggestion?
I want to create an index with 39 million passages each one containing at least four sentences in English. My queries are short and interrogative sentences. I know that a language model…
Omk
- 1
- 2
- 16
0
votes
1 answer
How to tell Alexa to punctuate user responses properly. Please see the use case
I am sorry if this looks like a stupid question!
My skill is recording user responses in the database. This part is working fine. But my concern is Alexa is not punctuating the response at all. Here is an example:
User: The loading speed of the…
Subrata Sarkar
- 2,975
- 6
- 45
- 85
0
votes
1 answer
BERT:Question-Answering - Total number of permissible words/tokens for training
Let's say I want to train BERT with 2 sentences (query-answer) pair against a certain binary label (1,0) for the correctness of the answer, will BERT let me use 512 words/tokens each for the query and the answer or together(query+answer combined)…
Soumya Ranjan Sahoo
- 133
- 2
- 9
0
votes
1 answer
In PyTorch, what's the difference between training an RNN to predict the last word given a sequence, vs predicting the entire sequence shifted?
Let's say I'm trying to train an RNN language model in PyTorch. Suppose I iterate over batches of word sequences, and that each training batch tensor has the following shape:
data.shape = [batch_size, sequence_length, vocab_dim]
My question is,…
user1500142
- 185
- 1
- 6
0
votes
1 answer
Transformer-XL: Input and labels for Language Modeling
I'm trying to finetune the pretrained Transformer-XL model transfo-xl-wt103 for a language modeling task. Therfore, I use the model class TransfoXLLMHeadModel.
To iterate over my dataset I use the LMOrderedIterator from the file…
Rafael-WO
- 1,434
- 14
- 24