Stack Overflow
Stack Overflow

Questions tagged [language-model]

266 questions
0
votes
1 answer

Condition FOR/WHILE in M function - Paginate API in Power BI

I created this API paginated below. It roughly works, but in the OFFSET property, I need to stipulate instead of the number of the next sequence number of the record, for example, for the second page, the number 251, the next record of the second…
Emerson Vaz
  • 1
0
votes
1 answer

What does 'theta' mean in a language model?

I know that if X denotes a text , p(X) denotes the language model of the text. And most often , we use maximum likelihood estimation to estimate the language model. But in many cases , I find a parameter $\theta$ used to represent a language model.…
Bharathi
  • 201
  • 1
  • 8
0
votes
1 answer

Size of input and output layers in Keras implementation of an RNN Language Model

As part of my thesis, I am trying to build a recurrent Neural Network Language Model. From theory, I know that the input layer should be a one-hot vector layer with a number of neurons equal to the number of words of our Vocabulary, followed by an…
Michael
  • 325
  • 3
  • 14
0
votes
1 answer

How to use Deespeech (v0.5.1) effectively and use of language models during training and inference?

I am trying to train and use a model using Deepspeech v0.5.1 for English. My aim to train two models, one with and without a language model. Request your help on several fronts please. Sorry this is long but trying be as detailed as possible; and…
rbewoor
  • 305
  • 1
  • 3
  • 14
0
votes
1 answer

How to fine tune BERT on its own tasks?

I wanted to pre-train BERT with the data from my own language since multilingual (which includes my language) model of BERT is not successful. Since whole pre-training costs a lot, I decided to fine tune it on its own 2 tasks: masked language model…
ozler.kb
  • 13
  • 2
  • 5
0
votes
1 answer

How to predict whether the given sentence is grammatically correct or not?

I am trying to create a predictive model where the model tells whether the give sentence is correct or not by checking the order of the words in the sentence. The model checks weather the particular sequence of words as already occurred in a huge…
Rahul Halappanavar
  • 1
  • 1
0
votes
1 answer

TextLMDataBunch Memory issue Language Model Fastai

I have a dataset with 45 million rows of data. I have three 6gb ram gpu. I am trying to train a language model on the data. For that, I am trying to load the data as the fastai data bunch. But this part always fails because of the memory issue.…
Jerry George
  • 71
  • 7
0
votes
1 answer

How to train ngram model on my own corpus

I have a corpus of list of strings: corpus = ["Hello I am Sam", "This is a white desk","I ate cereals", ...] I want to build a language model (preferably using nltk) on this corpus, to get the probability of a word in a sentence. So, my later…
Cranjis
  • 1,590
  • 8
  • 31
  • 64
0
votes
2 answers

Code to create a reliable Language model from my own corpus

I have a corpus of sentences in a specific domain. I am looking for an open-source code/package, that I can give the data and it will generate a good, reliable language model. (Meaning, given a context, know the probability for each word). Is there…
Cranjis
  • 1,590
  • 8
  • 31
  • 64
0
votes
1 answer

Running out of memory when building a language model in Fast.ai 1.0

I'm dealing with a rather large text dataset (5.4 million short texts) and I'm trying to perform sentiment analysis con them on 16GB of ram. I keep running out of memory whenever I try to build the language model: data_lm =…
Salva Marti
  • 1
  • 1
0
votes
1 answer

How word association mining is generalization of n-gram language model

I am working on text mining (reading book...) author said word association mining is actually the generalization of n-gram language model Can you please tell how word association mining is generalization of n-gram language model. For Me word…
Waseem Ahmad Naeem
  • 850
  • 1
  • 10
  • 22
0
votes
0 answers

Positional Encodings leads to worse convergence, language modeling

This is a tough question, but I might as well try. I'm implementing the architecture from this paper https://arxiv.org/pdf/1503.08895.pdf for language modeling. See page 2 for a diagram, and the top of page 5 for the section on positional or…
Sean Paulsen
  • 55
  • 1
  • 7
0
votes
1 answer

How to learn pocketsphinx for bi-lingual system?

I did create a dictionary with 2 languages(English/Persian) at the one file like this: بگو B E G U خزنده KH A Z A N D E قدت GH A D E T چنده CH A N D E قد GH A D من M A N شب SH A B hi H AA…
Hasani
  • 3,543
  • 14
  • 65
  • 125
0
votes
2 answers

TensorFlow - predicting next word - loss function logit na target shape

I'm trying to create a language model. I have logit and target of size: [32, 312, 512] Where: .shape[0] is batch_size .shape[1] is sequence_max_len .shape[2] is vocabulary size The question is - when I pass logit and target to the loss function…
Ziemo
  • 941
  • 8
  • 27
0
votes
1 answer

how to learn language model?

I'm trying to train a language model with LSTM based on Penn Treebank (PTB) corpus. I was thinking that I should simply train with every bigram in the corpus so that it could predict the next word given previous words, but then it wouldn't be able…
ytrewq
  • 3,670
  • 9
  • 42
  • 71
1 2 3
…
17 18