Hidden Markov models package in R

Question

I need some help implementing a HMM module in R. I'm new to R and don't have a lot of knowledge on it. So i have to implement an IE using HMM, i have 2 folders with files, one with the sentences and the other with the corresponding tags i want to learn form each sentence.

folder1 > event1.txt: "2013 2nd International Conference on Information and Knowledge Management (ICIKM 2013) will be held in Chengdu, China during July 20-21, 2013."

folder2 > event1.txt: 
"N: 2nd International Conference on Information and Knowledge Management (ICIKM 2013)
D: July 20-21, 2013
L: Chengdu, China"

N -> Name; D -> Date; L -> Location

My question is how to implement it on R, how do i initialize the model and how do i do to train it? And then how do i apply it to a random sentence to extract the information?

Thanks in advance for all the help!

score 10 · Answer 1 · answered Jul 17 '13 at 14:48

10

If you run the following command:

RSiteSearch('hidden markov model')

Then it finds 4 Task Views, 40 Vignettes, and 255 functions (when I ran it, there could be more by the time you run it).

I would suggest looking through those results (probably start with the views and vignettes) to see if anything there works for you. If not, then tell us what you have tried and what you need that is not provided there.

answered Jul 17 '13 at 14:48

Greg Snow

48,497
6
83
110

Didn't knew that command, as i said i'm a newbie on R so there are a lot of commands that i don't know. I will take a look using the one u said. Thanks! If it doesn't solve my problem i ask again. – Tiago Oliveira Jul 17 '13 at 17:29
Ok, i'm not being capable of understand how HMM library works. Can you just show a litle example of how to apply it to a sentence? I know how to read all the sentences in the different files to form a corpus, but then how i need to treat them (cause of the tags i have on the second files)? And how does i use those data to train the model? All i see is using "X, Y" and "a, b, c". I would love to see something more concrete. Thanks in advance! – Tiago Oliveira Jul 18 '13 at 15:52

score 8 · Answer 2 · answered Jul 29 '13 at 18:15

8

I'm not sure what exactly you want to do, but you might find this excellent tutorial on hidden Markov models using R useful. You build the functions and Markov models from scratch starting from regular Markov models and then moving to hidden Markov models. That's really valuable to understand how they work.

There is also the R package depmixS4 for specifying and fitting hidden Markov models. It's documentation is pretty solid and going through the example code might help you.

answered Jul 29 '13 at 18:15

Brian Fabian Crain

860
2
8
16

Hi. Thanks for the tutorial, i will take a look. Now i will try to explain a little better what i want. I have some sentences already tagged as i said on my first post. With them i want to train an HMM to extract those 3 entities. After i have the HMM trained i want to use it on new sentences (about events off course) to extract the name, location and date of the event. I'm correct when i talk about using HMM right? – Tiago Oliveira Jul 31 '13 at 10:04

score 3 · Answer 3 · edited Nov 21 '17 at 12:21

3

depmixS4 is most general and reasonably good package, if you get it to work on your data. It checked out on dummy data for me but gave error on real data. HMM also works but only if you have discrete variables and not continuous.

edited Nov 21 '17 at 12:21

Benjamin Christoffersen

4,703
15
37

answered May 06 '15 at 10:19

mlworker

281
3
9

score 1 · Answer 4 · edited Feb 18 '17 at 17:44

DepmixS4 is what you are looking for.

First of all, you need to identify best number of hidden states for your model. This can be done by taking model with least value of AIC for different hidden states.

I have created a function HMM_model_execution which will return the model variable and number of states for the model.

  library(depmixS4)

first column should be visible state and remaining external variable in doc_data
```
HMM_model_execution<-function( doc_data, k) 
```

k number of total hidden state to compare

{
     aic_values <- vector(mode="numeric", length=k-1) # to store AIC values

     for( i in 2:k)
     {
           print(paste("loop counter",i))
           mod <- depmix(response = doc_data$numpresc ~ 1, data = doc_data, nstates = i)
           fm <- fit(mod, verbose = FALSE)
           aic_values[i-1]<- AIC(fm)
           #print(paste("Aic value at this index",aic_values[i-1]))
           #writeLines("\n")
     }

     min_index<-which.min(aic_values)

no of hidden states for best model

#print(paste("index of minimum AIC",min_index))
mod <- depmix(response = doc_data$numpresc ~ 1, data = doc_data, nstates = (min_index+1))
fm <- fit(mod, verbose = FALSE)

best model execution

print(paste("best model with number of hidden states", min_index+1))
return(c(fm, min_index+1))
writeLines("\n")
writeLines("\n")

External variables( co-variates can be passed in function depmix ). summary (fm) will give you all model parameters.

Actual Code for all the model parameter calulation. Viterbi Algo, EM can be found at [link] https://github.com/koooee/depmixS4 — Arpit Sisodia, Feb 08 '17 at 15:56

Hidden Markov models package in R

4 Answers4

Linked