Inferring missing data with Restricted Boltzmann Machines

Question

Similar to the netflix competition, assume we have a movie dataset with missing ratings. How would I modify RBM to allow it to deduce the missing values? In related papers, one straightforward way is to impute random values to the missing visible features. However, I'm skeptical about the reconstruction accuracy because it can depend on the initial values given to these missing visible nodes.

What do you suggest?

Thanks

score 2 · Accepted Answer · answered Sep 14 '13 at 07:29

https://www.youtube.com/watch?v=laVC6WFIXjg , maybe this video will be of some help.

I think that sampling after imputing random values is a good idea. Hinton justifies this in this video. Also you can try to estimate prior, or to do many samples, or to make guesses based on some different method and then do the reconstruction.

In the video Hinton says that this method isn't very accurate indeed on itself, but when combined with matrix factorization (or other similar methods) can be very powerful.

That video no longer exists. Is there an alternative link? – a06e Apr 23 '21 at 15:10 — a06e, Apr 23 '21 at 15:10

score 2 · Answer 2 · answered Dec 25 '13 at 20:10

2

The idea is to perform alternating Gibbs sampling but keeping the non-missing values fixed to the data values in the reconstruction update. Doing this until the missing values reach a stationary distribution in their Markov Chains and you know what the network's best guess as to what they ought to be is.

answered Dec 25 '13 at 20:10

A.R.Ferguson

133
7

Is there a reference paper on this? – a06e Apr 23 '21 at 15:10

score 1 · Answer 3 · answered Jun 11 '15 at 23:26

Actually the dependency on the initial values given to these missing visible nodes can be used to get some extra 2-5% of accuracy. You can run the RBM several times under different initializations and then average the results. Every ending state will get errors but they'll be different form each other. I tried it and kept improving it until the +/-20th initialization...

Inferring missing data with Restricted Boltzmann Machines

3 Answers3