by Yubo 'Paul' Yang

Gibbs Sampling

The basic Gibbs sampler samples a joint probability distribution one variable at a time. Each random variable is sampled from its full conditional probability distribution with all other variables fixed. Independent variables can be sampled simultaneously, making the Gibbs sampler ideal for the restricted Boltzmann machine.

Presentation Summary

In this presentation, I present:

Basic idea behind the Gibbs sampling algorithm.
Basic application to sample the bivariate normal distribution.
Application to change-point detection
Application to restricted Boltzmann machine

Examples

example 1: sample bivariate normal distribution
example 2: optimize change-point detection model
example 3: train Bernoulli restricted Boltzmann machine on NMIST data

References

MCMC: The Gibbs Sampler, The Clever Machine
Beyesian Inference: Metropolis-Hasting Sampling, Ilker Yildirim
Beyesian Inference: Gibbs Sampling, Ilker Yildirim
A Practical Guide to Training Restricted Boltzmann Machines, Geoffrey E. Hinton
Introduction to Restricted Boltzmann Machines, Edwin Chen
deeplearning.net

All optimization.

Yubo "Paul" Yang 2017-01-24 ALGORITHM
machine learning