naive Metropolis-Hasting vs. numpy vs. Gibbs

by Yubo 'Paul' Yang

Gibbs Sampling

The basic Gibbs sampler samples a joint probability distribution one variable at a time. Each random variable is sampled from its full conditional probability distribution with all other variables fixed. Independent variables can be sampled simultaneously, making the Gibbs sampler ideal for the restricted Boltzmann machine.

Presentation Summary

In this presentation, I present:

  • Basic idea behind the Gibbs sampling algorithm.
  • Basic application to sample the bivariate normal distribution.
  • Application to change-point detection
  • Application to restricted Boltzmann machine

Examples

  • example 1: sample bivariate normal distribution
  • example 2: optimize change-point detection model
  • example 3: train Bernoulli restricted Boltzmann machine on NMIST data

References

All optimization.