by Yubo 'Paul' Yang

# Gibbs Sampling

The basic Gibbs sampler samples a joint probability distribution one variable at a time. Each random variable is sampled from its full conditional probability distribution with all other variables fixed. Independent variables can be sampled simultaneously, making the Gibbs sampler ideal for the restricted Boltzmann machine.

## Presentation Summary

In this presentation, I present:

- Basic idea behind the Gibbs sampling algorithm.
- Basic application to sample the bivariate normal distribution.
- Application to change-point detection
- Application to restricted Boltzmann machine

## Examples

- example 1: sample bivariate normal distribution
- example 2: optimize change-point detection model
- example 3: train Bernoulli restricted Boltzmann machine on NMIST data

## References

- MCMC: The Gibbs Sampler, The Clever Machine
- Beyesian Inference: Metropolis-Hasting Sampling, Ilker Yildirim
- Beyesian Inference: Gibbs Sampling, Ilker Yildirim
- A Practical Guide to Training Restricted Boltzmann Machines, Geoffrey E. Hinton
- Introduction to Restricted Boltzmann Machines, Edwin Chen
- deeplearning.net

### All optimization.

- Kevin Kleiner: Algorithm Interest Group on March 22, 2021 ·
**Exploring Stochastic Gradient Descent and its Modifications** - by Yubo "Paul" Yang ·
**Linear Sum Assignment** - by Kevin Ly ·
**PSLQ** - by Chad Germany ·
**Simulated Annealing** - by Brian Busemeyer ·
**BK-Tree** - by Brian Busemeyer ·
**Simulated annealing** - by Brian Busemeyer ·
**Cellular Automaton** - by Juha Tiihonen ·
**Automatic focusing of cameras** - by Yubo "Paul" Yang ·
**Particle Swarm Optimization** - by Brian Busemeyer ·
**Evolutionary algorithms**

Yubo "Paul" Yang ALGORITHM

machine learning