EECE524 Probabilistic Graphical Models, Spring 2010

Announcements

Textbooks

Lectures

Dates and Titles Topics Lecture Slides Suggested Further Readings
Lecture 1
Introduction to PGM
  • Gentle introduction
  • Probability distributions


Lecture 2
Conditional Independence and Factorization
  • Conditional independence
  • Factorization


  • Chapter 2 in Jordan's PGM.
Lecture 3
Message Passing
  • Elimination algorithm
  • Sum product algorithm
  • Max product algorithm
  • Bethe free energy
  • Factor graphs


  • Chapter 3 and 4 in Jordan's PGM.
  • J. S. Yedidia, W. T. Freeman, and Y. Weiss,
    "Constructing free-energy approximations and
    generalized belief propagation algorithms,"
    IEEE Trans. Information Theory, vol. 51, no. 7, 2009.
Lecture 4
Junction tree algorithm
  • Junction tree algorithm


  • Chapter 17 in Jordan's PGM.
Lecture 5
Chow-Liu Tree
  • Chow-Liu tree


  • C. K. Chow and C. N. Liu,
    "Approximating discrete probability distributions with dependence trees,"
    IEEE Trans. Information Theory, vol. 14, no. 3, 1968.
Lecture 6
Density Estimation
  • Density estimation


  • ...
Lecture 7
Variational Inference
  • Variational Inference
  • Variational PCA
  • Variational MoG
  • Variational Bayesian Linear Regression
  • Variational Logistic Regression


  • Chapter 10 in Bishop's PRML.
  • H. Attias (1999),
    "Inferring parameters and structure of latent variable models by variational Bayes,"
    Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, 1999.
  • C. Bishop (1999),
    "Bayesian PCA,"
    NIPS-1998.
  • M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul
    "An introduction to variational methods for graphical models,"
    In Learning in Graphical Models, Cambridge: MIT Press, 1999.
  • T. S. Jaakkola and M. I. Jordan (2000),
    "Bayesian logistic regression: a variational approach,"
    Statistics and Computing, vol. 10, 2000.
Lecture 8
Sampling methods
  • Monte Carlo methods
  • Markov chain Monte Carlo
  • Sequential Monte Carlo


  • Chapter 11 in Bishop's PRML.
  • Chapter on sampling methods in Jordan's PGM.
  • C. Andrieu, N. De Freitas, A. Doucet, and M. I. Jordan (2003),
    "An introduction to MCMC for machine learning,"
    Machine Learning,
    vol. 50, pp. 5-43, 2003.
Lecture 9
Time series models
  • Linear dynamical systems (LDS)
  • Hidden Markov models (HMM)


  • ...

Homework Assignments

  • Hwk 1