Dr Konstantinos Kalogeropoulos

This course is available on the MSc in Applied Social Data Science, MSc in Data Science, MSc in Quantitative Methods for Risk Management, MSc in Statistics, MSc in Statistics (Financial Statistics), MSc in Statistics (Financial Statistics) (Research), MSc in Statistics (Research), MSc in Statistics (Social Statistics) and MSc in Statistics (Social Statistics) (Research). This course is available as an outside option to students on other programmes where regulations permit.

Basic knowledge in probability and first course in statistics such as ST202 or equivalent Probability Distribution Theory and Inference; basic knowledge of the principles of computer programming is sufficient (e.g. in any of Python, R, Matlab, C, Java).This is desired rather than essential.

The course will introduce the basic principles and algorithms used in Bayesian machine learning. This will include the Bayesian approach to regression and classification tasks, introduction to the concept of graphical models, and Bayesian statistical inference, including approximate inference methods such as variational approximation and expectation propagation, and various sampling-based methods. The course will include also a module on the Bayesian modelling and inference for sequential data.  The examples will include timely applications found in the context of content recommendation systems, fraud detection, and skill rating systems.

20 hours of lectures and 15 hours of computer workshops in the LT.

Here is a tentative syllabus:

Bayesian inference concepts

-      Prior and posterior distributions

-      Bayes estimators, credible intervals, Bayes factors

-      Bayesian forecasting, Posterior Predictive distribution

Linear models for regression

- Linear basis function models

- Bayesian linear regression

- Bayesian model comparison

Linear models for classification

- Probabilistic generative models

- Probabilistic discriminative models

- The Laplace approximation

- Bayesian logistic regression

Graphical models

- Bayesian networks

- Conditional independence

- Markov random fields

- Inference in graphical models

Mixture models and Expectation Maximization

- K-means clustering

- Mixtures of Gaussians

- The EM algorithm

Approximate inference

- Variational inference

- Variational logistic regression

- Expectation propagation

Sampling methods

- Basic sampling algorithms

- Markov chain Monte Carlo

- Gibbs sampling

Sequential data

- Markov models

- Hidden Markov models

- Linear dynamical systems

If time permits also

Gaussian processes – Bayesian non-parametrics

- Gaussian processes for regression.

- Gaussian processes for classification.

Students will be expected to produce 10 problem sets in the LT.

10 problem sets in LT to prepare students for both summative assessment components. They will include theoretical exercises, targeting for learning outcomes a and b, as well as computer-based assignments (for learning outcome c) that will need to be presented in suitable form for the purposes of learning outcome d. Additionally, mostly related to learning outcome b, students will be encouraged to share and compare their responses in some challenging parts of the problem sets, through the use of dedicated Moodle forums.

• S. Rogers and M. Girolami, A First Course in Machine Learning, Second Edition, Chapman and Hall/CRC, 2016

• K. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012

• C. M. Bishop, Pattern Recognition and Machine Learning, Springer 2006

• D. J. C. MacKay, Information Theory, Inference and Learning Algorithms, Cambridge University Press, 2003

• D. Barber, Bayesian Reasoning and Machine Learning, Cambridge University Press 2012

Exam (50%, duration: 2 hours) in the summer exam period.
Project (50%) in the ST.