# COMPGI13 - Advanced Topics in Machine Learning

**This database contains the 2018-19 versions of syllabuses.**

**Note:** Whilst every effort is made to keep the syllabus and assessment records correct, the precise details must be checked with the lecturer(s).

- Code
- COMPGI13 (Also taught as: COMPM050)
- Year
- MSc
- Prerequisites
- The prerequisites are probability, calculus, linear algebra, COMPGI01 Supervised Learning and
- either COMPGI08 Graphical Models or COMPGI18 Probabilistic and Unsupervised Learning
- Term
- 2
- Taught By
- Arthur Gretton, with Dino Sejdinovic & Bharath Sriperumbudur (Gatsby Computational Neuroscience Unit (50%)

David Silver (Computer Science) (50%)

- Aims
- Aims: To learn: 1) Fundamentals of reproducing kernel Hilbert spaces, and learning algorithms that use them, 2) To understand methods for reinforcement learning, planning and control in seqential decision making processes.
- Learning Outcomes
- Learning outcomes: To gain in-depth familiarity with the selected research topics, understand their theory and applications, be able to individually read, understand and discuss research works in the field.

# Content:

**Kernel Methods**

- 2. Simple linear algorithms in RKHS (e.g. PCA, ridge regression)

- 3. Support vector machines for classification, regression, density support estimation

- 4. Kernel methods for hypothesis teating (two-sample, independence)

5. Further applications (feature selection, clustering, ICA)

6. Theory of reproducing kernel Hilbert spaces (optional, not assessed)

**Learning and control of stochastic dynamical systems**

1. Introduction

2. Markov Decision Processes

3. Planning by dynamical programming

4. Model-free prediction

5. Model-free control

6. Function approximation

7. Policy gradient

8. Integrating learning and planning

9. Exploration and exploitation

10. Case study: reinforcement learning in games

# Method of Instruction:

Lectures, reading, presentation and associated class problems.

# Assessment:

The course has the following assessment components:

Written Examination (2.5 hours, 50%)

Coursework Section (2 pieces, 50%)

To pass this course, students must:

Pass the coursework section

Obtain an overall pass mark of 50% for all sections combined

There will be one coursework assignment in each half of the course.

Coursework will be assessed on the maximum mark out of these two assignments.

The examination rubric is:

There will be two sections corresponding to the two halves of the course: section A and B, each containing three questions. You should answer any three questions.

# Resources:

**Kernel methods:**

Ingo Steinwart and Andreas Christmann, Support Vector Machines (Springer 2008)

A. Berlinet and C. Thomas-Agnan, Reproducing Kernel Hilbert Spaces in Probability and Statistics (Kluwer 2004)

H Wendland, Scattered Data Approximation (Cambridge University Press, 2005)

B Schoelkopf and A. Smola, Learning with Kernels (MIT Press, 2002)

J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis (Cambridge, 2004)

Carl E. Rasmussen and C.K.I. Williams: Gaussian Processes for Machine Learning (MIT Press, 2006)

**Reinforcement learning:**

R. Sutton and A. Barto, An Introduction to Reinforcement Learning

(http://webdocs.cs.ualberta.ca/~sutton/book/ebook/the-book.html)

C. Szepesvari, Algorithms for Reinforcement Learning

(http://www.sztaki.hu/~szcsaba/papers/RLAlgsInMDPs-lecture.pdf)