COMP0083 Advanced Topics in Machine Learning

This database contains the 2018-19 versions of syllabuses.

Note: Whilst every effort is made to keep the syllabus and assessment records correct, the precise details must be checked with the lecturer(s).

Academic session



Advanced Topics in Machine Learning



Module delivery

1819/A7U/T1/COMP0083 Masters (MEng)

Related deliveries

1819/A7P/T1/COMP0083 Postgraduate

Prior deliveries



Masters (MEng)

FHEQ Level


FHEQ credits



Term 1

Module leader

Gretton, Arthur


Gretton, Arthur

Ciliberto, Carlo

Module administrator

Ball, Louisa


Kernel methods

To gain an understanding of the theory and applications of kernel methods, including:

  • An overview of how kernel feature spaces can be constructed, including in infinite dimensions, and the smoothing properties of functions in these spaces.
  • Simple and complex learning algorithms using kernels (ridge regression, kernel PCA, the support vector machine)
  • Representations of probabilities in reproducing kernel Hilbert spaces. Statistical two-sample and independence tests, and learning algorithms using these embeddings (clustering, ICA)

Learning theory

To learn the fundamentals of statistical learning theory. In particular to:

  • Understand what characterizes a learning problem and what it means for an algorithm/system/machine to “learn”.
  • Understand the key role of regularization and the different approaches to use it efficiently in practice.
  • Acquire familiarity with a variety of statistically consistent learning algorithms, both from modeling and practical perspectives.

Learning outcomes

On successful completion of the module, a student will be able to:

  1. gain in-depth familiarity with the selected research topics, understand how to design and implement learning algorithms;
  2. individually read, understand and discuss research papers in the field.

Availability and prerequisites

This module delivery is available for selection on the below-listed programmes. The relevant programme structure will specify whether the module is core, optional, or elective.

In order to be eligible to select this module as optional or elective, where available, students must meet all prerequisite conditions to the satisfaction of the module leader. Places for students taking the module as optional or elective are limited and will be allocated according to the department’s module selection policy.

Programmes on which available:

  • MEng Computer Science (International Programme) (year 4)
  • MEng Computer Science (year 4)
  • MEng Mathematical Computation (International Programme) (year 4)
  • MEng Mathematical Computation (year 4)


In order to be eligible to select this module, students must have a strong understanding of Linear Algebra, Probability Theory, and Calculus.


Introduction to kernel methods

  • Definition of a kernel, how it relates to a feature space, The reproducing kernel Hilbert space
  • Simple applications: kernel PCA, kernel ridge regression
  • Distance between means in RKHS, integral probability metrics, the maximum mean discrepancy (MMD), two-sample tests
  • Choice of kernels for distinguishing distributions, characteristic kernels
  • Covariance operator in RKHS: proof of existence, definition of norms (including HSIC, the Hilbert-Schmidt independence criterion)
  • Application of HSIC to independence testing
  • Feature selection, taxonomy discovery.
  • Introduction to independent component analysis, kernel ICA
  • Large margin classification, support vector machines for classification

Introduction to supervised learning in the context of statistical learning theory

  • A taxonomy of learning problems
  • No free lunch theorem
  • Regularization
  • Model selection
  • Stability and generalization
  • Measures of complexity for hypotheses spaces
  • Sample complexity, generalization bounds

An indicative reading list is available via


The module is delivered through a combination of classroom-based lectures and self-directed learning.


This module delivery is assessed as below:



Weight (%)



Written examination (2hrs 30mins)




Coursework 2




Coursework 1



In order to pass this Module Delivery, students must:

  • achieve an overall weighted Module mark of at least 50.00%;

AND, when taken as part of MEng Computer Science and MEng Mathematical Computation:

  • achieve a mark of at least 40.00% in any Components of assessment weighed ≥ 30% of the module.

Where a Component comprises multiple Assessment Tasks, the minimum mark applies to the overall component.