COMP0089 Advanced Deep Learning and Reinforcement Learning

This database contains the 2018-19 versions of syllabuses.

Note: Whilst every effort is made to keep the syllabus and assessment records correct, the precise details must be checked with the lecturer(s).

Academic session

2018-19

Module

Advanced Deep Learning and Reinforcement Learning

Code

COMP0089

Module delivery

1819/A7U/T2/COMP0089 Masters (MEng)

Related deliveries

1819/A7P/T2/COMP0089 Postgraduate

Prior deliveries

COMPMI22

Level

Masters (MEng)

FHEQ Level

L7

FHEQ credits

15

Term/s

Term 2

Module leader

Graepel, Thore

Contributors

Graepel, Thore

van Hasselt, Hado

Borsa, Diana

Hessel, Matteo

Module administrator

Ball, Louisa

Aims

The module aims to introduce students to the foundations of deep learning, reinforcement learning, and deep reinforcement learning, and to equip students with the ability to successfully implement, apply and test relevant learning algorithms in TensorFlow.

Learning outcomes

On successful completion of the module, a student will be able to:

  1. understand the basics of deep learning and reinforcement learning paradigms;
  2. understand architectures and optimization methods for deep neural network training;
  3. understand how to implement deep learning methods within TensorFlow and apply them to data;
  4. understand the theoretical foundations and algorithms of reinforcement learning;
  5. understand how to apply reinforcement learning algorithms to environments with complex dynamics.

Availability and prerequisites

This module delivery is available for selection on the below-listed programmes. The relevant programme structure will specify whether the module is core, optional, or elective.

In order to be eligible to select this module as optional or elective, where available, students must meet all prerequisite conditions to the satisfaction of the module leader. Places for students taking the module as optional or elective are limited and will be allocated according to the department’s module selection policy.

Programmes on which available:

  • MEng Computer Science (International Programme) (year 4)
  • MEng Computer Science (year 4)
  • MEng Mathematical Computation (International Programme) (year 4)
  • MEng Mathematical Computation (year 4)

Prerequisites:

To be eligible to select this module, students must have:

  • a strong understanding of probability, calculus, and linear algebra; and
  • excellent coding skills in Python (in order to complete assessments);

And, must have taken in Term 1:

Content

The module has two interleaved parts that converge. One part is on machine learning with deep neural networks, the other part is about prediction and control using reinforcement learning. The two strands come together when we discuss deep reinforcement learning, where deep neural networks are trained as function approximators in a reinforcement learning setting.

The deep learning stream of the course will cover a short introduction to neural networks and supervised learning with TensorFlow, followed by lectures on convolutional neural networks, recurrent neural networks, end-to-end and energy-based learning, optimization methods, unsupervised learning as well as attention and memory. Possible applications areas to be discussed include object recognition and natural language processing.

The reinforcement learning stream will cover Markov decision processes, planning by dynamic programming, model-free prediction and control, value function approximation, policy gradient methods, integration of learning and planning, and the exploration/exploitation dilemma. Possible applications to be discussed include learning to play classic board games as well as video games.

An indicative reading list is available via http://readinglists.ucl.ac.uk/departments/comps_eng.html.

Delivery

The module is delivered through a combination of lectures, self-directed learning, and coursework, assignments, which will focus on the practical implementation of deep neural network training and reinforcement learning algorithms and architectures in Tensorflow.

Assessment

This module delivery is assessed as below:

#

Title

Weight (%)

Notes

1

Deep Learning Coursework 1

12.5

 

2

Deep Learning Coursework 2

12.5

 

3

Deep Learning Coursework 3

25

 

4

Reinforcement Learning Coursework 1

12.5

 

5

Reinforcement Learning Coursework 2

12.5

 

6

Reinforcement Learning Coursework 3

25

 

In order to pass this module delivery, students must:

  • achieve an overall weighted module mark of at least 50%; and
  • achieve a mark of at least 40% in any components of assessment weighed ≥ 30% of the module.

Where a component comprises multiple assessments, the minimum mark applies to the overall component.