COMPGI15 - Information Retrieval & Data Mining

This database contains 2016-17 versions of the syllabuses. For current versions please see here.

Note: Whilst every effort is made to keep the syllabus and assessment records correct, the precise details must be checked with the lecturer(s).


CodeCOMPGI15 (Also taught as: COMPM052)
PrerequisitesNormally offered only to students in computer science related programmes because basic programming skills are required. Basic understanding of probability and statistics and proficient in java programming, as demonstrated by a least one programing project in the past.
Taught ByJun Wang (50%), Emine Yilmaz (50%)
AimsThe course is aimed at an entry level study of information retrieval and data mining techniques. It is about how to find relevant information and subsequently extract meaningful patterns out of it. While the basic theories and mathematical models of information retrieval and data mining are covered, the course is primarily focused on practical algorithms of textual document indexing, relevance ranking, web usage mining, text analytics, as well as their performance evaluations. Practical retrieval and data mining applications such as web search engines, personalisation and recommender systems, business intelligence, and fraud detection will also be covered.
Learning OutcomesStudents are expected to master both the theoretical and practical aspects of information retrieval and data mining. At the end of the course student are expected to understand 1. The common algorithms and techniques for information retrieval (document indexing and retrieval, query processing, etc). 2. The quantitative evaluation methods for the IR systems and data mining techniques. 3. The popular probabilistic retrieval methods and ranking principles. 4. The techniques and algorithms existing in practical retrieval and data mining systems such as those in web search engines and recommender systems, including the recently popular topic of deep learning. 5. The challenges and existing techniques for the emerging topics of MapReduce, portfolio retrieval and deep learning.


Overview of the fields
Study some basic concepts of information retrieval and data mining, such as the concept of relevance, association rules, and knowledge discovery. Understand the conceptual models of an information retrieval and knowledge discovery system.
Introduce various indexing techniques for textual information items, such as inverted indices, tokenization, stemming and stop words.

Retrieval Methods
Study popular retrieval models: 1 Boolean, 2. Vector space, 3 Binary independence, 4 Language modelling. Probability ranking principle. Other commonly-used techniques include relevance feedback, pseudo relevance feedback, and query expansion.
Evaluation of Retrieval Performance


Average precision, NDCG, etc. "Cranfield paradigm" and TREC conferences.
Personalisation and Usage Mining
Study basic techniques for collaborative filtering and recommender systems, such as the memory-based approaches, probabilistic latent semantic analysis (PLSA), personalized web search through click-through data.

Data Mining
Study basic techniques, algorithms, and systems of data mining and analytics, including frequentpattern and correlation and association analysis, anomaly detection, and click-through modelling.
Emerging Areas
MapReduce and Sparck; Learning to Rank; Portfolio retrieval and Risk Management; Deep Learning

Method of Instruction:

Lecture presentations, Practical exercises


The course has the following assessment components:


  • Written Examination (2.5 hours, 60%)
  • Coursework (40%)


To pass this course, students must:


  • Obtain an overall pass mark of 50% for all sections combined.


Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Sch├╝tze, Cambridge University Press. 2008. 

Introduction to Data Mining, Pang-Ning Tan, Michael Steinbach and Vipin Kumar, Addison-Wesley, 2006

Gigabytes (2nd Ed.) Ian H. Witten, Alistair Moffat and Timothy C. Bell. (1999), Morgan Kaufmann, San Francisco,


Pattern Recognition and Machine Learning, Christopher M. Bishop, Springer (2006).