Skip to main content

Unit information: Information Theory 3 in 2021/22

Unit name Information Theory 3
Unit code MATH34600
Credit points 10
Level of study H/6
Teaching block(s) Teaching Block 1A (weeks 1 - 6)
Unit director Dr. Jaggi
Open unit status Not open

Year 2 Theoretical Physics

MATH10013 Probability and Statistics OR Year 2 Physics

MATH11400 Statistics 1 is helpful, but not necessary



School/department School of Mathematics
Faculty Faculty of Science

Description including Unit Aims

Unit Aims

To give a rigorous and modern introduction into Shannon's theory of information, with emphasis on fundamental concepts and mathematical techniques.

Unit Description

Shannon's information theory underlies many aspects of modern life, including streaming an MP3 or movie, or taking and storing digital photos. It is one of the great intellectual achievements of the 20th century, which continues to inspire communications engineering and to generate challenging mathematical problems. Recently it has extended dramatically into physics as quantum information theory. The course is about the fundamental ideas of this theory: data compression and reliable communication over noisy channels.

It is a statistical theory, so notions of probability play a great role, and in particular laws of large numbers as well as the concept of entropy are fundamental, culminating in Shannon's coding theorems. The course aims at demonstrating information theoretical modelling, and the mathematical techniques required will be rigorously developed.

Relation to Other Units

It is a natural companion to the Quantum Information course offered in Mathematics (MATHM5610), and to a certain degree to Cryptography B (COMSM0007), offered in Computer Science, and Communications (EENG 22000), in Electrical Engineering. It may also be interesting to physicists having attended Statistical Physics (PHYS30300).

Intended Learning Outcomes

This unit should enable students to:

  • understand how information problems are modeled and solved;
  • model and solve problems of information theory: data compression and channel coding;
  • discuss basic concepts such as entropy, mutual information, relative entropy, capacity;
  • use information theoretical methods to tackle information theoretical problems, in particular probabilistic method and information calculus.

Transferable skills:

Mathematical - Knowledge of basic information theory; probabilistic reasoning.

General skills - Modelling, problem solving and logical analysis Assimilation and use of complex and novel ideas

Teaching Information

The unit will be taught through a combination of

  • synchronous online and, if subsequently possible, face-to-face lectures
  • asynchronous online materials, including narrated presentations and worked examples
  • guided asynchronous independent activities such as problem sheets and/or other exercises
  • synchronous weekly group problem/example classes, workshops and/or tutorials
  • synchronous weekly group tutorials
  • synchronous weekly office hours

Assessment Information

80% Timed, open-book examination 20% Group projects

Raw scores on the examinations will be determined according to the marking scheme written on the examination paper. The marking scheme, indicating the maximum score per question, is a guide to the relative weighting of the questions. Raw scores are moderated as described in the Undergraduate Handbook.

If you fail this unit and are required to resit, reassessment is by a written examination in the August/September Resit and Supplementary exam period.


If this unit has a Resource List, you will normally find a link to it in the Blackboard area for the unit. Sometimes there will be a separate link for each weekly topic.

If you are unable to access a list through Blackboard, you can also find it via the Resource Lists homepage. Search for the list by the unit name or code (e.g. MATH34600).

How much time the unit requires
Each credit equates to 10 hours of total student input. For example a 20 credit unit will take you 200 hours of study to complete. Your total learning time is made up of contact time, directed learning tasks, independent learning and assessment activity.

See the Faculty workload statement relating to this unit for more information.

The Board of Examiners will consider all cases where students have failed or not completed the assessments required for credit. The Board considers each student's outcomes across all the units which contribute to each year's programme of study. If you have self-certificated your absence from an assessment, you will normally be required to complete it the next time it runs (this is usually in the next assessment period).
The Board of Examiners will take into account any extenuating circumstances and operates within the Regulations and Code of Practice for Taught Programmes.