Fundamentals of Information Theory
Grunderna inom informationsteori

EIT145F, 7.5 credits

Valid from: Spring 2016
Decided by: Professor Thomas Johansson
Date of establishment: 2016-11-02

General Information

Division: Electrical and Information Technology
Course type: Third-cycle course
Teaching language: English

Aim

Information theory studies the fundamental limits of information transmission and storage. The course is divided into two parts. The aim of Part I is to introduce the fundamental concepts of information theory, focusing on classical point-to-point transmission scenarios. This part is based on selected chapters of the book by Cover/Thomas, which has been widely used as graduate level entry into information theory. In Part II the perspective is extended to multi-user communication scenarios, following selected parts of the book by El Gamal/Kim, which provides an excellent overview of current state-of-art in the field. Using these two books makes the course both up-to-date and self-contained so that no prior background in information theory is required.

Goals

Knowledge and Understanding

For a passing grade the doctoral student must

• be well familiar with standard information theoretic concepts
• able to explain the proofs of the source coding theorem and the channel coding theorem
• understand how to generalize the concepts and proofs to multi-user scenarios

Competences and Skills

For a passing grade the doctoral student must

• be able to apply information theoretic methods in order to derive fundamental limits for different communication scenarios
• have acquired sufficient knowledge within the field to be able discuss information theory problems with established researchers

Judgement and Approach

For a passing grade the doctoral student must

• have demonstrated the ability to independently extract the important content of an extensive document, such as a textbook
• have demonstrated the ability to reflect the material of the course and to understand possibilities and limitations of information theoretic bounds

Course Contents

Part I: entropy, mutual information and their properties, typical sequences, data compression, source coding theorem, channel capacity, channel coding theorem, continuous random variables, differential entropy, Gaussian channels Part II: multiple access channels, broadcast channels, interference channels, channels with state: writing on dirty paper, distributed compression: Slepian-Wolf theorem, Gaussian vector channels, wireless fading channels, relay channels

Course Literature

• Thomas M. Cover, Joy A. Thomas: Elements of Information Theory. Wiley, 2006.
• Gamal, A. & Kim, Y.: Network Information Theory. Cambridge University Press, 2011. ISBN 9781107008731.

Instruction Details

Types of instruction: Lectures, exercises. The aim of the exercise classes is to discuss the home assignments. The doctoral students should present and discuss ways to solve the problems.

Examination Details

Examination formats: Oral exam, written assignments. Attendance of at least 80% of the lectures and exercise lessons are a requirement to pass the course.
Grading scale: Failed, pass
Examiner:

Admission Details

Assumed prior knowledge: Solid skills in engineering mathematics, including probability theory, linear algebra and calculus.

Further Information

Course Coordinator: Michael Lentmaier

Course Occasion Information

Contact and Other Information

Course coordinator: Michael Lentmaier <michael.lentmaier@eit.lth.se>