Information Theory National Taiwan University
Information Theory is a senior (undergraduate) level course designed for students who are interested in the quantitative fundamental limits of information. What is information and how to quantify information? What is the ultimate data compression rate and what is the ultimate transmission rate of communication? In this course, we introduce the fascinating theory originated from Claude E. Shannon, which addresses the above fundamental questions in communication theory. We will develop methods and coding techniques to achieve these fundamental limits. Finally, we will also demonstrate the application of information theory to other fields, including statistics (hypothesis testing and estimation) and statistical inferences.
1. Introduce basic topics in information theory, including measures of information, source coding theorem, channel coding theorem, and source-channel separation. 2. Develop methods and coding techniques to achieve these fundamental limits. 3. Show applications of information theory beyond communications, especially in high dimensional statistics and statistical inferences.
Prerequisite: Probability, Linear Algebra, Optional: Random Processes, Communication Systems Homework (30%), Midterm (30%), Final (40%)
Online Course Requirement