Applied Coding and Information Theory

This course introduces the concepts of coding and information theory with a focus on applications.

Starting from the original work of Claude Shannon, we will introduce the concept of entropy and discuss how to quantify information. We will then study source coding, discuss the tradeoffs between theoretical guarantees and practical implementations, and present the 2014 breakthrough that lead, after 30 years of stagnation, to an explosion of new data compression standards used by Apple, Facebook and Google. We will then present how to convey information over a noisy channel, discuss practical and theoretical tradeoffs, and study the most important classes of error-correcting codes used in modern digital applications such as distributed storage systems on the cloud.

Covered topics will be taken from the following list and depend on the interests of the students:

1. Introduction to entropy and information

2. Data compression

  • Shannon source coding theorem
  • Huffman coding
  • Dictionary techniques
  • Arithmetic coding
  • Asymmetric numerical systems and modern standards (e.g. Apple, Facebook and Google)

3. Information theory, gambling and portfolio theory

4. Communication over a noisy channel

  • Communication channel and capacity
  • Channel coding theorem
  • Error-correcting codes
  • Real channels

5. Modern error-correcting codes and applications

  • Message passing algorithms
  • Convolutional and turbo codes
  • low-density parity-check codes
  • Rateless codes and streaming
  • Polar codes

6. Codes and information theory for distributed storage systems
 

Learning Outcomes: 
  • Formulate the fundamental concepts of information theory
  • Understand the principles of source and channel coding
  • Understand the theoretical and practical tradeoffs between the difference classes of codes
     
Type: 
Course
Semester: 
S2018
ECTS: 
5
Lecturer: 
Site: 
N
Code: 
32089
62089
Language: 
english
Period: 
weekly
Schedule: 
Friday: 8:15 - 12:00
Location: 
UniNE, Unimail
Room: 
B104
Evaluation type: 
oral exam
Comment: 

First Lecture
The first lecture will take place on Friday, 23.02.2018 at 08:15 in UniNE, Unimail, room B104.

ILIAS
The course page in ILIAS can be found at https://ilias.unibe.ch/goto.php?target=crs_1166773&client_id=ilias3_unibe.
 


Organizer
The organizer of this teaching unit and its evaluation is the Faculty of Science of the University of Neuchâtel. For more details, please visit this web site.