Information Theory

Information theory is the study of how to store and transmit data in the most effective way. Some of the big questions that Information Theory tries to answer are:

1. How much Information Exists?
2. What is the smallest amount of data that can accurately describe the set?
3. How much information can be sent or stored given available resources?

I’ve broken some major results of information theory into the following sections:

1. Basic Probability
2. Shannon Entropy
3. Properties of Entropy
4. Sequences of Random Variables
5. Compression
6. Channel Communications
7. Quantum Error Correction