INF 723: Proseminar in Information Science (Info. Theory)
Spring 2008 Syllabus
Instructor Information
 |
Sanjay Goel Office: BA 310b (Uptown Campus) Hours: M 11:30AM-1PM & by appt. PH: (518) 442-4925 FX (518) 442-2568 Email: [email protected] |
CLASS INFORMATION
| Time: | M 9:00-11:00am |
| Room: | Draper 313 |
| Dates: | April 7 - May 12 |
| Credit(s): | 3 |
| Call #: | 9325 |
Text & Reference Books
See Resources section for required reading.COURSE OVERVIEW
In the first two lectures of this part of the class, we will learn how to measure information content; how to compress data and; how to communicate reliably across noisy channels. This includes the source coding and noisy channel coding theorems proposed by Claude Shannon.
In the next three lectures, we will learn the basics of decision analysis and techniques to make decisions under uncertainty. Decision analysis typically involves use of decision trees where the nodes of the tree represent events, arcs connecting the nodes represent dependencies between events, and uncertainties are represented via probabilities of events. In this class, students will solve problems using decision trees. Bayesian Probability Theory will also be discussed and multistage decision problems using Bayesian analysis will be covered. Utility functions that model the decision maker's attitudes and multi-attribute utility functions to model trade-offs among different decisions will be covered if time permits.
Supplemental readings for the class are primarily provided to not only ease the understanding of the material, but also help you reflect on the writers' thought process. Students are expected to be able to discuss the content of the readings during the class specified.
Learning Objectives
Students will be able to:- Understand the differences between information theory and coding theory
- Learn the algorithms for measuring information content in data
- Able to use theories proposed by Claude Shannon for error correction in communication channels
ASSESSMENT & GRADING
The assessment and grading is based on the papers that were assigned in the overall syllabus.Course Schedule
| Date | Topics | Instructor |
|---|---|---|
| 4/7 | Coding Theory I | Stephen F. Bush (Guest Speaker) |
| 4/14 | Coding Theory II | Sanjay Goel |
| 4/21 | HOLIDAY | |
| 4/28 | Bayesian Decision Theory I | Sanjay Goel |
| 5/5 | Bayesian Decision Theory II | Sanjay Goel |
| 5/12 | Bayesian Decision Theory III | Sanjay Goel |
COURSE DETAILS
April 7 & 14, 2008
Title: Coding Theory I & II
Details: In this lecture, we address the problem of reliable communication through a noisy channel including error-correcting strategies of repetition and hamming code as well as Shannon's Noisy Channel Coding theorem. We also discuss definitions of entropy, conditional entropy, and mutual information. Zero Error Information Theory and Graph Entropy will also be defined. A brief introduction to quantum information theory and classical information theory and their differences will also be presented. In addition, the lecture will cover Network Coding, Huffman Coding, and Arithmetic Coding. Finally, application to video compression will be presented.
The second lecture will be a continuation of the previous lecture where we look at compression algorithms including block codes, uniquely decodable codes, optimum code lengths and Huffman codes. Kolmogorov Complexity as well as estimation techniques for complexity will also be covered.
- The original classic paper by Shannon: The Bell System Technical Journal, Vol. 27, pp. 379-423, 623-656, July, October 1948, http://plan9.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf
- Lecture 3 slides from Dr. Stephen F. Bush from GE Global Research
- Dr. Mackay's class notes from MIT
April 28, 2008
Title: Bayesian Decision Theory I
Details: In this lecture, we will discuss the basics of probability theory and solve decision analysis problems using decision tables. Students learn both deterministic and probabilistic approaches for decision analysis as well as value of perfect information.
Readings: Tversky, A and Kahneman, D., The Framing of Decisions and the Psychology of Choice, Science, New Series, Vol. 211, No. 4481. (Jan. 1981), pp. 453-458.
May 5, 2008
Title: Bayesian Decision Theory II
Topics: In this class, we review Bayesian Probability Theory and problems in multistage decision analysis. Students will also learn the use of a decision tree tool for solving complex problems. Simple decision trees will also be covered in the class.
Readings: Howard, R. A., Heathens, Heretics, and Cults: The religious Spectrum of Decision Making, Interfaces 22:6, Nov.-Dec., 1992, pp. 15-27.
May 12, 2008
Title: Bayesian Decision Theory III
Topics: In this lecture, we look at the basics of utility theory and its application to decision analysis. Students will also understand the issues related to information security risk analysis.
Readings: Pat�-Cornell, M. E., Dillon, R. L., The Respective Roles of Risk and Decision Analyses in Decision Support, Decision Analysis, Vol. 3, No. 4, December 2006, pp. 220-232
Download syllabus: inf723syllabus.pdf