資訊理論與壓縮編碼的應用
Information Theory and Data Compression Practices
| 節 | 週二 | 週五 |
|---|---|---|
2 09:00–09:50 | 資訊理論與壓縮編碼的應用 ED301（光復） | |
5 13:20–14:10 | 資訊理論與壓縮編碼的應用 ED301（光復） 2 節連堂 | |
6 14:20–15:10 |
* 根據陽明交大上課時間表所列
Big data analysis has been changing our daily lives significantly. However, in practice, it is possible that several megabytes of data may contain only a few bytes of useful information. Therefore, it is essential to understand the "nature" of information, and how to derive a mathematical model such that the amount of information in a data set can be quantified for analysis. This course is based on the classical information theory proposed by C. E. Shannon, with special focus on its application to data compression. Practical algorithms for different types of data compression will be presented. Finally, two case studies on data compression, one for deep-learning neural network models and the other one for audio data, will be investigated.
Linear Algebra, Probability
無備註
The E3 website will be used to host all class materials.
Performance evaluation is based on some programming assignments, a midterm exam, and a final exam. You can use either C/C++ or Python for the assignments.
1. Introduction to information theory
The history of information theory and the concept of channel and coding.
- 講授：
- 3
2. Information measures
Defining information measure from a probability-based viewpoint, including entropy, relative entropy and mutual Information.
- 講授：
- 3
3. Data compression and Huffman code.
Information theory of data compression and practical algorithms for Huffman coding.
- 講授：
- 6
4. Entropy Rates
Introduction to AEP and the entropy rate of a data source.
- 講授：
- 3
5. Kolmogorov data complexity
Defining information measure from an algorithmic viewpoint.
- 講授：
- 3
6. Arithmetic coding and dictionary-based techniques
More on practical data compression algorithms.
- 講授：
- 6
7. Context-based compression
Introduction to data compression using context information.
- 講授：
- 3
8. Quantization and rate-distortion theory
Introduction to the theory of lossy data compression.
- 講授：
- 4
9. Transform-domain and sub-band coding
Introduction to data representation in transform domain and its application for data compression.
- 講授：
- 5
10. Case Study I: Deep-learning neural network model coding
Introduction to neural network model compressed.
- 講授：
- 3
11. Case Study II: Audio coding
Introduction to practical algorithms for speech and audio data compression.
- 講授：
- 3
| 週次 | 主題 |
|---|---|
| 第 1 週 | 2025-02-18(二),2025-02-21(五) |
| 第 2 週 | 2025-02-25(二),2025-02-28(五) |
| 第 3 週 | 2025-03-04(二),2025-03-07(五) |
| 第 4 週 | 2025-03-11(二),2025-03-14(五) |
| 第 5 週 | 2025-03-18(二),2025-03-21(五) |
| 第 6 週 | 2025-03-25(二),2025-03-28(五) |
| 第 7 週 | 2025-04-01(二),2025-04-04(五) |
| 第 8 週 | 2025-04-08(二),2025-04-11(五) |
| 第 9 週 | 2025-04-15(二),2025-04-18(五) |
| 第 10 週 | 2025-04-22(二),2025-04-25(五) |
| 第 11 週 | 2025-04-29(二),2025-05-02(五) |
| 第 12 週 | 2025-05-06(二),2025-05-09(五) |
| 第 13 週 | 2025-05-13(二),2025-05-16(五) |
| 第 14 週 | 2025-05-20(二),2025-05-23(五) |
| 第 15 週 | 2025-05-27(二),2025-05-30(五) |
| 第 16 週 | 2025-06-03(二),2025-06-06(五) |
| 第 17 週 | 2025-06-10(二),2025-06-13(五) |
| 第 18 週 | 2025-06-17(二),2025-06-20(五) |
1. Thomas M. Cover and Joy A. Thomas, Elements of Information Theory, 2nd Ed., John Wiley & Sons, Inc., 2006. 2. Khalid Sayood, Introduction to Data Compression, Third Edition, Morgan Kaufmann, 2005. Note that both textbooks are available in ebook format in the NCTU university library. They are not the latest editions of the textbooks from the publishers.
- 地點
- EC718
- 時間
- 教師未提供此項資料
- 聯絡方式
- cjtsai@cs.nycu.edu.tw (03) 573-1628 Campus extension number: 31628