Information Theory and Rate Distortion Theory for Communications and Compression

This book is very specifically targeted to problems in communications and compression by providing the fundamental principles and results in information theory and rate distortion theory for these applications and presenting methods that have proved and will prove useful in analyzing and designing real systems. The chapters contain treatments of entropy, mutual information, lossless source coding, channel capacity, and rate distortion theory; however, it is the selection, ordering, and presentation of the topics within these broad categories that is unique to this concise book. While the coverage of some standard topics is shortened or eliminated, the standard, but important, topics of the chain rules for entropy and mutual information, relative entropy, the data processing inequality, and the Markov chain condition receive a full treatment. Similarly, lossless source coding techniques presented include the Lempel-Ziv-Welch coding method. The material on rate Distortion theory and exploring fundamental limits on lossy source coding covers the often-neglected Shannon lower bound and the Shannon backward channel condition, rate distortion theory for sources with memory, and the extremely practical topic of rate distortion functions for composite sources.

Jerry Gibson is Professor of Electrical and Computer Engineering at the University of California, Santa Barbara. He has been an Associate Editor of the IEEE Transactions on Communications and the IEEE Transactions on Information Theory. He was President of the IEEE Information Theory Society in 1996, and he has served on the Board of Governors of the IT Society and the Communications Society. He was an IEEE Communications Society Distinguished Lecturer for 2007-2008. He is an IEEE Fellow, and he has received The Fredrick Emmons Terman Award (1990), the 1993 IEEE Signal Processing Society Senior Paper Award, the 2009 IEEE Technical Committee on Wireless Communications Recognition Award, and the 2010 Best Paper Award from the IEEE Transactions on Multimedia. He is co-author of the books Digital Compression for Multimedia (Morgan-Kaufmann, 1998) and Introduction to Nonparametric Detection with Applications (Academic Press, 1975 and IEEE Press, 1995) and author of the textbook, Principles of Digital and Analog Communications (Prentice-Hall, second ed., 1993). He is Editor-in-Chief of The Mobile Communications Handbook (CRC Press, 3rd ed., 2012), Editor-in-Chief of The Communications Handbook (CRC Press, 2nd ed., 2002), and Editor of the book, Multimedia Communications: Directions and Innovations (Academic Press, 2000).