Information Theory for Data Science

Information theory deals with mathematical laws that govern the flow, representation and transmission of information, just as the field of physics concerns laws that govern the behavior of the physical universe. The foundation was made in the context of communication while characterizing the fundamental limits of communication and offering codes (sometimes called algorithms) to achieve them.

The most significant achievement of the field is the invention of digital communication which forms the basis of our daily-life digital products such as smart phones, laptops and any IoT devices. Recently it has also found important roles in a spotlight field that has been revolutionized during the past decades: data science.

This book aims at demonstrating modern roles of information theory in a widening array of data science applications. The first and second parts of the book covers the core concepts of information theory: basic concepts on several key notions; and celebrated source and channel coding theorems which concern the fundamental limits of communication. The last part focuses on applications that arise in data science, including social networks, ranking, and machine learning.

The book is written as a text for senior undergraduate and graduate students working on Information Theory and Communications, and it should also prove to be a valuable reference for professionals and engineers from these fields.