2005•Unpublished venueRequires access

Entropy, Relative Entropy, and Mutual Information

Thomas M. Cover, Joy A. Thomas

Abstract

This chapter contains sections titled: Entropy Joint Entropy and Conditional Entropy Relative Entropy and Mutual Information Relationship Between Entropy and Mutual Information Chain Rules for Entropy, Relative Entropy, and Mutual Information Jensen's Inequality and Its Consequences Log Sum Inequality and Its Applications Data-Processing Inequality Sufficient Statistics Fano's Inequality Summary Problems Historical Notes

About this research paper

What this paper is about

Why it matters

OpenAlex reports 125 citations for this work. Citation counts describe recorded attention and do not establish research quality.

Key contribution

A contribution statement is not available in the OpenAlex record.

Method / approach

Method details are not available in the OpenAlex metadata.

Main findings

Findings are not separately available in the OpenAlex metadata.

Limitations

Limitations are not available in the OpenAlex metadata.

Applications

Application details are not available in the OpenAlex metadata.

Available abstract

Key concepts: Joint entropy, Generalized relative entropy, Joint quantum entropy, Mathematics, Mutual information, Conditional entropy, Quantum relative entropy, Entropy (arrow of time)