Robust Entropy Rate for Uncertain Sources: Applications to Communication and Control Systems

In this paper the notion of robust entropy and subsequently, robust entropy rate for a family of discrete time uncertain sources is introduced. When the uncertainty is described by a relative entropy constraint between the set of uncertain source densities and a given nominal source density, the solution to this robust notion of information is presented and its connection with other notions of entropy definitions, such as, Renyi entropy and Tsallis entropy is presented. Then, the robust entropy rate is calculated for 1) Uncertain sources corresponding to a partially observed Gauss Markov process, 2) Sources with uncertain frequency response, and 3) Uncertain sources corresponding to a partially observed controlled Gauss Markov Process. Finally, an application of the robust entropy rate in networked control systems is presented by defining necessary conditions for uniform asymptotic stabilizability and observability. I. I NTRODUCTION The entropy and entropy rate are information theoretic measures. They have applications in physics, probability and statistics, communication theory and economics. The importance of entropy in communication theory was first introduced by Shannon in terms of Shannon first coding theorem. Then, the application of entropy rate in joint source channel coding theorem, the AEP and etc. is shown (1). The objective of this paper is to extend the notion of entropy and subsequently entropy rate to the case when there is uncertainty in the source. The robust entropy is defined as the maximum of the Shannon entropy over a family of sources belonging to an uncertainty set. The explicit solution to the robust entropy is presented when the uncertainty is described by a constraint on the relative entropy between the set of uncertain source densities and the corresponding nominal source density. Subsequently, the connection between this solution with other entropies is shown. Then, for different families of uncertain source densities, the robust entropy rate is calculated and an application of the robust entropy rate in stabilizability and observability of networked control systems is presented. This paper is organized as follows. In Section II, the robust entropy and the robust entropy rate are defined. The solution to the robust entropy and its connection to other kinds of entropy are presented. In Section III, for different families of uncertain sources, the robust entropy rate is calculated. Finally in Section IV, an application of robust entropy rate in stabilizability and observability of networked control system is presented.