Limit behaviour for imprecise Markov Chains

An imprecise Markov chain is defined by a closed convex set of transition matrices instead of a unique one for a classical precise Markov chain. These imprecise Markov chains allow us to model situations where we do not have enough information to specify a unique transition matrix, or to approximate the behaviour of non‐stationary Markov chains. We show that there are efficient, dynamic programming‐ like ways to work and reason with these imprecise Markov chains; e.g. to calculate the resulting distribution over the states at any time instant. We prove that this distribution converges in time, similarly to the precise case and under very mild conditions. We thus effectively prove a Perron‐Frobenius theorem for a special class of non‐linear systems.