M arkov Chains and M arkov Processes

Abstract In many applications, the usual assumption of independent and identically distributed random variables has to be abandoned. Markov dependence is a first possibility. We deal with Markov chains in discrete and continuous time. We then extend the setting to Markov processes on more general state spaces. We pay special attention to stationary processes and to piecewise deterministic Markov processes where the behavior of the process between jumps is basically deterministic.