Stochastic Processes Problems and Solutions

1 Markov Chains.- 1: Definition.- 2: Transition and absolute probabilities.- 3: Determination of the higher transition probabilities.- 4: Classification of states.- 5: The limit of the higher transition probabilities.- 6: Classification of Markov chains.- 7: The limiting distributions of irreducible Markov chains.- Problems.- 8: Markov chains with continuous state space.- Problems.- 9: Stationary stochastic sequences.- Problems.- 2 Markov Processes.- 1: Introduction.- 2: Definition.- 3: Poisson process.- 4: Markov process with a finite or denumerably infinite number of states.- 5: Markov process with continuous transition.- 6: Mixed Markov processes.- Problems.- 3 Non-Markovian Processes.- 1: Recurrent processes.- 2: Stationary stochastic processes.- 3: Secondary stochastic processes generated by a stochastic process.- Problems.- 4 Solutions of Problems.- 1: Markov chains.- 2: Markov processes.- 3: Non-Markovian processes.- References.