Stochastic automata games

A class of learning stochastic automata can be defined by the sextuple {S, F, Φ, g, π→, T} where S is the input set, F is the finite set of r outputs (or strategies) of the automaton, Φ is the finite set of r states, g is the output function which is a one-to-one mapping between the states and the outputs, π→ is the state probability vector whose ith component is the probability of the ith state being chosen. T is the reinforcement operator which guides the automaton in its learning by specifying the manner in which π→ is to be changed in response to the environment. The environment is specified by the penalty structure; namely, its response in the form of penalties to the outputs of the automaton. The reinforcement scheme enables the automaton to choose its outputs in such a manner as to reduce the mean penalty. This paper considers the collective behavior of the above finite state stochastic automata. This is of interest in view of the possibility of modelling group behavior of subjects in terms of these automata. The natural language for considering the collective behavior is that of game theory. After a brief introduction to a class of deterministic automata, the stochastic automaton is formulated and a nonlinear reinforcement specified. The finite state stochastic automaton is first considered in a game with nature, and conditions under which the automaton's winnings reach the von Neumann value of the game are established. Next, two stochastic automata with arbitrary number of states for each are considered in a game, the game matrix being specified. Performance of the automata for various conditions on the elements of the game matrix is considered. In a comparison of performance with deterministic automata, it is established that for performance comparable to that of the finite state stochastic automaton, the deterministic automaton needs an infinite number of states. Finally some games are simulated on a computer which verifies the general analysis and further throws light on the details of the game.