Finding best counterstrategies for generalized Iterated Prisoner's Dilemma games

A class of games with two players, who base their actions on the results of the previous round, is considered. These games are generalizations of the Iterated Prisoner's Dilemma. The best counterstrategy for player 1, given a strategy for player 2, is found by treating these games as Markov processes. Several transitions between such best strategies are found and shown to be akin to first-order phase transitions. The roles of a number of special strategies are elucidated. It is shown that there is a strategy for player 2 that never loses, even if this player does not know what kind of game is being played.