Optimal defense and control of dynamic systems modeled as cyber-physical systems

With the increasing connectivity among computational cyber-connected elements and physical entities, a unified representation that captures the interrelationship between the cyber and the physical systems becomes increasingly important. In this paper, we propose a novel representation for developing cybersecurity schemes for physical systems wherein the cyber system states affect the physical system and vice versa. Subsequently by using this representation, an optimal strategy via Q-learning is derived for the cyber defense in the presence of an attack. Since the cyber system under attack will affect the physical system stability and performance, an optimal controller by using Q-learning is considered for the physical system with uncertain dynamics. As an example, cyber-attacks that increase the network delay and packet losses are considered and the goal of the proposed cyber defense and optimal controller is to thwart the attack and mitigate the performance degradation of the physical system due to increased delays and packet losses. An illustrative example is given where the proposed theory is evaluated on the yaw-channel control of an unmanned aerial vehicle. Simulation results show that on the cyber side, both the attacker and the defender gains their greatest payoff whereas on the physical system side, the optimal controller is able to maintain the linear system in a stable manner when the cyber state vector meets a certain desired criterion.