Two-level Markov switching state-space control processes for power management in cluster systems

Motivated by the power management and resource cooperation of the network communication systems, this paper proposes a two-level Markov switching state-space control processes (MSSSCP) to describe the hierarchical dynamic structure of the network systems. We first describe the bottom-up analytical model. Specifically, this model has two layers, the upper level and the lower level. The evolution of the lower level states only depends on the environment and the lower layer states and the evolution of the upper level states will be driven by the transition of the lower states. Then we prove such a MSSSCP can be characterized as a Markov Decision Process (MDP). Based on the theory of performance potential theory, an online adaptive optimization algorithm that combines potential estimation and policy iteration is presented. Finally, as an illustrative example, the power management in streaming media cluster system is formulated and addressed.