Multi-resource Attack-Defense Strategy Optimization in the Blotto Game Based on Pool-PPO

Effective command of heterogeneous combat systems, comprising attack, reconnaissance, and electronic-warfare units, is critical to determining the outcome of modern conflicts, the core of which can be abstracted as a complex dynamic resource allocation problem. This paper models the problem as an extended, multi-stage Colonel Blotto game with multi-resources. To address the high-dimensional state space and the difficulty of solving for a mixed-strategy equilibrium, we propose Pool-PPO: an alternating self-play framework with a bounded historical opponent pool that mitigates non-stationarity and promotes mixed-strategy learning. Each outer iteration consists of two phases: (A) update the attacker against a defender snapshot sampled from the pool; (B) update the defender against the current attacker. The pool is maintained as a FIFO queue, periodically appending new defender snapshots and discarding the oldest. Experiments show that, relative to a symmetric PPO baseline, Pool-PPO yields a significantly higher expected payoff for the attacker while maintaining higher policy entropy, producing more randomized, less exploitable behavior that better approaches mixed-strategy Nash solutions. Overall, constructing and leveraging a historical opponent distribution within self-play offers an effective pathway to solving complex dynamic adversarial problems and obtaining robust, advantageous strategies.