CDBTune ⁺: An efficient deep reinforcement learning-based automatic cloud database tuning system

Configuration tuning is vital to optimize the performance of a database management system (DBMS). It becomes more tedious and urgent for cloud databases (CDB) due to diverse database instances and query workloads, which make the job of a database administrator (DBA) very difficult. Existing solutions for automatic DBMS configuration tuning have several limitations. Firstly, they adopt a pipelined learning model but cannot optimize the overall performance in an end-to-end manner. Secondly, they rely on large-scale high-quality training samples which are hard to obtain. Thirdly, existing approaches cannot recommend reasonable configurations for a large number of knobs to tune whose potential values live in such high-dimensional continuous space. Lastly, in cloud environments, existing approaches can hardly cope with the changes of hardware configurations and workloads, and have poor adaptability. To address these challenges, we design an end-to-end automatic CDB tuning system, CDBTune⁺, using deep reinforcement learning (RL). CDBTune⁺ utilizes the deep deterministic policy gradient method to find the optimal configurations in a high-dimensional continuous space. CDBTune⁺ adopts a trial-and-error strategy to learn knob settings with a limited number of samples to accomplish the initial training, which alleviates the necessity of collecting a massive amount of high-quality samples. CDBTune⁺ adopts the reward-feedback mechanism in RL instead of traditional regression, which enables end-to-end learning and accelerates the convergence speed of our model and improves the efficiency of online tuning. Besides, we propose effective techniques to improve the training and tuning efficiency of CDBTune⁺ for practical usage in a cloud environment. We conducted extensive experiments under 7 different workloads on real cloud databases to evaluate CDBTune⁺. Experimental results showed that CDBTune⁺ adapts well to a new hardware environment or workload, and significantly outperformed the state-of-the-art tuning tools and DBA experts.