AuEx: Automatic Exploration for Fresh Creative Ads via Large-Scale Deep Reinforcement Learning

With the extensive growth of short-video sharing platforms (SVSPs), it has become one of the most important platforms for Internet advertising. As more and more advertisers join SVSPs, the volume of fresh creative ads is also increasing dramatically. These fresh creative ads can not only affect the current revenue, but also play an important role in the accurate placement of subsequent history-rich ads. Through the analysis of real-world industrial data, we for the first time deeply discuss the current situation of fresh creative ads on SVSPs, as well as potential issues. With the unique features of ads on SVSPs, we propose a novel framework to enable fresh creative ads Automatic Exploration, namely AuEx, which is based on large-scale deep reinforcement learning (DRL). Via environmental feedbacks, the DRL-based AuEx can simultaneously optimize the revenue, user experience and pure exploration, rather than simply pursuing improvements on the one side of the road. AuEx has been deployed in a real-world SVSP environment, satisfying the online stability requirements and request time constraints. Compared with two online running exploration frameworks, experimental results have demonstrated that AuEx has simultaneously achieved significant improvements in terms of revenue, user experience, and pure exploration.