The research and implementation of a large-scale real-time news recommendation algorithm

With the rapid development of computer and network technology, the Internet provides a fast way to obtain news. Every day, hundreds of millions of news items are reported around the world. Thus, people are faced with the problem of how to find interesting news quickly. This problem is addressed by personalized news recommendation systems. This paper notes that personalized news recommendation systems face four major challenges: real-time, scalability, novelty, and diversity. To address these four challenges, we analyze two existing news recommendation algorithms and their associated advantages and disadvantages. Due to the shortcomings of the existing algorithms, two novel methods are proposed: similar-document-based real-time news recommendation and user-preference-cluster-based real-time news recommendation. The former lacks diversity, and the latter lacks novelty. Then, we propose a novel hybrid method that combines the two algorithms to overcome their shortcomings. The hybrid algorithm satisfies the four characteristics, and we design several of experiments to evaluate whether our algorithm is better than others. Finally, we use large-scale data to test our news recommendation system to verify the feasibility of the proposed algorithm for large-scale data and real sys.