Efficient Skyline query processing of massive data based on Map-Reduce

Recently, Skyline query has been a research hot of Database and Information Retrieval. In addition, the amount of data for collecting and using by human is developing at an astonishing speed. Therefore, how to process Skyline query of massive data is an urgent problem. Map-Reduce is a new parallel programming model that processes vast number of data on large clusters with easy deployment. As a parallel programming model, Map-Reduce is suit for solving Skyline query of massive data. This paper resolves the problem of processing Skyline query of massive data on Map-Reduce framework. A straightforward implementation of Skyline query on Map-Reduce needs to scan all the candidate results before obtaining the final results. However, when the amount of final results is much smaller than the original data, there is a waste of processing unnecessary results on Map-Reduce framework. Consequently, in this paper, a series of efficient Skyline query algorithms and optimization have been proposed to prune the unpromising results effectively and enhance the performance of processing Skyline query of massive data on Map-Reduce. Our extensive experiments are built on top of Hadoop platform, an open-source implementation of Map-Reduce framework. The experiment results demonstrate that our algorithms have high efficiency, accuracy and scalability.