TY - GEN
T1 - Highly Parallel SPARQL Engine for RDF
AU - Feng, Fan
AU - Zhou, Weikang
AU - Zhang, Ding
AU - Pang, Jinhui
N1 - Publisher Copyright:
© 2020, The Author(s).
PY - 2020
Y1 - 2020
N2 - In this paper, a highly parallel batch processing engine is designed for SPARQL queries. Machine learning algorithms were applied to make time predictions of queries and reasonably group them, and further make reasonable estimates of the memory footprint of the queries to arrange the order of each group of queries. Finally, the query is processed in parallel by introducing pthreads. Based on the above three points, a spall time prediction algorithm was proposed, including data processing, to better deal with batch SPARQL queries, and the introduction of pthread can make our query processing faster. Since data processing was added to query time prediction, the method can be implemented in any set of data-queries. Experiments show that the engine can optimize time and maximize the use of memory when processing batch SPARQL queries.
AB - In this paper, a highly parallel batch processing engine is designed for SPARQL queries. Machine learning algorithms were applied to make time predictions of queries and reasonably group them, and further make reasonable estimates of the memory footprint of the queries to arrange the order of each group of queries. Finally, the query is processed in parallel by introducing pthreads. Based on the above three points, a spall time prediction algorithm was proposed, including data processing, to better deal with batch SPARQL queries, and the introduction of pthread can make our query processing faster. Since data processing was added to query time prediction, the method can be implemented in any set of data-queries. Experiments show that the engine can optimize time and maximize the use of memory when processing batch SPARQL queries.
KW - Multithreading
KW - Performance prediction
KW - Pthread
KW - SPARQL
UR - http://www.scopus.com/inward/record.url?scp=85090041965&partnerID=8YFLogxK
U2 - 10.1007/978-981-15-7981-3_5
DO - 10.1007/978-981-15-7981-3_5
M3 - Conference contribution
AN - SCOPUS:85090041965
SN - 9789811579806
T3 - Communications in Computer and Information Science
SP - 61
EP - 71
BT - Data Science - 6th International Conference of Pioneering Computer Scientists, Engineers and Educators, ICPCSEE 2020, Proceedings
A2 - Zeng, Jianchao
A2 - Jing, Weipeng
A2 - Song, Xianhua
A2 - Lu, Zeguang
PB - Springer
T2 - 6th International Conference of Pioneering Computer Scientists, Engineers and Educators, ICPCSEE 2020
Y2 - 18 September 2020 through 21 September 2020
ER -