TY - GEN
T1 - Distributed and real-time query framework for processing participatory sensing data streams
AU - Liu, Chi Harold
AU - Zhang, Zhen
AU - Huang, Yue
AU - Leung, Kin K.
N1 - Publisher Copyright:
© 2015 IEEE.
PY - 2015/11/23
Y1 - 2015/11/23
N2 - Detecting emergent events and monitoring civil infrastructures in modern metropolitan cities by participatory sensing have recently been identified as a critical part of the public service management. With fast information distribution, multimedia messages (e.g., sound, images, videos, and texts) collected from citizens' smart devices can provide useful information to infer such emergencies by processing application level queries initialized from end terminals. This requires to establish an efficient, real-time processing systems for participatory sensing that can cope with both the dynamic queries and a variety of information with diverse attributes. To this end, in this paper, we first design a distributed and real-time query framework for event-based stream processing in participatory sensing, including a Storm-based real-time query engine, a messaging queue on Kafka, and a data persistence module based on HBase. Second, a dynamic indexing division method that is aware of the change of query attributes and volume is proposed. Third, we implement an application for civil infrastructure monitoring, and finally we evaluate the performance of proposed framework compared with existing approaches, simulation results of which show its advantages.
AB - Detecting emergent events and monitoring civil infrastructures in modern metropolitan cities by participatory sensing have recently been identified as a critical part of the public service management. With fast information distribution, multimedia messages (e.g., sound, images, videos, and texts) collected from citizens' smart devices can provide useful information to infer such emergencies by processing application level queries initialized from end terminals. This requires to establish an efficient, real-time processing systems for participatory sensing that can cope with both the dynamic queries and a variety of information with diverse attributes. To this end, in this paper, we first design a distributed and real-time query framework for event-based stream processing in participatory sensing, including a Storm-based real-time query engine, a messaging queue on Kafka, and a data persistence module based on HBase. Second, a dynamic indexing division method that is aware of the change of query attributes and volume is proposed. Third, we implement an application for civil infrastructure monitoring, and finally we evaluate the performance of proposed framework compared with existing approaches, simulation results of which show its advantages.
KW - Participatory sensing
KW - Query framework
KW - Stream processing
UR - https://www.scopus.com/pages/publications/84961711461
U2 - 10.1109/HPCC-CSS-ICESS.2015.78
DO - 10.1109/HPCC-CSS-ICESS.2015.78
M3 - Conference contribution
AN - SCOPUS:84961711461
T3 - Proceedings - 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security and 2015 IEEE 12th International Conference on Embedded Software and Systems, HPCC-CSS-ICESS 2015
SP - 248
EP - 253
BT - Proceedings - 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security and 2015 IEEE 12th International Conference on Embedded Software and Systems, HPCC-CSS-ICESS 2015
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 17th IEEE International Conference on High Performance Computing and Communications, IEEE 7th International Symposium on Cyberspace Safety and Security and IEEE 12th International Conference on Embedded Software and Systems, HPCC-ICESS-CSS 2015
Y2 - 24 August 2015 through 26 August 2015
ER -