Predictable Low-Latency Event Detection With Parallel Complex Event Processing
- 27 January 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 2 (4), 274-286
- https://doi.org/10.1109/jiot.2015.2397316
Abstract
The tremendous number of sensors and smart objects being deployed in the Internet of Things (IoT) pose the potential for IT systems to detect and react to live-situations. For using this hidden potential, complex event processing (CEP) systems offer means to efficiently detect event patterns (complex events) in the sensor streams and therefore, help in realizing a “distributed intelligence” in the IoT. With the increasing number of data sources and the increasing volume at which data is produced, parallelization of event detection is crucial to limit the time events need to be buffered before they actually can be processed. In this paper, we propose a pattern-sensitive partitioning model for data streams that is capable of achieving a high degree of parallelism in detecting event patterns, which formerly could only consistently be detected in a sequential manner or at a low parallelization degree. Moreover, we propose methods to dynamically adapt the parallelization degree to limit the buffering imposed on event detection in the presence of dynamic changes to the workload. Extensive evaluations of the system behavior show that the proposed partitioning model allows for a high degree of parallelism and that the proposed adaptation methods are able to meet a buffering limit for event detection under high and dynamic workloads.Keywords
Funding Information
- German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) (RO 1086/19-1)
This publication has 39 references indexed in Scilit:
- A catalog of stream processing optimizationsACM Computing Surveys, 2014
- SliderPublished by Association for Computing Machinery (ACM) ,2014
- Elastic Scaling for Data Stream ProcessingIEEE Transactions on Parallel and Distributed Systems, 2013
- Integrating scale out and fault tolerance in stream processing using operator state managementPublished by Association for Computing Machinery (ACM) ,2013
- Auto-parallelizing stateful distributed streaming applicationsPublished by Association for Computing Machinery (ACM) ,2012
- Accurate latency estimation in a distributed event processing systemPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- S4: Distributed Stream Computing PlatformPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- Nephele/PACTsPublished by Association for Computing Machinery (ACM) ,2010
- DryadPublished by Association for Computing Machinery (ACM) ,2007
- Performance Modeling and Evaluation of Distributed Component-Based Systems Using Queueing Petri NetsIEEE Transactions on Software Engineering, 2006