EIRES: Efficient Integration of Remote Datain Event Stream Processing

Loading...
Thumbnail Image
File version

Accepted Manuscript (AM)

Author(s)
Zhao, Bo
van der Aa, Han
Nguyen, Thanh Tam
Nguyen, Quoc Viet Hung
Weidlich, Matthias
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
2021
Size
File type(s)
Location

China

License
Abstract

To support reactive and predictive applications, complex event processing (CEP) systems detect patterns in event streams based on predefined queries. To determine the events that constitute a query match, their payload data may need to be assessed together with data from remote sources. Such dependencies are problematic, since waiting for remote data to be fetched interrupts the processing of the stream. Yet, without event selection based on remote data, the query state to maintain may grow exponentially. In either case, the performance of the CEP system degrades drastically. To tackle these issues, we present EIRES, a framework for efficient integration of static data from remote sources in CEP. It employs a cost-model to determine when to fetch certain remote data elements and how long to keep them in a cache for future use. EIRES combines strategies for (i) prefetching that queries remote data based on anticipated use and (ii) lazy evaluation that postpones the event selection based on remote data without interrupting the stream processing. Our experiments indicate that the combination of these strategies improves the latency of query evaluation by up to 3,725x for synthetic data and 47x for real-world data.

Journal Title
Conference Title

Proceedings of the 2021 International Conference on Management of Data

Book Title
Edition
Volume
Issue
Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement

© ACM, 2021. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in SIGMOD/PODS '21: Proceedings of the 2021 International Conference on Management of Data, ISBN: 978-1-4503-8343-1, https://doi.org/10.1145/3448016.3457304

Item Access Status
Note
Access the data
Related item(s)
Subject

Information systems

Persistent link to this record
Citation

Zhao, B; van der Aa, H; Nguyen, TT; Nguyen, QVH; Weidlich, M, EIRES: Efficient Integration of Remote Datain Event Stream Processing, Proceedings of the 2021 International Conference on Management of Data, 2021, pp. 2128–2141