Volltext-Downloads (blau) und Frontdoor-Views (grau)

Quality-aware Algorithm Switching Framework for Adaptive Stream Processing Systems

  • Stream processing is a popular paradigm to process huge amounts of unbounded data, which has gained significant attention in both academia and industry. Typical stream processing applications such as stock trading and network traffic monitoring require continuously analyzed results provided to end-users. During processing, the characteristics of data streams such as volume or velocity can vary, e.g., peak load or bursty streams can occur at certain points. In order to cope with such situations, it requires the analytical systems to be able to adapt the execution of stream processing as quickly as possible. In literature, different approaches adapting data stream processing such as load-shedding and elastic parallelization do exist. However, each of them have their different shortcomings like skewed results (due to the dropped data) or strong limits on the adaptation due to the parallelization overhead. One specific challenge motivating us is to minimize the impact of runtime adaptation on the overall data processing, in particular for real-time data analytics. Moreover, while the need to create adaptive stream processing systems is well known, there is currently no systematic and broad analysis of the solution range of creating adaptation mechanisms for stream processing applications. In this dissertation, we focus on algorithm switching as a fundamental approach to the construction of adaptive stream processing systems. Algorithm switching is a form of adaptation, where stream processing algorithms, with fundamentally similar input-/output-characteristics but different runtime tradeoffs like resource consumption or precision, are replaced to optimize the processing. As our overall goal, we present a general algorithm switching framework that models a wide range of switching solutions (called switch variants) in a systematic and reusable manner as well as characterizes the switch variants with their quality guarantees. Concretely, we focus on developing a general model of algorithm switching to systematically capture possible variants of different switching behavior. We also present a theoretical specification to predict the timeliness-related qualities for the switch variants. Moreover, from the practical perspective, we also develop a component-based design to ease the realization effort of the algorithm switching variants. Finally, we provide a validation of the algorithm switching framework against the realized switch variants.

Download full text files

Export metadata

Additional Services

Share in Twitter    Search Google Scholar    frontdoor_oas
Author:Cui Qin
Referee:Klaus Schmid, Minos Garofalakis
Advisor:Klaus Schmid, Holger Eichelberger
Document Type:Doctoral Thesis
Year of Completion:2022
Publishing Institution:Stiftung Universität Hildesheim
Granting Institution:Universität Hildesheim, Fachbereich IV
Date of final exam:2022/01/13
Release Date:2022/04/13
Tag:adaptation; adaptive systems; algorithm switching; stream processing
Page Number:256
PPN:Link zum Katalog
Institutes:Fachbereich IV
DDC classes:500 Naturwissenschaften und Mathematik / 500 Naturwissenschaften
000 Allgemeines, Informatik, Informationswissenschaft / 000 Allgemeines, Wissenschaft / 005 Computerprogrammierung, Programme, Daten
Licence (German):License LogoCreative Commons - Namensnennung 4.0