Easing the benchmarking of parallel stream processing on multi-cores

• Main Author: Garcia, Adriano Marques
• Publication Date: 2023
• Format: Doctoral thesis
• Language: eng
• Source: Biblioteca Digital de Teses e Dissertações da PUC_RS
• Download full: https://tede2.pucrs.br/tede2/handle/tede/10884
• topic: Stream Processing; Framework; Benchmark; Multi-corre; Processamento de Fluxo; Framework; Benchmark; Multi-núcleo; CIENCIA DA COMPUTACAO::TEORIA DA COMPUTACAO

In today’s fast-changing data-driven world, there is increasing demand for realtime/low-latency data processing. Stream processing is a technique that envisages processing data as it becomes available, enabling near real-time data processing. Stream processing applications must resort to parallelism techniques to speed up processing and to cope with processing large volumes of data. Although there are parallel programming interfaces (PPIs) that add several abstraction layers, parallelism in stream processing is still a difficult task, usually demanding expert knowledge to achieve desired performance levels. This generates a lot of research effort toward boosting parallel stream processing performance and making parallel programming more accessible. Typically, benchmarks are used to evaluate the PPIs and new solutions in this context. However, there are a number of limitations in existing benchmarks, including not addressing some categories of stream processing applications, few or no parameterization options, difficulty extending the benchmarks to other PPIs, lack of appropriate performance metrics, poor usability, only targeting JVM-based languages, and others. This work proposes a framework called SPBench for creating custom benchmarks and evaluating parallel stream processing. Our main goal is to ease the benchmarking process in parallel stream processing, including the creation, building, execution, tuning, and evaluating of the benchmarks. Therefore, this doctoral dissertation provides the following main scientific contributions: (I) A framework that simplifies the benchmarking of stream processing applications, providing an API and a command-line interface to simplify, reuse code, customize, extend, and evaluate diferente aspects or properties regarding parallel stream processing. (II) A parallel C++ benchmark suite for stream processing that includes real-world applications and the most state-of-theart Parallel Programming Interfaces (PPIs) in this context. (III) A comprehensive comparative study of the most popular PPIs leveraging C++ stream parallelism. (IV) Mechanisms for dynamic data stream frequency simulation in stream processing applications with a set of algorithms for generating the literature’s most commonly used data stream frequency patterns and an analysis of the data frequency impact on the performance of stream processing applications. (V) An analysis of the performance impact of micro-batch sizing on stream processing applications, including mechanisms for real-time and dynamic batching management, allowing users to adjust batch sizes on the fly either based on specific size targets or time intervals. We test the SPBench framework with five real-world applications of video/image processing, data compression, and fraud detection. We show the benefits of SPBench by using it in combination with PPIs to generate parallel stream processing benchmarks and conduct various analyses. Overall, the results showed that the high-level abstractions of PPIs can cause significant performance penalties when they hide fine-tuning mechanisms. In the data frequency experiments, the FastFlow PPI benefited more from varying frequency scenarios than the TBB in our test cases. Finally, the experimental results showed that the potential performance advantage of using micro-batches on multi-cores tends to show up only in specific scenarios.