FLeeC: a Fast and Lock-Free Applicational Cache

• Main Author: Costa, André João César
• Publication Date: 2023
• Format: Master thesis
• Language: eng
• Source: Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
• Download full: http://hdl.handle.net/10362/165841
• topic: Caching; Concurrency; Non-Blocking; Lock-Free; Domínio/Área Científica::Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e Informática

Access to data in applications that make use of external storage systems (e.g., Databases) can be a major performance bottleneck. A common solution is to first query a cache application to reduce data access overhead. These cache applications leverage fast main memory access rates and the parallelism capabilities of hardware to provide high performance. To this end, a cache application needs a concurrency control mechanism to maintain correctness under conflicting concurrent accesses, of which mutual-exclusion locks (blocking concurrency control) are the most common mechanism used. Blocking concurrency control strategies fail to provide a high level of performance when under medium to high contention, as the pessimistic nature of blocking concurrency can not completely avoid needlessly synchronizing operations that do not conflict. Conversely, non-blocking (or lock-free) concurrency control strategies can provide high degrees of performance in parallel shared memory contexts, especially in high contention scenarios. This work proposes FLeeC, an application-level cache system based on Memcached, which leverages non-blocking re-designed data structures concurrency to improve performance by allowing any number of concurrent writes and reads to its main data structure. FLeeC features a hash table with embedded eviction policy to allowthe correct functioning of its non-blocking algorithms; a memory reclamation scheme adapted to cache semantics; a non-blocking hash table expansion mechanism to preserve the strong progress guarantees that non-blocking concurrency provides; and dedicated replacement algorithm to minimize false misses that can occur in non-mutually exclusive environments. We have extensively evaluated FLeeC under varied scenarios and workloads and found that, when compared to Memcached, it is capable of achieving 6× higher performance when under high contention scenarios, 1.2× higher performance when under low contention scenarios and equivalent performance in scenarios with no contention. FLeeC can thus be used to further improve the performance of any application that currently uses Memcached