PathFence: Reducing Cross-Path Dependencies in Microservices

Abstract

Maintaining low and consistent response times is crucial for mission critical, user-facing applications (e.g., e-commerce) that rely on microservices architectures. However, through extensive benchmarking of microservice applications in cloud environments, we find that response time stability in microservices is fragile, with delays ranging from milliseconds to seconds, particularly under moderate to high CPU utilization (e.g., 60%). An important cause of this instability is cross-path dependency, where certain component microservices are shared across multiple execution paths triggered by different client requests. A slowdown in one execution path (e.g., due to a transient bottleneck) can indirectly delay other paths, as they all rely on the same shared component microservices. To address this challenge, we propose PathFence, a novel approach that reduces the impact of cross-path dependencies in microservices applications by isolating the workload on shared microservices for different execution paths. By dynamically allocating software resources (e.g., thread and connection pools) and optimizing concurrency level on shared microservices, PathFence significantly improves response time stability. Our evaluation using multiple microservices benchmarks demonstrates that PathFence reduces the 99th percentile response time by up to 80% and minimizes dropped requests by more than 90%.