Interval-Aware Caching for Scaling Apache Druid at Netflix

Apache Druid has been a cornerstone of Netflix's data infrastructure, enabling the ingestion of millions of events per second and querying trillions of rows. This system provides the realtime insights necessary to ensure a high-quality user experience. However, as the scale of Netflix's data operations has grown, so too have the challenges associated with managing repetitive query loads. This article delves into the development and implementation of an experimental caching layer-referred to as Interval-Aware Caching-that addresses these challenges and optimizes query performance.

The Problem of Repetitive Query Loads

Netflix's internal dashboards play a critical role in realtime monitoring, particularly during high-profile events such as live shows or global launches. These dashboards often contain multiple charts, each triggering several Druid queries. For instance, one widely used dashboard with 26 charts generates 64 queries per load. When dozens of engineers access the same dashboard for monitoring purposes, the query volume can escalate to unmanageable levels. A single dashboard, refreshing every 10 seconds and viewed by 30 users, could generate up to 192 queries per second.

This heavy load poses a significant challenge, as the system must maintain capacity not only for these repetitive dashboard queries but also for other critical operations such as automated alerting, canary analysis, and ad hoc queries. Additionally, the rolling time-window nature of these dashboards-where each refresh adjusts the time range slightly-further complicates caching efforts. These dynamic shifts lead to frequent cache misses, as even minor changes in query parameters result in unique queries that bypass Druid's built-in caching mechanisms.

Limitations of Existing Druid Caching Mechanisms

Druid's built-in caching mechanisms, such as the full-result cache and per-segment cache, are highly effective for many use cases. However, they fall short in scenarios involving rolling-window dashboards. The full-result cache, for example, fails when the time window shifts slightly, as this results in a unique query that doesn't match any cached results. Furthermore, Druid deliberately avoids caching results that involve realtime segments, as these are subject to constant updates and changes.

The per-segment cache also struggles with rolling-window queries. While it can cache intermediate results for specific segments of data, the continuous overlap and shifting of time windows mean that many queries cannot fully leverage cached data. This inefficiency leads to increased query processing times and greater strain on Druid's resources, making it challenging to scale the system effectively.

Concept and Design of Interval-Aware Caching

To address these limitations, Netflix's engineering team developed an experimental Interval-Aware Caching system. This caching layer is designed to account for the unique characteristics of rolling-window queries, where time intervals shift incrementally with each refresh. By focusing on the specific intervals of data that remain consistent across queries, the system can identify opportunities for cache reuse even when query parameters differ slightly.

The core idea behind Interval-Aware Caching is to break down queries into smaller, interval-based components. These components can then be individually cached and reused across multiple queries, even if the overall query parameters have changed. For example, if a dashboard requests data for the past three hours and refreshes every 10 seconds, only the most recent interval of data needs to be queried anew. The remaining intervals can be served directly from the cache, significantly reducing the computational burden on Druid.

Implementation Challenges and Tradeoffs

Implementing Interval-Aware Caching required careful consideration of several tradeoffs. One of the primary challenges was balancing the complexity of the caching logic against the performance benefits. The system needed to efficiently identify reusable intervals without introducing excessive overhead or latency. Additionally, the caching layer had to integrate seamlessly with Druid's existing architecture, ensuring compatibility with its query processing pipeline.

Another challenge was determining the appropriate cache eviction strategy. Given the dynamic nature of rolling-window queries, cached intervals could quickly become outdated, necessitating frequent updates. The engineering team developed a custom eviction policy that prioritizes the retention of recently accessed intervals while discarding older data that is less likely to be reused. This approach helps maintain a high cache hit rate without overloading the system.

Performance Improvements and Future Directions

The introduction of Interval-Aware Caching has yielded significant performance improvements for Netflix's Druid-based systems. By reducing the volume of repetitive queries, the caching layer has alleviated strain on Druid's resources, enabling the system to scale more effectively. Internal dashboards now load faster, providing engineers with near-instant access to critical metrics and insights during high-stakes events.

Looking ahead, the engineering team plans to refine the caching algorithms further, exploring opportunities to extend the system's capabilities. Potential enhancements include support for more complex query patterns, improved cache management strategies, and broader integration with other components of Netflix's data infrastructure. These efforts aim to ensure that the system can continue to meet the demands of an ever-growing data scale and maintain its role as a key enabler of Netflix's operational excellence.

Conclusion

Interval-Aware Caching represents a targeted solution to the unique challenges posed by repetitive query loads in Apache Druid. By focusing on the specific requirements of rolling-window dashboards, this experimental caching layer has transformed Netflix's ability to scale its realtime monitoring systems. While implementation required navigating complex tradeoffs, the resulting performance gains underscore the value of innovative engineering approaches in addressing large-scale data challenges.