Scaling ArchUnit with Nebula ArchRules at Netflix

Netflix employs a polyrepo strategy with tens of thousands of Java repositories, requiring a scalable and efficient approach to share common build logic. By leveraging the Nebula suite of Gradle plugins, Netflix ensures reliable dependency management, artifact publication, and feedback mechanisms to detect technical debt. This article explores how Netflix scaled ArchUnit to improve its Java library lifecycle management.

Netflix's Polyrepo Strategy for Java Development

The polyrepo strategy at Netflix involves maintaining a vast number of independent repositories for Java projects. This approach allows teams to operate autonomously while minimizing cross-dependencies. However, it creates challenges in sharing common build logic and enforcing consistent standards across projects. The Java Platform team addresses these challenges by developing and maintaining tools like the Nebula suite of Gradle plugins to provide standardized solutions.

These tools facilitate tasks such as keeping dependencies up-to-date, publishing artifacts, and providing developers with build-time feedback. This feedback is critical when developers deviate from predefined guidelines or introduce potential technical debt. Such measures ensure a high level of quality across Netflix's software ecosystem.

Addressing Java Library Lifecycle Management

An incident involving a backwards-incompatible library change highlighted the need for improved lifecycle management of Java libraries at Netflix. Despite adhering to best practices like deprecating unused code, library authors often lack clarity on when it is safe to remove or refactor deprecated elements. This issue is further compounded during large-scale migrations, such as upgrading major frameworks like Spring Boot.

To resolve this, Netflix introduced a set of API lifecycle annotations. These annotations include:

• Deprecated: Standard Java annotation for indicating obsolete APIs.
• Public: A custom annotation for APIs intended for downstream use.
• Experimental: A custom annotation for APIs under development that may not yet be stable.

By clearly marking APIs, developers gain better visibility into intended usage patterns, reducing confusion and ensuring better lifecycle management.

Utilizing ArchUnit for Architectural Code Enforcement

ArchUnit, an open-source library with over 35,000 stars on GitHub and contributions from 84 developers, enables teams to enforce architectural code rules. Netflix integrated ArchUnit into its development processes to identify and address technical debt in both internal and shared libraries. Its usage as part of a JUnit testing suite ensures that architectural violations are detected early during the development cycle.

This proactive approach not only minimizes the risk of introducing breaking changes but also promotes adherence to architectural guidelines. By leveraging ArchUnit, Netflix empowers library authors to assess downstream API usage and make informed decisions about code refactoring or deprecation.

The Role of Nebula ArchRules in Scaling ArchUnit

To scale the application of ArchUnit across its extensive polyrepo environment, Netflix developed Nebula ArchRules. This tool extends ArchUnit by providing reusable architectural rule templates tailored to Netflix's unique requirements. Teams can utilize these pre-defined rules to enforce consistent standards across all Java repositories.

By integrating Nebula ArchRules with the existing Nebula Gradle plugins, Netflix streamlined the process of adopting architectural rules across teams. This integration ensures that developers receive immediate feedback on rule violations, reducing the accumulation of technical debt and maintaining code quality.

Improving Collaboration Among Teams

One of the primary objectives of scaling ArchUnit with Nebula ArchRules was to improve collaboration among teams delivering shared libraries. By providing a standardized framework for defining and enforcing architectural rules, Netflix enables teams to align their development practices without sacrificing autonomy.

This shared foundation reduces friction during cross-team collaborations and ensures that all libraries conform to organizational standards. Additionally, the visibility into API usage facilitated by ArchUnit and Nebula ArchRules fosters greater accountability among library authors and consumers.

Outcomes and Future Plans

The adoption of ArchUnit and Nebula ArchRules has resulted in significant improvements in code quality and reduced the frequency of incidents related to library changes. Developers now have access to better tooling for managing technical debt and adhering to lifecycle annotations, ensuring a smoother development process.

Looking ahead, Netflix aims to continue refining its tools and practices to further enhance its Java ecosystem. Future developments may include additional features for Nebula ArchRules and expanded support for other programming languages used within the organization.