Cloudflare's Internal AI Engineering Stack: Key Insights
Cloudflare has successfully integrated AI technology into its engineering workflows, achieving significant adoption across its R&D organization. This effort involved building MCP servers, AI Gateways, and tooling for seamless integration with Cloudflare's platform. The initiative has led to measurable improvements in developer productivity and organizational efficiency.
Overview of the AI Engineering Stack
The AI engineering stack at Cloudflare comprises multiple layers, each targeting specific organizational needs. The engineer-facing tools layer includes OpenCode, Windsurf, and other MCP-compatible clients that support both open-source and third-party coding assistants. These tools enhance development workflows by integrating coding aids directly into the engineering environment.
Central to this stack is the MCP servers, which act as the backbone for AI-powered functionalities. These servers facilitate authentication, communication with Large Language Models (LLMs), and other critical operations. Furthermore, the stack incorporates AI Gateways to manage token processing, request routing, and cost tracking at scale.
Key Metrics and Adoption Rates
Cloudflare's R&D organization has achieved a remarkable adoption rate of 93% for its AI coding tools. Over the past 30 days, the company recorded 3,683 internal users actively using these tools, accounting for 60% of the entire workforce. The tools have processed 51.83 billion tokens on Workers AI and routed 24.137 billion tokens through AI Gateway.
Developer velocity has seen a noticeable improvement, with the average number of weekly merge requests increasing from 5,600 to over 8,700. Specific weeks, such as March 23, observed a peak of 10,952 merge requests, nearly doubling the Q4 baseline.
The Role of the iMARS Tiger Team
To spearhead this initiative, Cloudflare formed the iMARS team (Internal MCP AgentServer Rollout Squad). This cross-functional team collaborated with the Dev Productivity group to design and implement the necessary infrastructure. Their efforts spanned 11 months and included rethinking standards, code reviews, onboarding processes, and repository management to accommodate AI-driven workflows.
The team's work extended beyond the initial MCP servers. They focused on creating scalable solutions that could support the evolving needs of the organization while maintaining the core principles of security, efficiency, and usability.
Security and Data Retention Considerations
Security was a fundamental aspect of the stacks design. The system employs Zero Trust authentication to ensure secure access. Additionally, Cloudflare implemented Zero Data Retention controls, ensuring no sensitive data is stored unnecessarily. These measures align with industry best practices and enhance trust in the AI-driven tools.
The AI Gateway also features cost-tracking mechanisms, enabling efficient resource allocation. This ensures that the infrastructure is both secure and cost-effective, addressing critical organizational concerns.
Impact on Developer Productivity
The integration of AI tools has had a transformative impact on developer productivity at Cloudflare. By automating routine tasks and providing intelligent coding assistance, these tools have empowered engineers to focus on complex problem-solving and innovation.
The increased volume of merge requests is a testament to the effectiveness of the AI engineering stack. As adoption grows, the company continues to refine its tools and processes, ensuring they meet the evolving needs of its teams.
Future Directions and Enhancements
Cloudflare plans to further enhance its AI stack, focusing on deeper integration with existing tools and workflows. The company aims to expand the capabilities of its AI Gateway and MCP servers to support more complex use cases and improve scalability.
Future efforts will also include optimizing the user experience of the engineer-facing tools layer, ensuring that these tools remain intuitive and effective. By continuously iterating on its AI infrastructure, Cloudflare is well-positioned to maintain its leadership in the field of AI-driven engineering.