AI Explain and AI Help were experimental tools that let MDN readers ask a large language model to clarify code snippets or answer documentation questions.

Development timeline

MDN started exploring generative AI in early 2023 to address the difficulty newcomers have finding and understanding code examples. The team built two prototypes—AI Explain for inline code and AI Help for conversational Q&A—using OpenAI's GPT‑3.5 API.

• January 2023: initial research and API integration.
• March 2023: prototype of AI Explain and AI Help released internally.
• June 28 2023: public launch of AI Explain behind an MDN Plus login.
• July 2023: rapid usage monitoring and feedback collection.
• August 2023: decision to pause AI Explain for further refinement.

Testing & validation process

The feature was evaluated with a mix of automated scripts and manual review to gauge answer quality across thousands of code examples.

• Ran 25,820 unique examples through gpt‑3.5‑turbo‑0301 and recorded explanations.
• Compared outputs to GPT‑4 as a higher‑quality benchmark.
• Classified results as accurate, partially accurate, or incorrect.
• Manually inspected a random sample of low‑scoring responses.
• Collected user thumbs‑up/down votes (3.34% participation) to surface real‑world sentiment.

Launch & early adoption

During its 65‑hour beta, AI Explain attracted over 24 000 unique visitors and generated more than 44 000 explanations, showing clear interest from the community.

• Average session length increased by 12% when AI Explain was used.
• 44 229 explanations were produced, with 68.9% receiving positive votes.
• Feedback highlighted both helpful clarifications and occasional inaccuracies.
• Data was logged to the AI‑ready deployment guide for future analysis.
• Insights fed into internal demos and partner briefings.

Reasons for pausing the feature

The pause was a deliberate response to community input and observed quality gaps. Maintaining trust in MDN’s reference material requires explanations that are consistently reliable.

• Non‑deterministic LLM output made automated testing fragile.
• Edge‑case technologies (e.g., emerging web APIs) often received vague answers.
• Positive feedback was offset by a notable share of inaccurate or misleading explanations.
• Resource constraints limited rapid iteration on model tuning.
• Commitment to deliver a polished experience before broader rollout.

Future roadmap

MDN plans to resume work on AI Explain in 2024 with a focus on higher answer fidelity and tighter integration with documentation sources.

• Explore newer GPT‑4 or open‑source alternatives for better reasoning.
• Implement a stricter validation pipeline using the OpenAI‑Broadcom accelerator study as a reference for scaling tests.
• Introduce a sandbox mode that flags uncertain explanations for review.
• Expand community feedback loops with targeted surveys.
• Plan a phased re‑launch tied to MDN Plus subscription upgrades.