Roy Derks (IBM): Effectively use AI Agents to Maintain Your Docs

This talk was presented at the AI The Docs 2025 online conference. We are thrilled to share the recording and the summary with you. 

Visit the talk summary page to see all of the presentations from the conference.

Presentation

Roy Derks (Principal Product Manager at IBM)

Summary

How can a large company like IBM efficiently maintain its extensive and diverse documentation? What approaches can be taken to build AI agents for documentation, and what are their pitfalls? 

In his talk, Roy Derks (Principal Product Manager at IBM) explores IBM’s strategy to automate documentation maintenance with agents. 

• The challenge: Maintaining IBM’s vast documentation required tedious manual work across teams, with limited CMS support for APIs or versioning.
• Tools as the bottleneck: Agents are only as effective as the tools they access. Poor tooling led to low accuracy, high costs, and fragile setups in early iterations.
• Iterations:
  • Single agent: Low accuracy, error prone, and hard to maintain.
  • Multi-agent (microservice style): Increased complexity and token costs, with accuracy issues in agent handoffs.
• Breakthrough: Model Context Protocol (MCP) separated tool logic from agentic logic, reduced duplication, and enabled reusable, testable MCP servers. This provided flexibility, easier debugging, and access to a broader ecosystem of tools.
• Outcome: IBM’s MCP-based solution significantly optimized documentation workflows, simplifying integration with GitHub and other sources while cutting maintenance overhead. 

As Roy Derks highlights, building agent systems is iterative; success depends less on the LLM itself and more on robust, well-designed tools and standards like MCP.