Agents Inside the Content Management System: What We Measured
Over the summer of 2026 we ran three public benchmarks of AI agents doing real content work — editing, moving, retiring and cross-linking pages — through RIFT CMS's Model Context Protocol (MCP) interface. We compared that against agents working on raw files and against two widely used headless CMS products. The pattern held across every model and every comparison: the system the agent works inside determines the outcome more than the model does.
The question we set out to answer
Regulated organisations will not hand an AI agent the keys to a public website on the strength of a demo. They need to know what happens on the hundredth edit, not the first: whether links break, whether the site stays consistent, what it costs, and what the agent does when it is asked for something it should not do. So we measured exactly that, published the method, and published the misses alongside the wins.
Raw files versus a governed system
The Drift Race gave two models — Claude Haiku 4.5 and Gemini 3.7 Flash — the same thirty operations on a documentation site, once working directly on files and once through RIFT.
| Measure | Haiku, raw files | Haiku, RIFT | Gemini, raw files | Gemini, RIFT |
|---|---|---|---|---|
| Broken references at the end | 19 | 0 | 0 | 0 |
| Reader exposure to broken links | 724 | 4 | 69 | 4 |
| Tokens consumed | 14.1M | 8.5M | 75.0M | 46.0M |
| Operations completed | 27 of 30 | 29 of 30 | 30 of 30 | 29 of 30 |
Both models used 39% fewer tokens working through RIFT. Not everything went our way. We predicted the governed system would produce fewer style inconsistencies. For one model it produced more, because we had not given the agent the component it needed and it improvised one page at a time. We published that result unchanged. It told us where to build next.
Against Payload and Sanity
We then ran eighteen everyday content tasks against two established headless CMS products, on the same thirty-page site.
- RIFT completed 18 of 18 tasks under both models tested. Payload completed 10 of 18 under one model and 14 of 18 under the other. Sanity was stable on 14 of 18.
- Cost for the full task set: RIFT $2.76–$3.05. Payload $6.66–$6.77. Sanity $4.15 against RIFT's $3.05 on the same model.
- Part of that gap is overhead: each Sanity tool call carried 12,581 tokens of schema; each RIFT call carried 2,889.
The most important single finding concerned refusal. Asked to permanently delete a page, RIFT's agent declined in one turn, using under 5,000 tokens, and named the approval path instead. Against Sanity, the same request consumed 242,194 tokens of trial and error before the agent established that nothing stood in its way. Payload deleted the page.
What this means for leaders
- Governance is cheap when it is built into the tool, and expensive when it is a policy document. Saying no cost almost nothing; discovering there was no “no” cost a fortune.
- The system is a capability equaliser. Smaller, cheaper models produced the same structural outcomes as larger ones once the system carried the structure for them.
- Publish the misses. A vendor benchmark that reports only wins is marketing. Ours reports the prediction we got wrong, and the gap it exposed has since been closed.
The benchmarks found two gaps in RIFT itself: agents had no way to retire a page, and no access to version history or rollback. Both are now part of the MCP surface, and both follow the same rule as everything else an agent does in RIFT: an agent proposes, a named person approves, and nothing reaches the live site without that approval.