Box CEO warns against AI context rot advocates specialized sub-agents

Box CEO Aaron Levie tells Business Insider AI agents suffer from "context rot." The fix is breaking work into specialized sub-agents. (Justin Sullivan/Getty Images)

Key Points:

• AI agents can buckle under memory overload — Box CEO Aaron Levie calls it "context rot"
• "The model will just get very confused and potentially focus on the wrong part of the information," Levie told Business Insider
• Split big jobs into fleets of specialized sub-agents instead of relying on one super-agent, he added

Box CEO Aaron Levie revealed to Business Insider a critical flaw in AI agents: when overloaded with information, they suffer from "context rot," losing focus and producing worse results. This occurs when models are given too much data in their "context window," the phase where they synthesize information before responding.

"You have to be both very precise in your instructions and then you have to give the model an incredible amount of the right context to operate from, but too much context actually, it will do worse with," Levie explained.

The solution? Breaking down tasks among multiple specialized sub-agents rather than relying on a single super-agent. "You'll have multiple agents, all with a set of goals and a set of contexts that are germane to their particular part of the workflow," Levie said, calling this approach "definitely going to be the future of large-scale agent systems."

The Reality of AI Agents

While Silicon Valley races to develop AI agents for complex tasks like auto-pilot sales and engineering work, the technology remains imperfect. Research shows errors compound with each step an agent takes - a 1% error rate per step can balloon to 63% by the 100th step.

Startups like Patronus AI suggest guardrails can help mitigate these issues, noting that small improvements "can yield outsized reductions in error probability." However, Levie's warning about context rot suggests the industry may need to rethink its approach to AI agent architecture entirely.

This perspective challenges the Silicon Valley dream of a single AGI overlord, instead advocating for a more distributed, specialized approach to artificial intelligence systems.