CIOs must navigate AI agent hype and vendor promises carefully

AI agents are dominating tech industry conversations, from vendors to enterprise CIOs and boards of directors. However, the gap between talk and measurable value remains wide. Vendors are aggressively marketing the latest AI innovations, leaving IT leaders to grapple with varying definitions of AI agents, technical immaturity, and agent washing—a term describing the rebranding of existing tools as AI agents.

The Confusion Around AI Agents

Matt Kropp, managing director and senior partner at Boston Consulting Group, highlights the confusion: "We had all the hype around generative AI, and then you had all these software companies that have to have something new to say, and so they say, ‘Well, now we have agents.’"

Tim Sanders, VP of research insights at G2, defines agents as "something that can perform some type of action using persistent memory and decision intelligence." He emphasizes that none are fully autonomous, despite vendor claims. Enterprises can categorize agents on a spectrum from weak (e.g., chatbots) to strong (e.g., systems of agents working together).

The Reality Gap

While businesses are excited about AI agents, the technology is still in its early stages. Kropp notes that while "all the software vendors... are building agents into their tools now, they haven’t released much." Gartner estimates that only about 130 of the thousands of agentic AI vendors are "real."

Gartner also predicts that over 40% of agentic AI projects will fail by 2027 due to escalating costs, unclear business value, and risks.

Validating Vendor Claims

CIOs must scrutinize vendor claims to avoid wasted resources. Sanders warns: "Vendors can make things work really well in the lab, and then it gets into the real world and is messy."

Eden Zoller, chief analyst of applied AI at Omdia, explains that evaluating agentic AI is more complex than traditional AI due to "emergent behaviors" from dynamic interactions. Enterprises should question vendors about:

• Security and identity access controls
• User interface changes
• Agent interoperability

The Road Ahead

Despite skepticism, analysts agree that AI agents will eventually have a "massive impact." Kropp advises: "Don’t mistake that for thinking agents are not going to have a massive impact."

For more insights, read Gartner’s predictions on agentic AI failures and Omdia’s analysis on mitigating risks.

Disclosure: Informa, which owns a controlling stake in Informa TechTarget, the publisher behind CIO Dive, is also invested in Omdia. Informa has no influence over CIO Dive’s coverage.