AI coding tools slow developers despite perceived speed boost

A recent study by Model Evaluation & Threat Research (METR) reveals that AI coding tools actually slow down software developers, contrary to widespread expectations of increased efficiency. The research, published in their report, found that while developers predicted a 24% speedup, the tools increased task completion time by 19%.

Key Findings

• Developers overestimated AI's benefits: Even after the study, participants believed AI had made them 20% faster, despite the opposite being true.
• Hallucinations and cleanup: Developers spent significant time reviewing and correcting AI-generated code, accepting less than 44% of suggestions.
• Context matters: AI performed poorly in large, complex repositories (1M+ lines of code) and struggled with implicit repository context.

Methodology

The study involved 16 experienced developers working on 246 real-world issues (bug fixes, new features) in open-source projects. Tasks were randomly assigned with or without AI tools (primarily Cursor Pro with Claude 3.5/3.7 Sonnet). Work was conducted between February and June 2025.

Why AI Slowed Developers Down

1. Over-optimism: Unrealistic expectations about AI's capabilities.
2. High developer familiarity: Experienced coders gained little from AI assistance.
3. Large repositories: AI struggled with scale and complexity.
4. Low reliability: Time spent fixing AI errors offset potential gains.
5. Context gaps: AI lacked understanding of project-specific nuances.

Broader Implications

This aligns with other research, including:

• A Qodo study showing AI-generated code requires extra validation.
• An Intel report finding AI PCs reduce productivity.
• Call center workers noting AI creates more work despite accelerating some tasks.