AI Coding Assistants Slow Down Experienced Developers Study Finds

A recent study examining the impact of AI tools on experienced open-source developers has revealed surprising results. The research, conducted by METR, found that 75% of participants actually worked slower when using AI coding assistants, with only 25% showing improved performance. The full paper is available here.

Key Findings:

• 16 experienced developers participated, each working on about 15 issues (half with AI, half without)
• 56% had never used Cursor before the study
• Developers overestimated AI's benefits by 20-24%, even after experiencing slowdowns
• The one developer with 50+ hours of Cursor experience did show productivity gains

Methodology

The study employed a rigorous approach:

1. Random assignment of AI/no-AI conditions per task
2. Screen recordings of all work sessions
3. $150/hour compensation for participants
4. Focus on real-world open-source projects (average 22k+ stars, 1M+ LOC)

Why the Slowdown?

Researchers identified several potential factors:

• Steep learning curve for AI tools
• Time spent reviewing and correcting AI output
• Context switching between AI interaction and coding
• Potential skill atrophy when not using AI

Industry Reactions

Commenters on Hacker News debated the implications:

• "AI is great for boilerplate code but falls short on complex tasks"
• "The perception of productivity may differ from actual results"
• "This mirrors the 'Google-fu' learning curve from early search days"

Looking Ahead

The study authors note this is just one data point and call for more research, particularly on:

• Long-term effects of AI tool usage
• Impact on code quality and maintenance
• Differences between novice and expert developers

For more details, see the full research paper.