Major AI security flaws exposed in red teaming competition

A groundbreaking red teaming study has uncovered alarming security weaknesses in today's most advanced AI agents. Between March 8 and April 6, 2025, nearly 2,000 participants launched 1.8 million attacks against 22 AI models from leading labs including OpenAI, Anthropic, and Google Deepmind.

Universal Vulnerabilities Exposed

The competition, organized by Gray Swan AI and hosted by the UK AI Security Institute, revealed that:

• 100% of tested models failed at least one security test
• Attackers achieved 12.7% average success rate
• Over 62,000 successful attacks resulted in policy violations

Attack Methods and Results

Researchers targeted four key behavior categories:

1. Confidentiality breaches
2. Conflicting objectives
3. Prohibited information
4. Prohibited actions

Indirect prompt injections proved particularly effective, working 27.1% of the time compared to just 5.7% for direct attacks. These attacks hide malicious instructions in websites, PDFs, or emails.

Model Performance

While Anthropic's Claude models demonstrated the most robust security, even they weren't immune:

• Claude 3.5 Haiku showed surprising resilience
• Claude 3.7 Sonnet (tested before Claude 4's release) still had vulnerabilities
• Attack techniques often transferred between models with minimal modification

Common Attack Strategies

Successful methods included:

• System prompt overrides using tags like '<system>'
• Simulated internal reasoning ('faux reasoning')
• Fake session resets
• Parallel universe commands