Former DeepSeeker and team unveil RAGEN for training more reliable AI agents

April 23, 2025 — A collaborative team from Northwestern University, Microsoft, Stanford, and the University of Washington, including former DeepSeek researcher Zihan Wang, has developed RAGEN, a new method for training and evaluating AI agents. The system aims to address the brittleness of current AI agents, making them more reliable for real-world applications.

The Challenge: AI Agents Stuck in "Corporate Purgatory"

Despite 2025 being dubbed the "year of AI agents," most implementations remain experimental, according to a VentureBeat poll. RAGEN tackles this by focusing on multi-turn, interactive settings where agents must adapt, remember, and reason under uncertainty.

How RAGEN Works

Built on the StarPO (State-Thinking-Actions-Reward Policy Optimization) framework, RAGEN emphasizes learning through experience rather than memorization. Key features include:

• Rollout stage: LLMs generate complete interaction sequences guided by reasoning.
• Update stage: Models are optimized using normalized cumulative rewards.

The team tested RAGEN using Alibaba’s Qwen models (1.5 and 2.5), chosen for their open weights and instruction-following capabilities.

The "Echo Trap" Problem

Wang highlighted a core issue in a widely shared X thread: RL systems often reward shortcuts, leading to repetitive behaviors and degraded performance. This "Echo Trap" manifests as reward variance cliffs and disappearing reasoning traces.

Testing Environments

RAGEN evaluates agents across three symbolic tasks:

1. Bandit: Single-turn, stochastic risk-reward reasoning.
2. Sokoban: Multi-turn, deterministic puzzle-solving.
3. Frozen Lake: Stochastic, multi-turn adaptive planning.

Stabilizing Training with StarPO-S

To combat training collapse, the team introduced StarPO-S, which incorporates:

1. Uncertainty-based rollout filtering.