Sakana AI's ALE-Agent Achieves Top 2% in AtCoder Heuristic Contest

Sakana AI has developed ALE-Agent, an AI system specialized in solving hard optimization problems, which recently achieved 21st place out of over 1,000 human participants in a live AtCoder Heuristic Contest (AHC). This milestone marks a significant advancement in AI's ability to tackle NP-hard problems with real-world applications, such as logistics and factory planning.

The Challenge of Hard Optimization Problems

Combinatorial optimization problems, like the Traveling Salesperson Problem, are computationally intensive and often lack exact solutions. Human experts spend weeks refining algorithms to approximate optimal solutions. Sakana AI's ALE-Bench benchmark, developed in collaboration with AtCoder, evaluates AI systems on 40 such problems from past AHCs. Unlike traditional coding benchmarks, ALE-Bench focuses on long-horizon reasoning and iterative refinement, mirroring real-world problem-solving.

ALE-Agent's Breakthrough Performance

ALE-Agent, built on Gemini 2.5 Pro, combines domain knowledge with inference-time scaling to generate diverse solutions. In the May 2025 AHC047 contest, it:

• Ranked 21st, outperforming 98% of human competitors.
• Improved solutions 100x faster than humans, leveraging rapid code revisions.
• Applied advanced techniques like Poisson approximation and simulated annealing to boost scores.

Key Insights and Limitations

• Strengths: ALE-Agent excels in short-duration contests and problems suited for simulated annealing.
• Weaknesses: Struggles with longer contests and experimental algorithm design.
• Future Work: Enhancing feedback mechanisms and integrating human expert tools could further improve performance.

Conclusion

Sakana AI's work with ALE-Bench and ALE-Agent demonstrates AI's growing capability in algorithm engineering. The team plans to refine the agent for broader applications, potentially revolutionizing industries reliant on optimization. For details, read the technical paper or explore the ALE-Bench dataset.

Interested in joining Sakana AI? Check their careers page.