Sakana AI's TreeQuest Boosts LLM Performance by 30% with Multi-Model Collaboration

Japanese AI lab Sakana AI has unveiled a groundbreaking technique called Multi-LLM AB-MCTS, which enables multiple large language models (LLMs) to collaborate on complex tasks, outperforming individual models by 30%. This method, detailed in their research paper, leverages Monte Carlo Tree Search (MCTS) to dynamically allocate tasks to the most suitable LLM, optimizing performance.

Key Innovations

• Adaptive Branching Search: The algorithm balances "searching deeper" (refining existing solutions) and "searching wider" (generating new solutions), ensuring optimal problem-solving strategies.
• Multi-Model Collaboration: The system intelligently assigns tasks to models like OpenAI's o4-mini, Gemini 2.5 Pro, and DeepSeek-R1, leveraging their unique strengths.
• Open-Source Framework: Sakana AI has released TreeQuest under an Apache 2.0 license, enabling developers to implement this technique for commercial use.

Performance Highlights

• ARC-AGI-2 Benchmark: The multi-model system solved 30% of the 120 test problems, a significant improvement over individual models.
• Error Correction: In one instance, a flawed solution from o4-mini was corrected by Gemini 2.5 Pro and DeepSeek-R1, demonstrating the system's ability to combine models for superior results.

Real-World Applications

• Complex Coding: AB-MCTS has been successfully applied to algorithmic coding tasks.
• Latency Optimization: The technique can improve response times for web services.