Frontiers Enhancing Memory Retrieval in Generative Agents through LLM-Trained Cross Attention Networks

Published in Frontiers in Psychology (Volume 16 - 2025)
View all articles{rel="nofollow"}

Key Findings

• Researchers developed a novel Auxiliary Cross Attention Network (ACAN) to improve memory retrieval in generative AI agents.
• The system ranks attention weights between an agent's current state and stored memories, selecting the most relevant ones.
• First study to utilize LLMs to train a dedicated agent memory retrieval network, achieving significant improvements in adaptability and behavioral consistency.

Methodology

1. Created a text-based simulation of a generative agent world with multiple agents and interactive locations.
2. Implemented ACAN to calculate and rank memory relevance using attention mechanisms.
3. Used LLM-assisted training by comparing retrieved memories with base methods, creating a novel loss function for optimization.

Implications

• Addresses the high cost of individually training LLMs for each AI agent.
• Enhances agents' ability to maintain unique characteristics and memories.
• Opens new possibilities for LLM applications in AI memory management.

Citation: Chuanyang and He (2025). Front. Psychol. 16. doi: 10.3389/fpsyg.2025.1591618
Creative Commons License{rel="nofollow"}