The Art of Forgetting: Why True Memory is the Next Frontier for Autonomous AI Agents
VTAI is on a clear trajectory. A couple of years ago we were introduced to systems like ChatGPT, which acted as powerful consultants for exploring new domains. We'd ask questions, gather information, and use the responses to guide our own creative work. Today, we've moved to the next stage: AI agents are now our supervised partners, taking on complex tasks like writing software code based on our high-level requirements.
The next steps in this evolution seem inevitable. First, these agents will become so reliable that we trust them to work without direct supervision. From there, we may trust them to not only create but also execute their own code, autonomously iterating and improving based on real-world outcomes. To make this leap from supervised tool to autonomous partner, however, agents need more than just larger models. They need a sophisticated, active memory.
Memory: The Engine of Reliability and Efficiency
As foundation models continue their march toward AGI, they will undoubtedly become more capable. That said, they will still make mistakes and encounter problems they cannot solve on the first try. Memory is the function that allows an agent to learn from its experiences—to remember what worked, what didn't, and how it solved a specific problem in the past. After all, human intelligence is our only working example of AGI, and humans rely on external memories like books and libraries to avoid reinventing the wheel.
Beyond performance enhancement, memory directly impacts operational costs. Repeated derivation of solutions from first principles is computationally intensive and economically unsustainable at scale. An effective memory mechanism facilitates the recall of previously established solutions, thereby conserving computational resources, reducing token usage, and significantly lowering the total cost of autonomous operation.
Didn't RAG Solve That?
When we talk about external memory for LLMs, the first thing that comes to mind is Retrieval-Augmented Generation (RAG). RAG has been a powerful, early form of memory, allowing us to ground models in factual data. But it's a simplistic approach. RAG's philosophy is to memorise everything that might ever be useful and then retrieve the most relevant pieces at inference time.
Just because we can store vast amounts of information doesn't mean we should. Biological memory, particularly in humans, is an active, dynamic process involving abstraction, knowledge synthesis, decomposition, and the generation of novel insights from stored information. Crucially, it also encompasses forgetting, which serves as a vital mechanism for pruning irrelevant or obsolete data, preventing interference, and promoting generalisation. For AI agents to become truly useful, they too will need to move beyond simply storing information and toward actively curating it into knowledge and insight.
The Bitter Lesson and the Search for a Fitness Function
This raises a question: should we be meticulously designing these memory systems, or should we trust in Richard Sutton's "bitter lesson"? The lesson argues that brute-force, data-based optimisation will always outperform systems based on human-designed knowledge. It's a compelling idea that reflects how evolution optimises for fitness within a given environment.
While appealing, a purely data-driven approach to memory system development merely shifts the design challenge from the system itself to the definition of its fitness function. Natural selection benefits from a universally unambiguous fitness function: species survival. In technological design, a clear and simple objective function is frequently absent. The optimal fitness function for an AI agent's memory system remains an open research question for now.
The pragmatic path forward involves a co-evolutionary design process. We are at a stage where we must begin the development of advanced memory components through deliberate engineering. This iterative process will progressively deepen our understanding of what constitutes "good" memory, enabling the concurrent refinement of the fitness function. Through this human-led, co-evolutionary cycle, a robust fitness function and an optimised memory system will emerge. At that point we can hand the baton to AI to take the optimisation further.
Building truly autonomous agents requires us to solve the memory problem. This means moving beyond simple retrieval and designing active systems that can learn, generalise, and strategically forget.
This is exactly what we've been working on at memco, starting with Spark - a memory solution that you can plug into your favourite coding agent. Join the waitlist at memco.ai if you want to be the first to know more!
And coding is just the beginning. Get in touch if you want to discuss memory for agents and how this could work for your AI use case.
