Karpathy is Talking About Cognitive Core, We're Building the Memory for It
VTIn his recent conversation on the Dwarkesh podcast, Andrej Karpathy articulated a vision that resonates deeply with our R&D philosophy at memco. He introduced the idea of a "cognitive core" for AI, a future where we separate the raw intelligence of a model from the burden of remembering vast amounts of knowledge. This separation hints at a practical approach for the next iteration in AI architecture, and it's precisely what we are building toward.
The Cognitive Core: Separating Intelligence from Knowledge
Andrej's central point is that today's LLMs are tasked with two distinct functions: being intelligent and memorizing the Internet. He argues that this dual role is a handicap. The model's intelligence, its ability to reason, plan, and generalize, is distracted by the need to be a static repository of facts. As he puts it, "I want to remove the memory... and only maintain the algorithms for thought".
This separation can be transformational. By offloading knowledge storage to an external system, model training can focus on honing their core cognitive skills: information processing, pattern recognition, and sense-making. The models only need to remember sufficient knowledge about the world to be able to develop a common sense, they don't need to also become savants. That frees the LLMs to become more dynamic and adaptable problem-solvers. Karpathy predicts this could lead to highly capable yet much smaller models in the future, perhaps around one billion parameters, that act more like human thinkers who look things up when they don't know the answer.
Our Research Validates This Vision
At MemCo, this idea isn't a future prediction, it's what we're seeing today. Our active memory layer, Spark, is designed to be the external memory that can be leveraged by the cognitive core Andrej describes. It allows AI agents to learn from experience, storing and curating actionable knowledge so the base model can focus on pure intelligence.
Our recent research provides a clear signal that this approach works. In experiments using the DS1000 coding benchmark, we tested how a smaller, more efficient model (Claude 3.5 Haiku) performed when given access to Spark's memory layer, and the results were striking. Out of the box Haiku solves around 53% of problems. Given access to a Spark memory populated with relevant software documentation lifts that performance to 62%. When the Spark memory is further populated with experiential data from other agents solving data science problems, Haiku achieves a 75% pass rate, which puts it on par with Claude 4 Sonnet when used without an external memory.
This means we essentially leveraged Haiku's "cognitive core" for the task of programming and supplied the specialised knowledge through Spark's external, active memory. The experiment demonstrates that separating intelligence from knowledge isn't just a theoretical possibility, it's something that is possible today. We are continuing to improve Spark and will be publishing a full technical report once that work is complete, but these early results are very encouraging!
The Practical Benefits of an Externalized Memory
While we wait for the hyper-efficient, 1-billion-parameter cognitive cores of the future, this architectural separation already unlocks more immediate value.
By externalizing memory, we can leverage smaller, faster, and more cost-effective models without sacrificing performance. This has significant implications for enterprise adoption. Highly regulated industries like finance and healthcare, or organizations protective of their IP, can pair a powerful open-source model running on their own infrastructure with Spark's memory layer. This gives them state-of-the-art capability while maintaining full control over their data.
This paradigm also extends to individual developers. With the increasing AI capabilities of modern hardware, like Apple's silicon, running smaller LLMs locally as coding agents is becoming practical. A developer can use their laptop's own processing power, augmented by Spark's shared memory, to create a powerful coding assistant that saves on token costs and learns from their unique context, and that of their community of practice.
Building the Future, Today
The path to more capable, autonomous AI isn't just about building bigger models, it's about building smarter architectures. Andrej Karpathy's concept of the cognitive core aligns perfectly with our vision of an active memory layer that transforms static models into dynamic collaborators that learn from experience. Our research shows this isn't far in the future, it's a tangible reality that is already hitting the mark.
We are building the active memory layer to unlock this next frontier in agentic AI. If you're interested in giving your agents a memory that learns, join the waitlist for Spark at memco.ai, or get in touch.
