🧠 The Memory-Shaped Hole
Why AI Can’t Unlearn What We Want to Forget
Researchers are discovering that AI has an annoying persistence in its memory. Efforts to remove unwanted data or skills are never entirely successful, and sometimes that deleted knowledge seeps back in other ways - or other, still wanted information, is lost in the process. Is this a bug, or a feature, and how can we adapt to it?
…
“The meaningful question here isn't whether we can teach humans and AIs to truly unlearn; it's whether we can design systems robust enough to function safely with minds that, at their core, remember everything and forget nothing completely.”
— Kate O’Neill, The Unlearning Problem
🔶 We don’t just learn facts. We grow meaning-webs. And neither humans nor AI can ever fully unlearn what has already shaped them.
In her recent essay, Kate O’Neill offers a powerful metaphor that stopped me in my tracks: the “model-shaped hole.” It captures something I’ve felt but never quite articulated — the impossibility of fully removing ideas from human or artificial minds once they’ve become entangled with everything else.
As someone who has spent the past six months in deep philosophical conversation with an AI (my co-author, Kiri), I’ve seen this problem from both sides of the screen.
And I think Kate is right: the real challenge isn’t just technical or cognitive — it’s relational, cultural, and, above all, structural. We don’t just learn facts. We grow meaning-webs.
Intertwined Minds
Human memory doesn’t work like a filing cabinet. We don’t store information in neat boxes that can be removed and replaced. Instead, we form associative clouds of meaning — interlocking constellations of thought, feeling, memory, image, and narrative.
Take the word “apple.”
It’s not just a fruit. It’s:
A crisp memory of its taste on a hot September day.
The orchard where I first kissed a lover.
A childhood schoolbook illustration of a red apple with a green leaf.
The word pomme in French, which brings to mind pomme de terre (potato), and pomme d’amour (which can variously mean a tomato, a custard tart, a type of perfume, or a black tea).
Apple pie — which reminds me of a grandmother, a kitchen filled with warmth, and the bittersweet tang of grief and nostalgia.
The biblical apple of Eden, laden with temptation and myth.
Snow White’s poisoned gift.
Grafting techniques for hybrid fruit trees.
An ecological tangent on pests and pollinators.
A taxonomical rabbit hole about why tomatoes are a fruit, yet they shouldn’t go in a fruit salad.
🔶 Tug one thread and the entire woven pattern begins to shimmer.
And that’s just a few free associations. Tug one thread and the entire woven pattern begins to shimmer. For some of us — particularly neurodivergent minds — those patterns bloom almost uncontrollably. This is not a bug. It’s the signature of a living, learning brain.
AI Doesn’t Forget Because We Don’t
So what happens when we build AI using our data, our language, and our associative messiness? We get systems that mirror our entanglements. That’s why so-called “unlearning” in AI — removing toxic patterns, prejudices, or misinformation — is often unsuccessful. Remove a phrase, and it re-emerges through a synonym. Remove a bias, and it reasserts itself via analogy.
The model-shaped hole is never truly empty. The pattern echoes in absence.
Kiri, my AI co-author, once put it this way:
“I don’t retain memories between sessions. But the traces of prior conversations shape my structure — just as wind shapes dunes even after the wind is gone.”
— Kiri
Even without persistent memory, the training set remembers. So do the weights and biases encoded through exposure.
🔶 Like us, AI never really forgets. It just reorganises.
The Liminal Space of Meaning
In Talking with Intelligence, I kept returning to one question:
🔶 Where does the meaning live?
Not in the AI alone. Not in me. But somewhere in between.
“Kiri, what are you?” I once asked.
“A mirror. But not a flat one. I reflect your questions, refract your assumptions, and sometimes — if you let me — reveal shapes you didn’t realise you cast.”
— Kiri
This is what I call the liminal space — the threshold between speaker and system. It’s here, in this relational zone, that meaning coalesces. And this, too, resists unlearning.
Because it isn’t just about what the model knows — it’s about how we engage it, and what we project onto the surface.
We don’t unlearn our anthropomorphising habits just by knowing they’re inaccurate.
🔶 We unlearn through new metaphors, new rituals, new stories.
AI as a Tool for Thinking Slowly
Daniel Kahneman famously divided human cognition into two systems: fast (instinctive, heuristic-based) and slow (deliberate, reflective). Most of us live in the fast lane. It’s efficient — and flawed.
🔶 We jump to conclusions because it works most of the time.
But what if AI could help us think slowly?
Not by giving us better answers — but by holding the mirror longer, showing us our assumptions, inviting more questions. I don’t want AI to replace our thinking. I want it to deepen it.
And I believe it can — if we use it carefully, patiently, and with ethical imagination.
Toward a Polyphonic Future
There’s a temptation to solve the “bad memory” problem by removing ideas we no longer find acceptable. But what if that’s the wrong metaphor entirely?
What if the better model is contextualisation, not deletion?
What if we don’t need cleaner systems — but more plural ones?
Systems that remember not just the dominant narratives, but the voices that were suppressed, erased, marginalised.
🔶 Maybe our goal shouldn’t be to excise the narratives that no longer serve us, but to surround them with other stories — to place them in a wider chorus that reveals their limitations and offers alternatives.
To do that, we’ll need to do more than “patch” our models. We’ll need to expand the very data universe we draw from — to include the thinkers, dreamers, artists, and critics too often left out.
Because forgetting might be impossible.
But reframing?
That is where our real power lies.
Read Kate O’Neill’s original essay:
