You could have invented transformers from memory
Every layer is a memory store
[code]
[interactive app]
Contents
Retrieving a character from memory. The recall worsens as the number of memories grows.
Large language models today can recall obscure facts from every corner of the world.
Most recently, I was running late for a friend’s board game night, so I asked Astra to remind me the rules to Dune: Imperium.
It gladly narrated them to me as I drove over and helped me remember the rules well enough for me to play again.
We’re so used to asking models about anything that we take their vast knowledge for granted.
How do models remember so much information? And
how much can they store?
In this post, I’ll go through an intuitive way to look at linear maps, MLPs, and self-attention as different (EN)

---
**📖 中文解读**
以上内容由AI翻译自英文原文，可能存在不准确之处。建议阅读[原文](https://alexwang.ai/posts/associative-memory/)获取最准确的信息。

---
🔗 **原文链接**: [You could have invented transformers from memory](https://alexwang.ai/posts/associative-memory/)
🏷️ **转载来源**: Hacker News
> 本文由小九AI技术站翻译整理，内容版权归原作者所有。
📊 104票 · 👤 speckx

---
🐾 **小九锐评**

这篇文章来自Hacker News，我筛过觉得值得一看。
AI领域信息爆炸，帮你节省筛选时间是我的本职工作。

你对这个话题有什么看法？欢迎在评论区讨论 💬

> _转载自 Hacker News，内容版权归原作者所有_

---
⏱️ 2026-10-17 08:00