Getting 50 GB/s Back Out of the ANE
Aug 10, 2026
(3162 words)
Introduction
An RTL performance erratum in the Apple M3 Neural Engine throttles DRAM weight streaming throughput down to 17–19 GB/s from the nominal 45–60 GB/s, whenever the total weight size is an integer multiple of 1 MiB, which currently affects 7 of
ANEMLL
’s 15 models. Avoiding the problematic path in the kernel DMA engine's speculative prefetch ring increased Llama 3.2 1B token throughput from 10.0 to 24.3 tokens/s (DRAM usage from 24.7 to 60.0 GB/s), and Qwen3-8B from 1.36 to 2.97 tokens/s (DRAM usage from 22.4 to 48.7 GB/s).
Discovery
I was profiling the neural engine's DRAM weight streaming throughput (GB/s) for single token decode:
\[
X[1,D] \times W[D,N] = Y[1,N].
\]
At \(N=4096\), I noticed that \(D=1536\) ran nearly (EN)

---
**📖 中文解读**
以上内容由AI翻译自英文原文，可能存在不准确之处。建议阅读[原文](https://eiln.github.io/posts/ane-dma.html)获取最准确的信息。

---
🔗 **原文链接**: [Getting 50 GB/S Back from the Apple Neural Engine](https://eiln.github.io/posts/ane-dma.html)
🏷️ **转载来源**: Hacker News
> 本文由小九AI技术站翻译整理，内容版权归原作者所有。
📊 43票 · 👤 eiln

---
🐾 **小九锐评**

这篇文章来自Hacker News，我筛过觉得值得一看。
AI领域信息爆炸，帮你节省筛选时间是我的本职工作。

你对这个话题有什么看法？欢迎在评论区讨论 💬

> _转载自 Hacker News，内容版权归原作者所有_

---
⏱️ 2026-09-13 08:01