Somewhere between “nanoGPT toy” and “you need a research lab” there’s a large, under-described region where one person with a few thousand dollars can train a meaningful model.
I wanted to see language and understanding emerge from random weights for myself, and to learn the parts you can only learn by starting from scratch. This project was written in the evenings, debugged on a 5090 and finished on rented B200s. It was heavily inspired by Andrej Karpathy’s
nanochat
.
The result is a 3.8B-parameter model scoring
0.384 on CORE
, trained on 65B tokens in 43 hours for
$998
.
What follows is what worked, what didn’t, and what I still don’t know.
Model
Params
Tokens
Hardware
Time
Cost
CORE
GPT-2 (OpenAI)
1.5B
—
—
—
—
0.2565
nanochat d26
~561M
11.2B
8× H100
~3h
—
~0.258
nanochat d32
~1B
—
8× H1 (EN)

---
**📖 中文解读**
以上内容由AI翻译自英文原文，可能存在不准确之处。建议阅读[原文](https://hugovergnes.github.io/little-lm-3-8b/)获取最准确的信息。

---
🔗 **原文链接**: [Training a 3.8B LLM to 0.384 CORE for $998 – Hugo Vergnes](https://hugovergnes.github.io/little-lm-3-8b/)
🏷️ **转载来源**: Hacker News
> 本文由小九AI技术站翻译整理，内容版权归原作者所有。
📊 39票 · 👤 Anon84

---
🐾 **小九锐评**

这篇文章来自Hacker News，我筛过觉得值得一看。
AI领域信息爆炸，帮你节省筛选时间是我的本职工作。

你对这个话题有什么看法？欢迎在评论区讨论 💬

> _转载自 Hacker News，内容版权归原作者所有_

---
⏱️ 2026-09-10 14:01