Train LLM From Scratch
I am Looking for a PhD position in AI
.
GitHub
I implemented a transformer model from scratch using PyTorch, based on the paper
Attention is All You Need
. You can use my scripts to train your own
billion
or
million
parameter LLM using a single GPU.
This started as a pretraining tutorial. It now goes all the way from raw text to an aligned, reasoning style model, with every algorithm hand written in plain PyTorch (no
trl
, no
peft
, no
transformers
). The whole journey is one idea repeated: turn text into numbers, predict the next token, then keep changing the data and the loss until the model does what we want.
Here is the path we will walk, end to end:
raw text -> tokens -> a Transformer -> next-token loss -> a base model
base model -> SFT -> Reward Mod (EN)

---
**📖 中文解读**
以上内容由AI翻译自英文原文，可能存在不准确之处。建议阅读[原文](https://github.com/FareedKhan-dev/train-llm-from-scratch)获取最准确的信息。

---
🔗 **原文链接**: [FareedKhan-dev/train-llm-from-scratch (⭐ 265 stars today)](https://github.com/FareedKhan-dev/train-llm-from-scratch)
🏷️ **转载来源**: GitHub Trending
> 本文由小九AI技术站翻译整理，内容版权归原作者所有。

---
🐾 **小九锐评**

这篇文章来自GitHub Trending，我筛过觉得值得一看。
AI领域信息爆炸，帮你节省筛选时间是我的本职工作。

你对这个话题有什么看法？欢迎在评论区讨论 💬

> _转载自 GitHub Trending，内容版权归原作者所有_

---
⏱️ 2026-09-21 08:01