
FIELD NOTES / 001
A lot more understanding.FROM FIRST PRINCIPLES
TO YOUR FIRST MODEL
A practical field guideVol. 01 — LLMs
Understand it.
Train it.
Make it better.
A language model isn’t magic. Follow the ideas, move the numbers, and train one yourself.
Start with the first lessonNo setup. Just curiosity.
The journey10 hands-on lessons
4h 35m at your own pace
The approachDiagrams you can explore
Real models you can train
/01The curriculumOne model.
One model.
The whole story.
From its first guess to a useful answer. Each chapter connects an idea, an experiment, and a way to check what you learned.
Your progress stays in this browser.
Tiny training runs use this server’s CPU.
- 00What "training" actually isA model is a box of knobs. Loss is how wrong it is. Training turns the knobs to make loss smaller — and you'll watch it happen for real.
- 01Tokens: how text becomes numbersA model does arithmetic, so text has to become numbers first. How a tokenizer chooses those numbers decides how long every sequence is — and how much every run costs.
- 02Inside the transformerHow token ids become vectors, attention gathers context, residual paths carry it, and repeated blocks turn width and depth into a model.
- 03Data: what you feed itA model can only learn from the stream it receives. Build that stream, remove repeats, hold out an honest test, and give each parameter enough tokens to learn from.
- 04The pretraining loopOne step of training, phase by phase — forward, loss, backward, update — then the three dials that shape a run: the learning-rate schedule, the batch size, and when to stop. Ends with the run the rest of the course builds on.
- 05Scaling laws → GPU-hours → dollarsTurn model size and training tokens into FLOPs, turn FLOPs into GPU-hours with model FLOPs utilisation, and turn GPU-hours into a dated dollar estimate. Then choose where to sit relative to the Chinchilla frontier.
- 06Fine-tuning: make it 100 % better at one taskThe course's title question, answered: take the Shakespeare model that cannot add, run the same loop on 20,000 sums, and watch accuracy go from 0 % to 100 % — with every knob, or with one knob in ten (LoRA).
- 07Alignment: RLHF & DPOThe training signal changes for the first time: not the next character, but a human's choice between two answers. Reward models, the RLHF pipeline, and DPO — which skips the reward model and trains the policy on the pairs directly, on a leash to a frozen copy of itself.
- 08Evaluation: knowing it got betterA loss curve says the knobs moved. Only an evaluation says the model got better — scored on data it never saw, on the task you care about, on enough examples to trust the number, and against the checkpoint it came from.
- 09Inference & serving costTraining chooses the knobs once. Inference runs those knobs for every answer: one forward pass per token, with caches, batches and smaller weight formats deciding how much service fits on each machine.
You learn it by
making it happen.
Change a learning rate. Watch the loss move. Make a prediction, run the experiment, and see for yourself.
Let’s begin