Small experiments.
Big-picture numbers.
Explore how model size, training data, and hardware shape the bill. These are estimates for rented GPUs; the course’s tiny CPU labs run right here.
training FLOPs (6·N·D)7.44e18
GPU-hours5.2 h
wall-clock on 8 GPUs39.2 min
cost (on-demand, low–high)$10.40–$18.28
training memory (16 B/param), GB2
GPUs just to hold the training state1
Chinchilla-optimal tokens (20·N)2.5B
Prices as of 2026-09. Wall-clock ignores multi-GPU communication overhead, which is real above a few hundred GPUs.