// Notes & essays
Blog
On AI research, reasoning systems, engineering, and the things in between.
Nothing published yet - check back soon.
Latest videos
YouTube ↗-
I gave my game's NPC a language model | Building nanoMMO #25
Watch on YouTube ↗ -
MMO zones and portals in C++ with raylib | Building nanoMMO #22
Watch on YouTube ↗ -
MMO items in C++ with protobuf text format | Building nanoMMO #26
Watch on YouTube ↗ -
Pretraining a GPT on a laptop in 10 minutes | Minimum Viable GPT #2
Watch on YouTube ↗ -
Fine-tuning a GPT to follow instructions | Minimum Viable GPT #3
Watch on YouTube ↗ -
Where a GPT's time goes, profiled | Minimum Viable GPT #6
Watch on YouTube ↗ -
Build a GPT from scratch: 14.6M parameters | Minimum Viable GPT #1
Watch on YouTube ↗ -
A KV cache makes a GPT write 4× faster | Minimum Viable GPT #4
Watch on YouTube ↗ -
A GPT in the browser, a quarter the size | Minimum Viable GPT #5
Watch on YouTube ↗ -
DPO: a GPT learns from preferences | Minimum Viable GPT #7
Watch on YouTube ↗ -
PPO: a GPT learns from its own stories | Minimum Viable GPT #8
Watch on YouTube ↗ -
A BPE tokenizer in C++, 116× faster | Minimum Viable GPT #11
Watch on YouTube ↗ -
RoPE: a GPT that reads past its window | Minimum Viable GPT #10
Watch on YouTube ↗ -
LoRA: fine-tune 2% of a GPT's weights | Minimum Viable GPT #9
Watch on YouTube ↗ -
Flash attention: 12 GB down to 48 MB | Minimum Viable GPT #13
Watch on YouTube ↗
