Build a tiny GPT-style transformer from scratch — token and positional embeddings, masked multi-head self-attention, and a transformer block — then train it as a character-level language model on a small built-in corpus and generate new text. Small enough to train in seconds on a GPU, complete enough to show how a real LLM works under the hood.
Corpus & Tokenizer
Batching
Build & Train the Transformer
Generate & Inspect
Premium Project
This guided, GPU-powered project is available to Premium members. Upgrade to train models end-to-end in an interactive notebook.