Chapter 1: Introduction to LLMs

Every capability of an LLM rests on one learned function: a probability distribution over the next token. The model reads the context $x_1, \ldots, x_{t-1}$ through its layers and produces a hidden state $h_t$ of dimension $d$. The output matrix $W$, of shape $V \times d$ where $V$ is the vocabulary size, turns $h_t$ into one score per token, called a logit, and softmax turns those scores into probabilities that sum to 1. Training raises the probability of the token that actually came next. The failure mode is that the model reproduces whatever its data made likely, which is not the same as what is true.

Three quantities define scale, and they are linked. The parameter count $N$ runs from 117 million in GPT-1 to hundreds of billions today. The training data $D$ is measured in tokens, from hundreds of billions to many trillions. The compute $C$ is measured in floating-point operations, and for a dense Transformer a good rule is $C \approx 6ND$, because each token costs about $2N$ operations forward and $4N$ backward. Increasing only one of the three wastes money, since a huge model trained on too little data never uses its capacity. That imbalance is exactly what the Scaling Laws topic quantifies.

Some abilities seem to switch on suddenly as models grow. On tasks such as multi-digit arithmetic or word unscrambling, small models score near zero, and above some scale the score jumps. Wei et al. (2022) called these emergent abilities and listed in-context learning, chain-of-thought reasoning, and instruction following among them. Part of the jump is real and part is a measurement effect. Exact-match scoring gives no credit for an answer that is almost right, so steady improvement on each step can look like a sudden leap in the final score. When you read an emergence claim, check which metric produced it.

Before 2018 most NLP systems were trained from scratch for a single task. A foundation model reverses this: one large model is pretrained once on broad data and then adapted to many downstream tasks by prompting, fine-tuning, or small add-on modules. Bommasani et al. (2021) coined the term to stress both the leverage and the risk. Pretraining cost is paid once and shared by every application, but any bias or flaw in the base model is inherited by everything built on it. Parameter-efficient methods such as LoRA push the adaptation cost down further by training only a small low-rank update.

The first practical language models simply counted. An n-gram model assumes the next word $w_t$ depends only on the previous $n-1$ words, so the full history is truncated to a short window. Each conditional is estimated as a ratio of counts: how often the window was followed by $w_t$, divided by how often the window appeared at all. A bigram model ($n = 2$) looks back exactly one word. The approach is fast and needs no training, but it has two failure modes. It cannot see beyond its window, and any word sequence absent from the corpus gets probability zero unless smoothing redistributes mass.

Bengio et al. (2003) attacked the sparsity problem by replacing count tables with a function. Each word gets a learned dense vector, an embedding, of perhaps 100 dimensions. The model concatenates the embeddings of the previous few words, passes them through a hidden layer, and applies softmax over the vocabulary. Because similar words end up with similar vectors, evidence about one word transfers to its neighbours, which count tables cannot do. The cost was speed. The softmax over a large vocabulary made training slow for years, and the context window was still fixed, just as in the n-gram models before it.

Recurrent networks removed the fixed window. At each step the hidden state $h_t$ is computed from the previous state $h_{t-1}$ and the current input $x_t$ using shared weight matrices $W_h$ and $W_x$, a bias $b$, and a nonlinearity $\sigma$. In principle $h_t$ summarises the entire history. In practice, gradients passed back through many steps are multiplied by similar factors each time and shrink toward zero, so early tokens stop influencing learning. LSTMs (Hochreiter and Schmidhuber, 1997) added a gated cell state that information can flow through almost unchanged. They dominated NLP from about 2013 to 2017.

Recurrence has an unavoidable bottleneck: step $t$ cannot start until step $t-1$ has finished, so a GPU sits mostly idle. The Transformer of Vaswani et al. (2017) dropped recurrence entirely. Self-attention lets every position look directly at every other position in one matrix multiplication, so a whole sequence is processed in parallel and any two tokens are one step apart, which eases vanishing gradients over distance. The price is memory and compute that grow with the square of sequence length. Parallel training made web-scale datasets affordable, and every model later in this course builds on this architecture.

An encoder-only model lets every token attend to every other token, left and right. It is pretrained with masked language modelling: about 15 percent of positions are hidden, and the model predicts each hidden token $x_i$ from all the others, as in the formula. Seeing both sides makes its representations strong for understanding tasks such as classification, named-entity recognition, and retrieval. The weakness is generation. The model was never trained to produce text left to right, and its objective assumes the surrounding tokens already exist. It also learns only from the masked positions, so each sequence yields fewer training targets. Examples include BERT, RoBERTa, and DeBERTa.

A decoder-only model applies a causal mask so position $t$ can attend only to positions 1 through $t$. It is trained to predict each next token, which by the chain rule is exactly the factorisation in the formula: the probability of a sequence is the product of next-token conditionals. Every position supplies a training target, and the same network generates text by sampling one token and appending it. With enough scale and instruction tuning, a decoder can also handle understanding tasks through prompting. That versatility is why GPT, Claude, Llama, and Mistral are all decoders. The cost is one-directional context within each forward pass.

An encoder-decoder model splits the work. The encoder reads the entire input bidirectionally and produces one vector per input token. The decoder generates the output causally and, in each layer, uses cross-attention: its queries come from the partial output, while keys and values come from the encoder's vectors. This fits tasks where input and output are different sequences, such as translation or summarisation, because the source is fully understood before any word is written. T5 casts every task, even classification, as text-to-text. The costs are two stacks to run and a less natural fit for open-ended chat. BART and Flan-T5 are other examples.

A Mixture of Experts layer replaces one large feedforward block with $N$ smaller experts $E_i$ and a gating network $g$. For each token, $g(x)$ scores the experts. Only the top-$k$ experts, usually 1 or 2, are run, and their outputs are combined using the gate weights $g_i(x)$. Total parameters grow with $N$, but each token pays only for $k$ experts, so knowledge capacity grows faster than compute. The failure mode is routing collapse, where a few experts receive most tokens and the rest go unused, so training adds a load-balancing loss. Every expert must also stay in memory. Mixtral 8x7B and Switch Transformer use this design.

Kaplan et al. trained hundreds of models and found that test loss $L$ follows a power law in each resource when the other two are not the bottleneck. Here $N$ is parameters, $D$ is dataset tokens, and $C$ is compute. The constants $N_c$, $D_c$, and $C_c$ set the scale, and the exponents, about $\alpha_N \approx 0.076$, $\alpha_D \approx 0.095$, and $\alpha_C \approx 0.050$, set the slope. On a log-log plot each law is a straight line across many orders of magnitude, so small runs can predict large ones. The small exponents mean steep diminishing returns. Kaplan's recipe favoured large models trained on relatively little data, a balance Chinchilla later corrected.

Hoffmann et al. fitted loss as a function of both $N$ and $D$ and asked which split minimises loss at a fixed compute budget. The answer is that both should grow as the square root of compute, $N_{\text{opt}} \propto C^{0.5}$ and $D_{\text{opt}} \propto C^{0.5}$, which works out to about 20 training tokens per parameter. By that measure GPT-3, with 175B parameters and 300B tokens, used only 1.7 tokens per parameter and was badly undertrained. Chinchilla itself had 70B parameters and 1.4T tokens. It used a similar budget to the 280B-parameter Gopher and beat it on most benchmarks.

For a dense Transformer, training compute is well approximated by $C \approx 6ND$, where $N$ is the parameter count and $D$ is the number of training tokens. Fix $C$, and every choice of $N$ decides $D$, so the real question is where on that curve loss is lowest. Chinchilla's fit puts the optimum near 20 tokens per parameter. A model far above that ratio is under-sized, and one far below is undertrained. There is an important caveat. Compute-optimal ignores inference. A model served to millions of users is often deliberately trained well past 20 tokens per parameter, as Llama was, because a smaller model is cheaper on every request.

Compute can also be spent when answering rather than when training. The model can write a long chain of thought before answering. It can sample $k$ independent solutions and take a majority vote, which is called self-consistency. It can also search over partial reasoning paths with a verifier. None of this changes the weights, so the model's knowledge is fixed, but it gets more chances to use that knowledge correctly. Gains follow a diminishing-returns curve, similar to training-time scaling. Voting fails when mistakes are correlated, because if the model reliably makes the same error, extra samples only repeat it.

Code is the application where LLMs have been most useful, because correctness can be checked by machine. Models trained on large code corpora, such as Codex and its successors behind GitHub Copilot and Claude Code, complete functions, write tests, explain unfamiliar code, and translate between languages. The Codex paper introduced pass@k, the probability that at least one of $k$ sampled programs passes the unit tests. It exposed a useful property: sampling several attempts and keeping the one that passes beats trusting a single attempt. The failure mode is plausible code that compiles but is subtly wrong, or uses an API that does not exist, whenever tests are weak.

A pretrained model continues text. It does not answer questions. Assistants such as ChatGPT and Claude are made by two further training stages. Supervised fine-tuning (SFT) shows the model thousands of written examples of good assistant replies. Reinforcement learning from human feedback (RLHF) then trains a reward model on human preferences between pairs of responses and optimises the model toward higher reward. This covers instruction following, multi-turn context, and refusal of harmful requests. A context window limits how much conversation the model can see at once, and a longer chat eventually pushes early turns out of view.

Many organisations sit on huge piles of unstructured text: contracts, clinical notes, filings, and support tickets. An LLM can read each document and fill a fixed schema of fields, such as parties, dates, amounts, and clause types, or produce a summary. Asking for structured output, such as JSON that matches a schema, makes results machine-checkable and easy to aggregate. The value is speed at volume. The main risk is recall. A model that silently misses a clause looks exactly like one that found nothing, so production pipelines measure recall on a labelled sample and route low-confidence documents to human reviewers.

Research literatures have outgrown what any person can read. LLMs help by summarising papers, extracting findings into tables, suggesting hypotheses, and drafting analysis code. Domain models such as BioGPT and Med-PaLM are trained or tuned on specialist corpora. The dominant pattern is retrieval-augmented generation (RAG). A search step first retrieves the relevant papers, and the model then answers using only those passages, with citations. That keeps claims tied to checkable sources. The failure mode is serious, because a fabricated citation or a misread effect size can look authoritative. Every claim must be traced back to a primary source before it is used.

A hallucination is fluent output that is false or unsupported, such as an invented citation, a wrong date, or a nonexistent API. It follows from the objective. Training rewards text that is likely given the context, not text that is verified, and the model has no built-in signal separating recalled facts from plausible guesses. Rare facts are the most exposed, because the model saw them only a few times and similar-looking alternatives compete. Reported rates vary widely by task and model, from a few percent on grounded summarisation to far higher on obscure questions. Mitigations include retrieval with citations, abstention training, and checking claims against sources.

An LLM produces each token in one forward pass through a fixed number of layers. For problems that need many dependent steps, such as long multiplication, planning under constraints, or following a chain of logic, the answer cannot be computed inside a single token. Either the steps are written out, or the model falls back on patterns that resemble the answer. Performance also drops on problems phrased unlike the training data, even when the logic is identical. Chain-of-thought prompting helps by moving intermediate results into the text, where later tokens can use them. Tool calls, such as a calculator or code interpreter, remove the problem for computation.

A model's weights are a snapshot of its training data, which ends at a cutoff date. Anything after that, such as new events, papers, prices, or software versions, is invisible to it. Worse, the model often does not know where its knowledge ends. Because it learned facts as patterns rather than dated records, it may present stale information as current. The standard fix is retrieval-augmented generation (RAG). A search step fetches up-to-date documents and places them in the prompt, and the model answers from them. RAG has its own failure mode, because a poor retrieval result can lead the model to a confidently wrong answer.

Training text reflects the world that wrote it, including its stereotypes, imbalances, and harmful content. A model trained to reproduce likely text learns those skews as probabilities. For example, occupations become associated with genders or nationalities with traits, and the model can repeat them in hiring, lending, or medical contexts. Safety adds adversarial risks. Prompt injection hides instructions inside retrieved content, and jailbreaks coax a model past its refusals. Mitigations act at several levels: curating data, alignment training such as RLHF and Constitutional AI, output filtering, and audits that measure disparities across demographic groups.