v9.1.1
Menu
Sign In

Chapter 1: The Agent Loop & ReAct Foundations

Week 1-2

What an LLM agent actually is, the perceive-think-act loop that sits at the core of every modern agent, the ReAct pattern that fused reasoning with action, what early autonomous agents got wrong, and the design space you choose from when building your own.

Topics
5
Demo
1
Videos
3
Min read
25

Chapter Overview

An LLM agent is a system that wraps a language model in a loop: it observes the world, thinks about what to do, takes an action, sees the result, and repeats until a task is complete. That sounds trivial, but it is the single most important shift in how we use LLMs since the introduction of chat. A bare LLM is a function — text in, text out, one shot. An agent is a process — a controller that decides when to call the model, what context to give it, what tools it can invoke, and when the work is finally done.

The central object of study is the agent loop itself. Concretely, at every step the agent's controller composes a context from a system prompt, the user goal, the history so far, and any retrieved memory, then asks the LLM to produce one of two things: another thought ("I should look up the user's order ID first") or a structured action (search_orders(user_id='alice')). If it produced an action, the controller executes the corresponding tool, captures the observation, appends (thought, action, observation) to the history, and runs the model again. The loop terminates when the model produces a special final_answer action — or when the controller hits a step budget.

The paper that crystallized this design is ReAct (Yao et al., 2022): Reasoning + Acting. Before ReAct, prompting research had treated reasoning ("chain-of-thought") and acting ("tool use") as separate techniques. ReAct's contribution was simply to interleave them in the same generation: alternating Thought: and Action: lines in a single transcript. That tiny change made the model grounded — its reasoning could be checked against the world via observations, instead of running open-loop and hallucinating its way to a wrong answer.

T1,A1,O1⏟step 1,T2,A2,O2⏟step 2,…,Tk,final_answer(y)⏟step k\underbrace{T_1, A_1, O_1}_{\text{step 1}}, \underbrace{T_2, A_2, O_2}_{\text{step 2}}, \ldots, \underbrace{T_k, \text{final\_answer}(y)}_{\text{step } k}

This chapter covers:

  • What is an agent? — agents vs chatbots vs workflows; when the loop is overkill
  • The ReAct loop — the canonical interleaved Thought/Action/Observation pattern
  • Thought / Action / Observation — what each primitive does and why you need all three
  • Early agent failures — AutoGPT, BabyAGI, and the lessons from open-ended planning
  • The agent design space — the decision tree from "just call the LLM" to "deploy a multi-agent crew"

Chapter Roadmap

Click any topic to jump in

Explainer videoInteractive demo
1
What Is an Agent?

Agents vs workflows vs chatbots — the LLM owns the control flow at runtime.

Agent vs Workflow vs ChatbotThe Three Capabilities Required
The definition crystallizes into a concrete loop

From the abstract definition to the implementation

2
The ReAct Loop

Interleaved Thought / Action / Observation — the canonical pattern that grounded reasoning in real observations.

The Loop, in One PictureWhy Interleave Reasoning and Action
3
Thought / Action / Obs

What each primitive does, and how each one fails independently.

Thoughts: the Plan and the Self-CritiqueActions: the Bounded Interface to the WorldObservations: the Reality Check
What goes wrong when the loop runs unchecked
4
Early Failures

AutoGPT and BabyAGI — what open-ended planning, vector-soup memory, and missing self-critique cost us.

Failure Mode 1: Open-Ended Planning Without a Termination SignalFailure Mode 2: Memory as Vector-DB SoupFailure Mode 3: No Self-Critique, No Recovery
The lessons crystallize into a design rubric
5
The Agent Design Space

A decision rubric: when a workflow beats a loop, and when a single agent beats a crew.

The Spectrum, From Simple to ComplexWhen to Reach for a Multi-Agent System

Topics

5 topics in this chapter

Everyone uses the word agent loosely — a chatbot, a script, a cron job, an LLM with a function call — and that looseness makes the category almost useless for design decisions. Before you can choose an architecture you need a definition sharp enough to exclude things. The one that has held up in practice is behavioural: an agent is a system in which an LLM dynamically directs its own control flow. It decides at runtime which tool to call next, whether to ask for clarification, when to retry a failed step, and when the task is actually finished. Contrast that with a workflow, where you the developer wrote the branches and the model merely fills slots inside a structure fixed ahead of time. This topic draws that line precisely, because the rest of the chapter — the ReAct loop, the three primitives, the early failures, the design rubric — only makes sense once you can tell an agent from the workflow it is so often confused with. Anthropic's Building Effective Agents essay makes the sharp, and initially surprising, case that a workflow is usually what you actually want.

Definition

An agent is a system in which an LLM directs its own control flow at runtime — choosing which tool to invoke, when to ask for clarification, when to retry, and when the task is complete. A workflow, by contrast, has that control flow hard-coded by the developer, with the model confined to fixed, pre-arranged steps.

In this topic

  1. 1Agent vs Workflow vs Chatbot
  2. 2The Three Capabilities Required
1 of 2
Agent vs Workflow vs Chatbot

A useful three-way split:

  • Chatbot: stateless function. One LLM call per user message. No tools, no loop.
  • Workflow: you wrote the control flow. The LLM is called from inside fixed branches and chains — summarize → classify → route — but the structure of the program is determined ahead of time.
  • Agent: the LLM writes its own control flow at runtime. Given a goal, it picks the next step from an open-ended menu of tools, possibly for many iterations, until it decides it is done.

The practical implication: agents are slower, more expensive, and harder to test than workflows. Use them only when the task is genuinely open-ended (you cannot enumerate the steps in advance) or when the user input space is so wide that no fixed pipeline can cover it.

Mathematical Intuition

Think of an agent as a stochastic policy πθ(at∣st)\pi_\theta(a_t \mid s_t) over an action space A\mathcal{A} that includes both external actions (tool calls) and an internal final_answer action that terminates the episode. A workflow is the degenerate case where ∣A∣=1|\mathcal{A}| = 1 at every state — there is no decision to make, only computation to perform. As ∣A∣|\mathcal{A}| grows, you gain expressive power but pay for it: variance in the policy goes up, episodes get longer, and the value of a strong base model dominates over any clever orchestration.

Example:

You are building "a feature where a user types a question and gets the answer from our internal docs." Should you build an agent?

2 of 2
The Three Capabilities Required

Every modern LLM agent depends on three capabilities that did not exist (or were unreliable) in pre-2023 LLMs:

  1. Instruction following at depth. The model must obey a multi-page system prompt that lays out the loop, the tool schema, output format constraints, and safety rules — across dozens of turns without drifting.
  2. Structured output. The model must reliably emit either valid JSON for a tool call or a final answer in the exact shape the controller expects. Function-calling APIs and JSON mode are how this is now done in practice.
  3. In-context reasoning. Given a long history, the model must extract the relevant facts, plan the next step, and avoid repeating work. This is what chain-of-thought and ReAct give you.

If any of the three is missing, you do not have a working agent — you have a system that appears to work on demos and falls apart in production.

Mathematical Intuition

These three capabilities can be cast as constraints on the conditional distribution P(actiont∣history<t)P(\text{action}_t \mid \text{history}_{<t}). Instruction following requires that PP assigns mass only inside the format constraints implied by the system prompt — measurable as the rate at which sampled outputs parse. Structured output requires PP to be peaked enough that greedy or low-temperature decoding produces valid JSON. In-context reasoning requires the effective context length to actually be useful — a model with a 200k window but a 10k useful window will fail on long agent traces.

Theory Exercise

Problem:

A team is building a customer-support assistant that can either (a) answer FAQs from a knowledge base, (b) refund an order, or (c) escalate to a human. They are debating: agent loop or fixed workflow?

Hints:
  • How many distinct branches does the system actually need?
  • What goes wrong if the model picks the wrong tool?
  • Could a router-then-workflow design get most of the value without the loop?
Papers
ReAct: Synergizing Reasoning and Acting in Language ModelsA Survey on Large Language Model based Autonomous Agents
Blogs
Building Effective Agents — Anthropic EngineeringLLM Powered Autonomous Agents — Lilian Weng

Related Problems on PixelBank

Should the Agent Loop ContinuePROAudit an Agent Trace for Step Limits and LoopsFREE