Chapter 2: Planning Patterns: Plan-and-Execute, ReWOO, HTN
When the ReAct loop wastes too many tokens re-planning every step, you need explicit planning patterns. This chapter covers Plan-and-Execute (plan once, execute many), ReWOO (decouple reasoning from observations to slash tool calls), Hierarchical Task Networks (compose plans from reusable pieces), evolutionary and Monte Carlo planning, and the rubric for when to plan vs. just react.
- Topics
- 5
- Videos
- 3
- Min read
- 17
Chapter Overview
ReAct's strength is also its weakness: every iteration re-runs the planning process from scratch, paying for the full prompt + thought + action on every step. For tasks with many steps, this is expensive and slow. Planning patterns decouple the plan from the execution, so the model only does the hard reasoning once.
Three families dominate:
- Plan-and-Execute (Wang et al., 2023) — a planner LLM emits a sequential plan up front, then a cheaper executor LLM (or even non-LLM code) carries out each step.
- ReWOO (Xu et al., 2023) — decouple Reasoning from Observations entirely: the planner emits a DAG of steps with placeholder variables (#E1, #E2), all tools execute in a batch, and a final solver LLM combines results. Cuts tool calls dramatically.
- Hierarchical Task Networks (HTN) — classical AI planning, adapted for LLMs. Compound tasks decompose into sub-tasks recursively until you reach primitive (executable) actions.
Plus two stochastic search techniques: evolutionary planning (mutate-and-select on candidate plans) and Monte Carlo Tree Search for plan exploration.
The big question this chapter answers: when should you plan up-front vs. plan-as-you-go? The answer depends on (a) how predictable the environment is, (b) how expensive failed steps are, and (c) how large your context budget is.
This chapter covers:
- Plan-and-Execute fundamentals — the planner / executor split
- ReWOO — DAG plans with placeholder variables, batched execution
- Hierarchical Task Networks (HTN) — compound → primitive task decomposition
- Evolutionary & Monte Carlo planning — when search beats greedy
- When to plan vs. react — a decision rubric
Chapter Roadmap
Click any topic to jump in
Plan-and-Execute
Plan once, execute many — slash token cost on long, predictable trajectories.
DAGs (ReWOO) and trees (HTN)
ReWOO
DAG of placeholders, batched tool calls, single solver — constant token cost in the number of steps.
HTN
Compound → primitive task decomposition — bounded depth, plan reuse, human-readable structure.
Evolutionary & MCTS
When greedy planning misses the right plan — search the plan space when stakes are high.
Plan vs React Rubric
A 3-signal decision aid: predictability, cost of mistakes, latency budget.
Topics
5 topics in this chapter
In Chapter 1 you built the ReAct loop, and its greatest strength — re-deciding what to do at every step — is also, on a long task, its greatest cost. Each iteration resends the entire growing transcript and pays for a fresh round of reasoning, even when the overall shape of the work was obvious from the start. Plan-and-Execute is the simplest fix and the natural first planning pattern to learn. One LLM call produces a complete, ordered plan; a second, much cheaper loop executes that plan one step at a time, feeding tool outputs back but not re-reasoning about the whole goal. This topic dissects the split into a planner that does the expensive thinking exactly once and an executor that is little more than a tool-calling harness. We will trace where the token savings come from — often 5–10× on long trajectories — and, just as importantly, where the pattern breaks when reality diverges from the plan the planner committed to. That failure, the stale plan, motivates every more sophisticated pattern in the rest of the chapter.
Definition
Plan-and-Execute is a planning pattern that separates a single expensive planner LLM call, which emits a complete ordered plan from the goal, from a cheap executor loop that carries out each step in turn without re-planning. It trades ReAct's per-step flexibility for a large reduction in tokens on long, predictable trajectories.
In this topic
- 1The Planner / Executor Split
- 2When the Plan Goes Stale
The Planner / Executor Split
The planner sees the user's goal and emits an ordered list of steps in natural language: "1. Look up the user's recent orders. 2. For each order, check the refund eligibility. 3. Refund the eligible ones. 4. Email the user a summary."
The executor then processes each step in order. For each step, it produces a structured tool call (or LLM completion), captures the output, and moves on. The executor does not re-plan unless a step fails — at which point it can either retry, ask the planner to re-plan from the failure point, or escalate to a human.
The big win: the executor's prompt at step is small (just the current step + the most recent observations), not the full ReAct trace. Token usage often drops by 5–10× compared to ReAct on long trajectories.
Compare token usage on a 10-step task between ReAct and Plan-and-Execute.
When the Plan Goes Stale
The Achilles' heel of Plan-and-Execute is the stale plan problem. The planner makes assumptions about what each step will return; if reality diverges, downstream steps become wrong.
Three mitigation strategies:
- Trigger re-planning on failure. When a step throws an error or returns an unexpected result, kick back to the planner with the full failure context. The planner emits a revised plan from that point forward.
- Adaptive batching. Don't commit to all steps up front. Plan the next steps, execute them, then re-plan. Tunes the plan/execute trade-off.
- Conditional plans. Have the planner emit branching plans ("if step 2 returns more than 5 orders, do X; else Y"). Costs more planning tokens but reduces re-planning frequency.
Plan-and-Execute already does the heavy reasoning once, but its executor still calls the model at each step and each of those calls tends to see a growing context. ReWOO (Reasoning Without Observation, Xu et al. 2023) pushes the idea to its logical end and is the pattern to reach for when tool calls are expensive or numerous. Instead of a sequential plan whose steps consume intermediate observations, ReWOO has the planner emit a DAG of steps threaded together by placeholder variables — #E1, #E2, #E3 — reasoning purely from the structure of the goal without ever seeing a single tool result. A mechanical worker then materializes those placeholders in dependency order, and a final solver LLM composes the answer from the resolved values. This topic builds that planner/worker/solver triad and shows why the total LLM token cost becomes roughly constant in the number of tool calls rather than linear. You will also learn the sharp boundary of the technique: ReWOO assumes the plan's shape is knowable up front, and it degrades back toward ReAct precisely when it is not.
Definition
ReWOO (Reasoning Without Observation) is a planning pattern in which a planner emits a DAG of steps linked by placeholder variables without seeing any tool output, a worker resolves the placeholders in dependency order, and a solver composes the final answer. Its LLM token cost is roughly constant in the number of tool calls rather than growing with the trajectory.
In this topic
- 1The Planner / Worker / Solver Triad
- 2When ReWOO Wins, When It Loses
The Planner / Worker / Solver Triad
ReWOO runs in three stages:
- Planner. Sees the goal. Emits a list of steps where each step's input can reference a placeholder for any prior step's output: #E1 = search['Napoleon birthplace']; #E2 = search['#E1 population 2020']; #E3 = compute['Final: #E2']. Critically, the planner emits all placeholders without knowing what #E1 will return — it reasons purely from the goal's structure.
- Worker. Executes each step in dependency order, materializing placeholders as it goes. This is mechanical — no LLM calls needed (or just function calls, depending on the tool).
- Solver. Sees the original goal and the resolved placeholders. Composes the final answer.
The planner sees no observations; the solver sees the original question + final resolved values. Neither sees the full transcript, so total tokens stay roughly constant in the number of steps.
ReWOO claims to be more efficient than ReAct. Compare on a 5-tool-call task: total LLM tokens used.
When ReWOO Wins, When It Loses
ReWOO wins when:
- The plan structure is predictable from the goal — "look up X, then look up Y based on X, then combine"
- Tool calls are expensive (long-running, paid APIs) and you want to batch them
- You want a predictable LLM cost regardless of trajectory length
ReWOO loses when:
- The next step's type depends on the prior step's content (e.g., "if it's a software bug, file a Jira; if it's a billing issue, refund")
- Tools have cascading failures — a downstream step needs to react to an upstream error, but the planner already committed
- The task involves exploration ("search the codebase until you find the bug")
In practice, hybrid approaches dominate: ReWOO for the well-structured prefix of a task, ReAct for the exploratory tail.
Theory Exercise
Problem:
You have a customer support task: 'Fetch the user's last 3 orders, summarize them, and email the user.' Would ReWOO outperform ReAct here?
Hints:
- Are the steps statically known from the goal alone?
- Does step 2's content depend on step 1's content (vs. just the placeholder)?
- What if step 1 returns zero orders?
ReWOO and Plan-and-Execute both produce flat plans, but many real goals are naturally nested: 'onboard an employee' contains 'create an account', which itself contains 'provision email' and 'assign a password'. Forcing that structure into one flat list throws away the hierarchy that makes it comprehensible and reusable. HTN planning — Hierarchical Task Networks, imported from classical AI and rediscovered for LLM agents — keeps the hierarchy explicit. It splits the world into compound tasks (abstract goals like book a flight) and primitive tasks (single executable tool calls), with a library of methods that say how to decompose each compound task into smaller pieces. The planner recurses, expanding compound tasks into sub-tasks until every leaf is a primitive action the executor can run. This topic teaches that decomposition and why LLMs make it newly practical: instead of a hand-authored method library, the model proposes decompositions on the fly from its world knowledge. The payoffs are plan reuse, bounded depth, and a plan that reads like the user's own mental model — which is why HTN dominates in structured, repetitive domains like IT operations and customer support.
Definition
A Hierarchical Task Network (HTN) plan decomposes compound tasks — abstract goals — into sub-tasks via reusable methods, recursing until every leaf is a primitive task that maps to a single tool call. For LLM agents the model proposes decompositions on the fly, yielding plan reuse, bounded branching, and structure that mirrors the user's mental model.
In this topic
- 1Compound vs Primitive Tasks
- 2Why HTN Is a Good Fit for LLM Agents
Compound vs Primitive Tasks
Primitive tasks map to single tool calls. They have a fixed signature: search(query), send_email(to, body), query_db(sql). The agent executes them directly.
Compound tasks are abstract goals: book_flight(passenger, origin, destination, date). The HTN library has one or more methods for each compound task — each method is a recipe for decomposing it:
method book_flight(passenger, origin, destination, date):
options = search_flights(origin, destination, date)
chosen = pick_best(options, passenger.preferences)
payment_id = charge_card(passenger.card, chosen.price)
confirmation = book(chosen.id, passenger, payment_id)
notify(passenger.email, confirmation)
Each line is itself a task — possibly compound. The planner recurses until every leaf is a primitive task. The agent then executes the leaves in order.
An agent is asked to 'set up a new employee in our HR system.' Decompose this with HTN.
Why HTN Is a Good Fit for LLM Agents
Classical HTN planners need a hand-authored library of methods, which historically limited their adoption. LLM-based HTN flips this: the LLM proposes decompositions on the fly, drawing from its world knowledge.
Three wins of HTN over flat planning:
- Plan reuse. The same decomposition is used wherever a compound task appears. A new top-level goal that includes 'create_account' as a sub-task gets the same well-tested 3-step decomposition automatically.
- Bounded depth. Each method has a fixed length (3–5 sub-tasks usually). The plan is a tree of bounded branching factor, much easier to reason about than an open-ended sequential plan.
- Natural decomposition for users. Compound tasks read like the user's actual mental model ("set up the employee"), while primitive tasks are at the API level. The planner bridges the two.
HTN dominates in domains with strong structural reuse: customer support, IT operations, structured data ETL, multi-step business processes.
The planners so far are essentially greedy: they commit to one plan and run it. That works when a good plan is easy to name, but some goals live in a huge space of plausible plans where the first one the model writes is rarely the best, and where the quality of a plan is only knowable noisily, after some simulation. When that is the situation, search beats greedy planning, and this topic adapts two classic search techniques to LLM agents. Evolutionary planning treats plans as a population: generate many candidates, score them with a fitness function, keep the best, and breed new ones by mutation and crossover — the shape of AlphaCode's competitive-programming pipeline. Monte Carlo Tree Search (MCTS), the algorithm behind AlphaGo, instead grows a tree of partial plans, using the explore-exploit UCB1 rule to decide where to look next and LLM-imagined rollouts to score leaves. Both cost far more model calls than greedy planning, so this topic is as much about when the expense is justified — single-shot, high-stakes tasks — as about the mechanics. It also sets up LATS in Chapter 3, which is MCTS wrapped around a full agent.
Definition
Evolutionary planning searches the plan space by generating a population of candidate plans, scoring them with a fitness function, and breeding survivors through mutation and crossover. Monte Carlo Tree Search grows a tree of partial plans, balancing exploration and exploitation with UCB1 and scoring leaves via rollouts — both trade many extra LLM calls for finding plans greedy reasoning misses.
In this topic
- 1Evolutionary Planning
- 2Monte Carlo Tree Search (MCTS) for Plans
Evolutionary Planning
Evolutionary planning treats plans as a population:
- The LLM generates candidate plans for the goal (the initial population)
- A fitness function scores each (heuristic checks, simulated execution, or LLM-as-judge)
- The top plans are kept; new candidates are generated by mutation (LLM rewrites a section) and crossover (LLM combines two plans)
- Repeat for generations
This dominates greedy planning when (a) the goal admits many plausible plans (so diversity helps) and (b) you have a cheap fitness function. AlphaCode used a similar approach for competitive programming — generate millions of solutions, filter by passing the public tests, then sample the survivors.
Monte Carlo Tree Search (MCTS) for Plans
MCTS explores the tree of partial plans by repeatedly: (1) Selecting a path from root to leaf using UCB1 (explore-exploit balance), (2) Expanding the leaf with one new candidate next step, (3) Simulating a rollout to a terminal state (executing or imagining further steps), and (4) Backpropagating the outcome score up the path.
For LLM agents, MCTS is what makes Tree-of-Thoughts and LATS (Language Agent Tree Search, Chapter 3) work. The 'rollout' is the LLM imagining further steps; the 'value' is an LLM-as-judge score. Even at depth 3 with branching factor 5, MCTS routinely finds plans the greedy LLM misses.
The cost: MCTS does many more LLM calls than greedy planning. Worth it when: the task is single-shot and high-stakes (a single contract review, a one-shot scientific experiment), not for high-volume serving.
You have now met a spread of planning patterns — Plan-and-Execute, ReWOO, HTN, evolutionary and Monte Carlo search — alongside the plain ReAct loop from Chapter 1, and the practical question is no longer how each works but which one to reach for. Choosing the fanciest pattern by default is the same mistake as reaching for an agent when a workflow would do: every step up in sophistication costs tokens, latency, or engineering effort you may not need to spend. This closing topic gives you a compact decision rubric built on three signals — how predictable the trajectory is, how expensive a mistake is, and how much latency budget you have — that between them point you at the right pattern most of the time. It then makes the honest admission that most production agents are hybrids: HTN at the top to decompose the goal, ReWOO or Plan-and-Execute to run each well-structured branch cheaply, and a small ReAct loop at the leaves to recover when a step surprises you. Getting this boundary right is what separates an agent that is merely clever from one that is affordable to serve.
Definition
The plan-versus-react decision rubric selects a control pattern from three signals: trajectory predictability, the cost of a mistake, and the latency budget. Predictable, expensive, or high-stakes work favors up-front planning (Plan-and-Execute, ReWOO, or search); unpredictable, cheap, exploratory work favors reactive looping. Most production agents combine both in a hybrid.
In this topic
- 1Three Signals
- 2The Hybrid Default
Three Signals
1. Predictability of the trajectory.
- High predictability (you can write the steps as a sequence): Plan-and-Execute or ReWOO.
- Low predictability (each step's output radically changes the next): ReAct.
2. Cost of mistakes.
- Cheap (text-only outputs you can throw away): ReAct — explore freely.
- Expensive (irreversible API calls, money, side effects): Plan-and-Execute with human approval at the boundary.
3. Latency budget.
- High budget, single-shot job (research, contract review): MCTS / evolutionary planning — search wins.
- Tight budget, high-volume serving: ReWOO — fixed token cost, parallel tool calls.
Choose a pattern: an agent that books travel — flights, hotels, ground transport — for one trip.
The Hybrid Default
Most production agents are hybrids. A common pattern:
- Top-level: HTN. The agent decomposes the user's goal into a tree of compound and primitive tasks.
- Sub-task: ReWOO or Plan-and-Execute. Each compound branch is planned up front and executed in batch.
- Leaf-level: ReAct. When a primitive task fails or returns unexpected data, drop into a small ReAct loop to recover.
This pattern gives you the token efficiency of planning at the top, with the resilience of ReAct at the bottom. Claude Code, Devin, and most production coding agents use a variant of this.
Theory Exercise
Problem:
Your team has a production agent that does customer-support email triage. It's slow and expensive. Logs show 80% of trajectories follow the exact same 4-step pattern (categorize → look up account → draft response → send). Pick a redesign.
Hints:
- Is 80% predictability enough to commit to a fixed plan?
- What would cover the other 20%?
- Where does the planning vs reacting boundary go?