The planner loop: perceive, plan, call, monitor, replan

Where you are. Your skills sit behind tool calls that report the world honestly and can be replayed without moving the arm twice. This lesson is the loop that decides which one to call next.

Six steps, and the second one fails

You ask for the desk to be cleared. The model comes back with six steps, and they are good steps. You would have written the same six.

Your runner walks the list. Step one, look, returns a scene. Step two, pick(mug), returns the failure you wrote yourself in the last lesson: grasp failed, the mug is on the floor, the gripper is empty. Step three runs anyway, because step three is simply the next element of a list. place(mug, tray) places nothing. Steps four, five and six execute perfectly against a desk that stopped matching the plan somewhere around second nine.

The run ends. Your runner prints done.

Nothing crashed. Nothing threw. No assertion failed and the log is clean. The model was not wrong, the skills were not wrong, and the result is a mug on the floor and a report saying the desk is clear.

The fix is not a better model and it is not a better plan. It is the shape of the loop.

The idea in one paragraph

A planner loop is the sense-decide-act loop from the first lesson of this course, run three orders of magnitude slower: perceive the scene, plan a few steps, call one skill, monitor what actually happened, and replan when the world disagrees with the plan. Its only real job is to keep the model’s picture of the desk matched to the desk. A plan is a hypothesis about a world you observed once; every skill call is an experiment that can falsify it; and a loop that never checks is not a loop at all, only a list with extra latency. Almost everyone gets perceive, plan and call right on the first attempt, because those three are the parts that produce a demo. Monitor and replan are the parts that produce a system.

The five steps and where each one breaks

Notice what the diagram does not have: a box labelled replan. Replanning is an edge, not a stage - the path taken when monitoring disagrees with the plan. Things drawn as edges are the things that get left out of implementations, which is most of why the failure at the top of this lesson is so common.

Perceive: small, slow, never the whole truth

The observation you hand a planner is far smaller than instinct suggests. Google’s shipped robotics API caps video into the session at one frame per second, which is worth adopting as a discipline even where nothing enforces it: the planner is not watching the robot, it is glancing at it. Send one frame, plus the structured state you already have for nothing - joint angles, gripper open or closed, what the last skill reported - and leave everything that changes faster than a second to the layers built for it.

What that frame is actually worth is the subject of grounding, and the short version is that a detector returning a confident label for the wrong object will corrupt every later step of this loop without raising anything. Assume perception is imperfect, and build the loop so that being wrong once is survivable.

Plan: a hypothesis with a shelf life

Ask for three steps, not twelve. Every step beyond the next one or two is a prediction about a world you have not seen, and the further out it reaches the more confidently wrong it becomes. Long plans feel more capable and are mostly a way of committing early to guesses you will pay for later.

The counter-argument is that structure, not plan length, is what carries long tasks, and the evidence supports it strongly.

Call: one at a time, and read the reply as an observation

One skill per turn, blocking, with the result fed back before the next decision. Batching two calls means the second one runs against a world the model has not seen, which is the failure from the top of this lesson in miniature.

The reply itself is your cheapest observation. It arrives with no camera frame and no extra model call, and if you wrote it as the last lesson asked - a sentence about the desk rather than about the request - it tells you most of what the next decision needs. Use it. Do not trust it alone.

Monitor: the step that actually closes the loop

Monitoring is two different checks, and collapsing them into one is the most expensive mistake in this lesson.

The step check asks whether this skill’s postcondition held. It is cheap, local and usually answerable from the skill’s own report plus the gripper state, and it catches the loud failures: the grasp that missed, the arm that faulted.

The goal check asks whether the task is done. It is a separate question, it has to be written by you, and it must not route through the planner that wrote the plan - because a planner able to reliably tell you its plan was wrong would not have written the wrong plan.

Replan: from what, exactly?

Here is the bug that survives code review. A step fails, so the loop replans. Replans from what?

If the answer is “from the plan’s own record of the world”, the loop is now reasoning about a desk that stopped existing several seconds ago. It will confidently generate steps for objects that have moved, re-pick things already in the tray, and undo its own progress - all while every individual call succeeds, so the repeat-failure guard you added never fires.

Replanning also cannot fix everything, and knowing which failures it cannot fix is what three ways a plan dies is for. Some world changes are irreversible; a plan that assumed otherwise cannot be repaired by writing a new plan.

Every loop needs a budget and a written memory

Two pieces of scaffolding that are not optional.

A budget, in three hard caps: consecutive failures on the same objective, total skill calls per task, and wall-clock time. A loop with no cap does not fail; it thrashes, with a real arm, in a real room, for as long as you let it. Choosing these numbers is a real trade rather than a formality, and the exercise below makes you measure it; recovery is a choice, not a reflex turns them into a ladder with a cap on each rung.

A written memory of what has already been done, passed explicitly as state rather than left to survive in the context window. In one long-horizon benchmark, failures over 301 failed tasks were attributed roughly 46% to execution, 28% to task planning and 26% to memory. A quarter of all failures were an agent forgetting or misremembering what it had already done - a share nobody predicts in advance, and the cheapest of the three to fix.

Check yourself

Do this

The same fake desk from the last lesson, now with a loop around it. Everything here runs on a laptop in a second, and the point is to measure the difference between the two loop shapes rather than to read about it.

import random

GOAL = {"cup": "tray", "pen": "tray"}
REACHABLE = ("desk", "floor", "tray", "gripper")

class Desk:
    def __init__(self, seed=0, slip=0.3):
        self.rng, self.slip = random.Random(seed), slip
        self.where = {"cup": "desk", "pen": "desk"}
        self.held, self.calls = None, 0

    def look(self):                                  # the only truth on offer
        return dict(self.where)

    def pick(self, obj):
        self.calls += 1
        if self.held is not None:
            return False, f"gripper already holds {self.held}"
        if self.where[obj] not in REACHABLE:
            return False, f"{obj} is in the {self.where[obj]}, out of reach"
        if self.rng.random() < self.slip:
            self.where[obj] = "floor"
            return False, f"grasp failed: {obj} on the floor, gripper empty"
        self.where[obj], self.held = "gripper", obj
        return True, f"holding {obj}"

    def place(self, obj, target):
        self.calls += 1
        if self.held != obj:
            return False, f"not holding {obj}"
        self.where[obj], self.held = target, None
        return True, f"{obj} in {target}"

def plan_from(world):                                # stands in for the planner
    steps = []
    for obj, place in world.items():
        if place != "tray" and place in REACHABLE:
            steps += [("pick", obj), ("place", obj)]
    return steps

def run_step(desk, step):
    verb, obj = step
    return desk.pick(obj) if verb == "pick" else desk.place(obj, "tray")

def goal_met(world):
    return all(world.get(o) == t for o, t in GOAL.items())

def run_open(seed):                                  # a for loop over the plan
    desk = Desk(seed)
    for step in plan_from(desk.look()):
        run_step(desk, step)
    return goal_met(desk.look()), desk.calls

def run_closed(seed, budget=12, limit=2, stale=False, disturb=False):
    desk = Desk(seed, slip=0.0 if disturb else 0.3)
    world = desk.look()
    steps = plan_from(world)
    if disturb:
        desk.where["pen"] = "drawer"                 # a human tidies it away
    fails = 0
    for _ in range(budget):
        if goal_met(desk.look()):
            return "done", desk.calls
        if not steps:
            steps = plan_from(world)
            if not steps:
                return "blocked", desk.calls
            continue
        ok, _msg = run_step(desk, steps.pop(0))
        if not stale:
            world = desk.look()                      # monitor: look, never trust
        fails = 0 if ok else fails + 1
        if not ok:
            steps = plan_from(world)
        if fails >= limit:
            return "gave up", desk.calls
    return "out of budget", desk.calls

N = 200
print(f"open loop  : claims done {N}/{N}, desk actually clear "
      f"{sum(run_open(s)[0] for s in range(N))}/{N}")
for limit in (1, 2, 3, 4):
    runs = [run_closed(s, limit=limit) for s in range(N)]
    print(f"closed loop, give up after {limit}: clear "
          f"{sum(r[0] == 'done' for r in runs)}/{N}, "
          f"mean skill calls {sum(r[1] for r in runs) / N:.1f}")
print("world changed, replanning from the first look:",
      run_closed(0, disturb=True, stale=True))
print("world changed, replanning from a fresh look:  ",
      run_closed(0, disturb=True, stale=False))

Run it and you get:

open loop  : claims done 200/200, desk actually clear 91/200
closed loop, give up after 1: clear 91/200, mean skill calls 2.8
closed loop, give up after 2: clear 168/200, mean skill calls 4.3
closed loop, give up after 3: clear 195/200, mean skill calls 4.8
closed loop, give up after 4: clear 198/200, mean skill calls 4.8
world changed, replanning from the first look: ('out of budget', 12)
world changed, replanning from a fresh look:   ('blocked', 3)

Four things to take from those six lines.

1. The open loop reports success every single time and is right 91 times out of 200. That is a shade under the 49% you get by multiplying two 70% grasps together, which is the whole story: with no monitoring, task success is simply the product of the skill success rates, and the report is decoration.
2. Giving up after one failure is exactly as good as not looping at all. 91 out of 200, the same number, because a loop that never retries is a for statement wearing a monitor. Retrying is where the recovery lives.
3. The retry budget is a real trade. Two retries buys 84%, three buys 97.5%, four buys almost nothing more and the mean call count stops rising. That flattening is where you stop; on a real robot the calls are seconds of motion and each retry is another chance to knock something over.
4. Stale replanning is worse than the failure it was recovering from. With the world changed, the fresh-look loop stops after three calls and reports blocked, which is a sentence a human can act on. The stale loop burns its entire twelve-call budget, and the failure counter never reaches its limit, because it spends the whole time succeeding at taking the cup back out of the tray.

Then change three things and watch what happens. Set slip=0.6 and find the retry limit where the trade turns around. Delete the if not ok: steps = plan_from(world) line so the loop retries the failed step blindly instead of replanning, and see how much of the recovery survives. Finally, add a third object that starts in the drawer and make plan_from return steps for it anyway - which is what an ungrounded planner does - and watch a loop that is correct in every other respect fail to terminate for a reason that has nothing to do with its logic.

What you can now do

You can write a planner loop that behaves like a loop rather than a list: one skill call per turn, a step check after every call, a goal check written before the run and evaluated independently of the planner, and a fresh look before every replan. You can say why replanning from a stale world model is worse than the failure it was recovering from, why a repeat-failure counter is blind to that particular pathology, and why a budget needs three caps rather than one. You can also state the loop’s own limit honestly: it recovers well from skills that fail loudly, and it is much weaker at noticing that a plan was wrong from the start.