Where you are. Module 0 handed you a robot split into a fast half and a slow half. Modules 1 through 5 built the fast half: kinematics, a controller, a simulator, a trained policy, a fine-tuned VLA. This lesson lays out the slow half, and the first thing it does is add a tier, because “fast and slow” turns out to be one short.
One second, while the arm is in the air
Put a stopwatch on the robot.
The gripper closes around a mug at 12:00:00.00. The wrist camera frame that shows it closed is stamped 40 milliseconds later. The model that asked for the grasp learns about it at 12:00:01.3.
A second and a bit of blind time. In an agent loop on your laptop, that number is so small you would not bother instrumenting it. Here it is a full second in which a robot arm holds a ceramic object thirty centimetres above a desk and the thing that decided to pick it up has no idea what happened.
Except that nothing was waiting. The arm was already lifting, because it had been told to lift some time ago and lifting takes longer than a second. The model’s silence cost nothing at all.
That is not luck, and it is not a scheduling trick you bolt on afterwards. It is the shape of the entire system, and getting it right is what this module is about.
The idea in one paragraph
An agentic robot is not two brains. It is three loops nested by rate. At the bottom a controller holds each joint where it was told, two hundred to a thousand times a second. Above it a learned policy turns pixels and joint angles into the next fraction of a second of motion, ten to two hundred times a second. Above that a planner, an LLM or a VLM, decides what should happen next and whether the last thing worked, roughly once or twice a second at best. Each tier runs about a hundred times slower than the one below it. Very little that is interesting happens inside a tier. Almost everything interesting happens at the boundaries: what representation crosses them, and how the slow tier stays off the fast tier’s critical path.
Wider than the screen; scroll it sideways.
Three tiers, not two
Conflating the middle and the bottom is the most common mistake in writing about this architecture, and it hides the fact that the two boundaries are completely different problems.
| Tier | One decision takes | Rate | If it is late |
|---|---|---|---|
| Planner | 0.3-3 s | 0.2-2 Hz | the robot keeps doing the last thing it was told |
| Policy | 3-50 ms | 10-200 Hz | motion stutters, or a chunk runs out mid-reach |
| Controller | 1-5 ms | 200-1000 Hz | the joint is uncontrolled, and that is a safety event |
Read the right-hand column downward. The consequence of lateness gets worse the further down you go, and only the bottom row is unrecoverable. This is the asymmetry the architecture is built around: the bottom tier must emit something every two milliseconds even when it has nothing new to say, and the top tier is permitted to take a second and come back with something better.
What crosses the boundary
Here is the question that actually separates one architecture from another. When the planner has decided what should happen next, what object does it hand down?
Two answers have shipped, and they are genuinely different bets.
Wider than the screen; scroll it sideways.
The other bet passes words. Google’s Gemini Robotics 2 family, released in July 2026, splits into a VLA that acts and an embodied-reasoning model, Gemini Robotics-ER 2, that orchestrates. Its model card states that ER 2 is built on Gemini 3.5 Flash, takes text, images, video and audio, and carries a context window of up to 128k. Google describes it as observing the room, reasoning about the steps needed, coordinating with the VLA to carry them out, and tracking progress until the task is done. It watches video continuously, which lets it notice a step that failed and retry that step rather than restarting the task.
The trade is clean enough to memorise. Text is inspectable and correctable; a latent vector is faster and carries more. If the intermediate representation is words, you can log it, diff two runs, put it in a demo, and let a human overwrite it while the robot is moving. If it is a vector, you get more bandwidth and no reading. Every recovery, debugging and safety mechanism in the rest of this module assumes you can read what the planner asked for, which is a bias worth naming out loud rather than pretending is a law.
Does the hierarchy earn its keep
It is fair to ask whether any of this beats one big model that takes the instruction and emits joint commands. What Matters in Orchestrating Robot Policies: A Systematic Study of Hierarchical VLA Agents (Hu et al., arXiv:2606.10267, June 2026) measured exactly that, comparing a carefully designed hierarchy against a naive one and against a flat policy.
| Setting | Best hierarchy | Naive hierarchy | Flat policy |
|---|---|---|---|
| Short horizon | 78.2 | 69.6 | 69.6 |
| Long horizon | 67.1 | 40.6 | 25.3 |
| Reasoning-heavy | 80.9 | 66.5 | 50.9 |
| Real robot placements | 12 / 15 | 9 / 15 | 3 / 15 |
On short tasks the hierarchy barely matters. On long ones it is the difference between finishing two thirds of the time and finishing a quarter of the time. That gradient is the argument for this module: the longer the horizon and the more reasoning a task needs, the more the split pays.
The planner is not in the loop
One more thing to fix in place before the rest of the module, because everything downstream depends on it.
Wider than the screen; scroll it sideways.
In one second the controller makes about five hundred decisions, the policy makes about thirty, and the planner makes one. The planner is not a slow controller. It is not in the loop at all, in the control-theory sense, because a loop that closes once per second cannot correct anything that moves.
Check yourself
1. Why is “the fast layer and the slow layer” not enough to describe an agentic robot?
It merges the policy and the controller, which sit two orders of magnitude apart and fail in completely different ways. A late policy chunk makes motion stutter. A late controller tick means a joint is momentarily uncontrolled, which is a safety event. They also have different interfaces: the policy consumes pixels and emits target angles, and the controller consumes target angles and emits currents. Naming three tiers keeps two separate boundary designs from being confused for one.
2. The planner goes quiet for a second. Why is the robot usually fine, and when is it not?
It is fine when the skill currently running lasts longer than a planner call, because the arm is still executing an instruction it already has. It stops being fine when the current skill finishes first, in which case the robot has nothing to do and idles, or when the world changes in a way the running skill cannot react to. The second case is the dangerous one: the arm carries on confidently into a situation nobody has looked at recently.
3. Helix passes a latent vector between its two models, and Gemini Robotics 2 passes language. What do you gain and lose with each?
The vector carries more information per unit time, needs no decoding step, and can be trained end to end with the policy. But there is nothing to read, so you cannot log it usefully, cannot diff two runs, and cannot let a human overwrite it mid-episode. Language is lower bandwidth and needs the policy to be conditioned on text, but it is inspectable and correctable, which is what makes debugging, human intervention and audit possible.
4. A hierarchical system and a flat policy score about the same on short tasks but very differently on long ones. Why would that be?
Short tasks need little decomposition, so there is nothing for the top tier to contribute and its latency and error are pure overhead. Long tasks require ordering, tracking what has already been done, and noticing that a step failed. A flat policy has to hold all of that implicitly in its own activations across a long episode, and errors compound. The hierarchy makes the task structure explicit and gives failures somewhere to be caught between steps.
5. Someone proposes closing a visual servoing loop through the planner: look, ask the model for a small correction, move one centimetre, repeat. What breaks?
The loop rate. One turn costs a planner call, so the loop runs at roughly 0.5 Hz. Anything that moves faster than a couple of centimetres per second is uncorrectable, and the correction that eventually arrives was computed from a picture two seconds old. Visual correction belongs at the policy tier, where the loop turns every 30 milliseconds. The planner’s job is to decide that a correction is needed and which skill to invoke, not to be the corrector.
Do this
Fifteen minutes, pure Python, no dependencies.
1. Feel the ratio. Run this.
WINDOW_S = 5.0
TIERS = [("planner", 0.8), ("policy", 30.0), ("controller", 500.0)]
for name, hz in TIERS:
print(f"{name:11s} {hz:6.1f} Hz -> {hz * WINDOW_S:7.0f} decisions in {WINDOW_S:.0f} s")Five seconds of robot life. The planner gets four decisions. The controller gets two and a half thousand. Every architectural rule in this module is downstream of that gap.
2. Break the layering on purpose. Now make the fast tier wait on the slow one, which is what happens the first time somebody wires a model call into a control loop.
POLICY_PERIOD_S = 1 / 30
PLANNER_CALL_S = 1.2
OBJECT_SPEED_MS = 0.3
free_hz = 1 / POLICY_PERIOD_S
blocked_hz = 1 / (POLICY_PERIOD_S + PLANNER_CALL_S)
print(f"policy alone: {free_hz:6.2f} Hz")
print(f"policy waiting on a call:{blocked_hz:6.2f} Hz")
for label, hz in (("free", free_hz), ("blocked", blocked_hz)):
print(f" {label:8s} a mug at 0.3 m/s moves {100 * OBJECT_SPEED_MS / hz:6.1f} cm between glances")One centimetre becomes thirty-seven. Write one sentence explaining why that number, and not any accuracy benchmark, is the reason the planner is not allowed inside the control loop.
3. Sort five decisions. For each of these, name the tier that should own it and say why in one line: the gripper is slipping, tighten; the mug is not where I expected, look again; this joint is 3 degrees off its target; the bin is full, use the other one; a hand just entered the workspace, stop. One of them is genuinely contested. Decide which, and hold your answer until the safety lesson.
What you can now do
You can name the three tiers of an agentic robot, state each one’s rate, deadline and failure mode, and explain why merging the bottom two hides a real design decision. You can describe the two shipped answers to what crosses the planner-to-policy boundary, say what each buys and costs, and give a reason from measured results why the hierarchy is worth its overhead on long tasks and not on short ones. And you can spot the mistake of putting a planner call inside a loop that needs to correct anything.