The fast/slow split: three clocks, not two models

Where you are. You know what pretraining buys, where vision-language-action models came from, and how an action gets represented. This lesson is the structural constraint every one of those models is bent around, and it is the last piece you need before opening the architectures one at a time.

Stand on one leg and name five countries

Do it now, properly. Stand on one leg. While you are up there, name five countries beginning with B.

Two things happen and they do not interfere. The naming is slow and effortful; you can feel yourself searching, discarding, going back. The balancing is fast and invisible; your ankle, hip and eyes are making corrections many times a second and not one of them reaches you.

Now try to balance deliberately. Decide on each ankle correction before you make it. Think the thought, then move. You will wobble within a second or two and probably put your foot down.

That is not one skill failing. It is a fast process being forced to run on a slow clock, which is a different kind of failure and a much more interesting one.

The idea in one paragraph

Questions in robotics have deadlines, and the deadlines are set by physics rather than by design taste. “What is this instruction actually asking for” can take a second; nothing bad happens while you think. “How much current goes into this motor right now so the foot does not slide” has to be answered every millisecond or the robot is on the floor. No single network is good at both ends of that range, so every serious system splits into layers, each running on its own clock, joined by a narrow interface. Module 0 named the split for you as System 2 and System 1. This lesson is the engineering underneath the name: why the split is forced rather than chosen, what actually crosses the line, and the one trick that lets a model far too slow for the control loop drive it anyway.

The deadlines belong to the world

Read the ladder from the bottom, because that is where the constraint lives.

The servo layer runs at hundreds to a thousand times a second, and Module 0 gave you the reason: contact. A position-controlled joint that meets an obstacle keeps pushing harder until something gives, and the only thing standing between that and a broken gripper is a loop that notices fast. Balance is the same argument with a bigger consequence. A falling humanoid has roughly a few hundred milliseconds of usable warning, and a controller that samples every millisecond gets hundreds of chances to intervene inside that window. One that samples every hundred milliseconds gets two or three.

The semantic layer has no such deadline. Working out that “put the ketchup away” means the fridge and not the cupboard is a question a person will happily wait a second for. Nothing physical degrades while the model thinks.

Between the two sits the layer this module is about. A vision-language-action model is not fast enough to be a servo loop and is wasted running at 1 Hz. Where it lands, and whether it lands anywhere usable, is an arithmetic question rather than an opinion.

What a big model costs you per decision

Take the fastest published number in this module. NVIDIA reports GR00T N1.7 at 35.9 Hz on an H100 with TensorRT, using four denoising steps and a single camera view. That is about 28 milliseconds per decision, on a datacentre accelerator, with every setting turned toward speed.

Twenty-eight milliseconds is a fine number. It is also roughly thirty times too slow for a 1 kHz balance loop, and the H100 it was measured on is not bolted to the robot.

Action chunking: how a slow model drives a fast loop

The trick is to stop asking the model for one action at a time.

A chunked policy predicts $H$ future actions in a single forward pass. π₀ uses $H = 50$. If the robot consumes those at 50 Hz, one forward pass has bought a full second of motion. Your Module 3 policy already did this: chunking is not a foundation-model invention, it is the same mechanism ACT used, scaled up.

Chunking buys two separate things and they are worth separating. The first is arithmetic: fewer forward passes per second of motion, so a slow model becomes affordable. The second is behavioural: the policy commits to a short plan instead of re-deciding from scratch every step, which stops it from stuttering between two equally good ways of doing the task. That second benefit is why chunking helps even when compute is free.

The cost is written on the diagram. Inside a chunk you are running open loop. The robot is executing a plan made from a photograph of the past, and if the mug moved, nobody knows yet. So chunk length is a dial between reactivity and compute, and in practice you rarely play a whole chunk before replanning.

Three places to draw the line

Every design in this module splits fast from slow. They disagree about where the boundary goes and, more importantly, about what crosses it.

• Two experts, one transformer. π₀ and π₀.₅ put a vision-language expert and an action expert inside a single shared self-attention stack. One forward pass, one set of weights on disk, and what crosses the line is attention itself. The next lesson opens this one up.
• One backbone, one action head. GR00T N1.7 runs a vision-language backbone that hands a hidden state to a 16-layer diffusion transformer, trained with flow matching, which produces the actual motor commands. What crosses the line is a tensor.
• Two models, two machines. Gemini Robotics splits the halves into separate models: a reasoning model plans in language and emits a sentence, and a vision-language-action model turns that sentence into motion. What crosses the line is text you can read in a log. Lesson 6 takes both of these apart.

And Figure’s Helix adds a third rung below all of them, a small controller running at 1 kHz whose only job is to keep the robot standing. Lesson 7 covers it, and it is the clearest illustration in the field that these are clock domains rather than fashion.

Check yourself

Do this

Fifteen minutes, no GPU required.

1. Work out your own budget. Save this and run it.

latency_s  = 0.030   # one forward pass of your policy, measured
chunk_H    = 50      # actions the policy emits per forward pass
control_hz = 50      # rate your arm accepts commands

chunk_s = chunk_H / control_hz
print(f"one forward pass buys {chunk_s:.2f} s of motion")
print(f"blind interval if you play the whole chunk: {chunk_s:.2f} s")
print(f"share of wall-clock spent inferring, synchronous: "
      f"{latency_s / (latency_s + chunk_s):.1%}")

for replan_s in (0.10, 0.25, 0.50, 1.00):
    executed = min(replan_s, chunk_s)
    print(f"replan every {executed:.2f} s -> {1 / executed:5.1f} forward passes/s, "
          f"needs latency under {executed:.3f} s to keep up synchronously")

2. Measure the real number. Time a single forward pass of the ACT policy you trained in Module 3, on the machine you actually own, averaged over a hundred calls with the first ten discarded as warm-up. Put that number into latency_s and run the script again.

3. Answer one question in writing. At your measured latency, what is the fastest you can replan without going asynchronous, and is that fast enough for a task where the object can be nudged mid-episode? You will need this answer again when you fine-tune SmolVLA later in this module, and it is the difference between a policy that recovers and one that plays out a plan into empty air.

What you can now do

You can explain why robot architectures split into layers running at different rates, and point at the physical fact that sets each deadline rather than waving at System 1 and System 2. You can tell rate from latency, compute how much motion one forward pass buys, and say what you gave up to get it. And you can name the three places the field draws the fast-slow line, which is the map for the next three lessons.