How you write an action down decides everything else

There are two ways to get a number out of a neural network, pick it from a list or produce it directly, and that choice sets your training cost, your control rate and how much language ability survives.

Where you are. You know what pretraining buys, and you can sketch OpenVLA’s four blocks. Every architecture in the rest of this module is that diagram with the last two blocks swapped out. This lesson is those two blocks.

Say twelve point three

A model has just decided the wrist should move to 12.3 degrees. There are exactly two ways to get that number out of a neural network, and every design in this module is one of them or a deliberate combination.

Pick it from a list. Chop the wrist’s range into 256 slices. Have the network output a score for each slice, softmax them, take one. This is exactly what a language model does for every word it emits, which means the machinery already exists and already works.

Produce it directly. Attach a small network to the last hidden state whose output is a real number, and train it to land near 12.3.

That looks like a plumbing decision and it reads like one in every paper. It is the opposite. It sets the training objective, so it decides whether the pretrained backbone’s representations transfer or get damaged. It sets how many forward passes each control step needs, so it decides whether your robot runs at 5 Hz or 50 Hz. And it sets how expensive your fine-tune is. As of August 2026 the strongest published answer is not either one. It is both, in the same training step, wired so they cannot interfere.

The idea in one paragraph

A robot action is a small block of real numbers - a few dozen timesteps by a handful of joints - and a neural network can emit real numbers in two ways. Discretize them into bins and predict the bin as a token, which reuses the pretrained model’s exact objective and keeps its language ability intact, but costs one sequential forward pass per token at inference. Or attach a continuous action head that outputs the numbers directly in a single pass, which is fast enough for a real control loop but trains with a different loss than the backbone ever saw, and can damage the backbone if you are careless. The field spent 2024 and 2025 discovering that these are not competing answers to one question. They are answers to two different questions - what teaches the backbone well, and what runs fast on the robot - and the current state of the art uses one for each.

What an action actually is here

You already built this in Module 3, so it needs only a name.

Chunking is orthogonal to everything below: you can chunk with tokens or with a continuous head. It matters because it reduces compounding error, and because it lets a slow model drive a fast control loop - predict 50 steps at 5 Hz and execute them at 50 Hz.

Two routes out of the model's last hidden state: a softmax over bins producing tokens that are de-tokenized into a chunk, and a continuous action head producing the chunk directly, with the cost of each annotated

Wider than the screen; scroll it sideways.

Route one: pick from a list

Quantize each action dimension into bins, treat each bin index as a vocabulary token, predict them autoregressively. This is RT-1’s design, RT-2’s design and OpenVLA’s design.

What it buys is unusually clean. The objective is cross-entropy on next-token prediction, which is identical to the objective the backbone was pretrained with. Nothing about the model’s loss surface changes when robot data arrives, so pretrained representations transfer well and the model’s hold on language stays strong. Training is fast and stable, in the way that training language models is fast and stable, because it is the same thing.

What it costs is two-sided.

The first cost is quantization. Robot motion is smooth and strongly correlated between neighbouring timesteps, and uniform binning throws that structure away: it treats each sample as an independent symbol drawn from an alphabet. For slow, coarse motion this is fine. For high-frequency dexterous motion it falls apart, because the interesting signal is exactly the fine variation the bins are averaging over.

The second cost is decode time. Tokens come out one at a time, each needing its own forward pass. A 50-step chunk on a 7-joint arm is 350 tokens, which is 350 sequential passes through a multi-billion-parameter model before the robot may move.

FAST: compress the chunk before you tokenize it

Physical Intelligence published the fix in January 2025. It has two stages and both are borrowed from image compression.

First, apply a discrete cosine transform to the action chunk. This rewrites a time-series of 50 samples as 50 frequency components: a slow drift, a slower wobble, and a long tail of fast components that are almost all near zero for real robot motion. Keep and prioritise the low frequencies. Second, run byte-pair encoding over the resulting coefficients, which merges common runs - including all those runs of near-zero high-frequency terms - into single tokens.

The reported result is a large reduction in token count against naive per-sample binning - the figure usually quoted is roughly tenfold, which I did not confirm against the paper, so take the order of magnitude and check the multiple. What is confirmed is that Physical Intelligence released FAST+, a universal robot action tokenizer trained on one million real robot action trajectories, published on Hugging Face as physical-intelligence/fast. So this is a component you can use rather than a technique you must implement.

The honest scorecard, from the authors and dated January 2025 (arXiv:2501.09747): pi0-FAST cuts training time “by up to 5x” against diffusion baselines at similar dexterity - and its autoregressive decoding at inference is significantly slower than flow matching. Compression fixed the quantization problem. It did not fix the sequential-decode problem, because that is inherent to predicting tokens one at a time.

Route two: produce the number directly

Attach a separate module that maps the backbone’s hidden state to real-valued action chunks in a single forward pass. Three flavours are in production as of August 2026.

L1 regression. The simplest possible head: output the numbers, train against mean absolute error. OpenVLA-OFT, from Stanford in February 2025 (arXiv:2502.19645), argues that its limited expressivity is a feature rather than a limitation: an L1 objective commits to the median of the demonstrated behaviours and therefore filters out demonstration noise instead of reproducing it. Combined with parallel decoding and action chunking, the authors report 97.1% average on LIBERO against base OpenVLA’s 76.5%, at 26 times the action-generation throughput. Those are the authors’ own numbers on a simulation benchmark.

A diffusion transformer head. A stack of transformer blocks that turns noise into an action chunk conditioned on the backbone’s output. GR00T uses one. Read its layer count off the release you are actually running rather than off any write-up: NVIDIA’s own repository notes that the head “changes from 32 to 16 diffusion layers” between GR00T N1.6 and N1.7, so the number quoted in most coverage is a version behind.

Flow matching. The dominant choice, and it cuts across the previous row rather than competing with it: pi0, pi0.5 and SmolVLA all use it, and so does GR00T N1.7, whose head NVIDIA describes as a “flow-matching DiT”. “Diffusion transformer” names the shape of the head; flow matching names what it was trained to predict, and the two labels are routinely used as if they were alternatives. It is worth understanding properly rather than by name.

Flow matching, intuition first

Diffusion asks a question that needs many small answers: given a noisy chunk, what noise should I remove? Flow matching asks a simpler one.

Take a real action chunk from your dataset, call it $\mathbf{a}_1$ . Draw a random noise chunk of the same shape, $\mathbf{a}_0$ . Now draw the straight line between them. A point a fraction $\tau$ of the way along is

\mathbf{a}_\tau = (1 - \tau)\,\mathbf{a}_0 + \tau\,\mathbf{a}_1

which just says: mix noise and truth in proportion $\tau$ . The velocity of that straight line - the direction and speed you would travel to get from the noise to the real chunk - is the same everywhere along it:

\mathbf{v} = \mathbf{a}_1 - \mathbf{a}_0

Training picks a random $\tau$ , forms $\mathbf{a}_\tau$ , and trains the network $v_\theta$ to output that velocity given the mixed chunk, the timestep and whatever the backbone saw:

\min_\theta \; \left\| v_\theta(\mathbf{a}_\tau, \tau) - (\mathbf{a}_1 - \mathbf{a}_0) \right\|^2

In words: from wherever you are between noise and a real action, point at the real action.

Flow matching in two rows: training draws a straight line from a noise chunk to a real chunk and learns the line's velocity at a random point on it, and inference starts at noise and takes a small fixed number of equal steps along that velocity

Wider than the screen; scroll it sideways.

Inference starts at pure noise and walks that field forward in $K$ equal steps:

\mathbf{a} \leftarrow \mathbf{a} + \tfrac{1}{K}\, v_\theta(\mathbf{a}, \tau), \qquad \tau = 0, \tfrac{1}{K}, \ldots, \tfrac{K-1}{K}

The reason this matters for robotics is the straight line. Because the training target is a straight path rather than a curved one, the learned field is close to straight, and a straight path needs very few integration steps to follow accurately. Few steps means few forward passes means a control loop that can run at 50 Hz. That is the entire argument, and it is why flow matching displaced diffusion heads in most 2025 and 2026 designs.

Both at once, and the wire between them

Here is the synthesis, and it is the most teachable architectural idea in this module.

Discrete tokens teach the backbone well and run slowly. Continuous heads run fast and teach the backbone badly. Knowledge insulation takes both and blocks the interference.

Get the attribution right, because almost every summary of this gets it wrong and the two papers are five weeks apart. pi0.5 (arXiv:2504.16054, 22 April 2025) trains in two stages: a pretraining stage where the backbone does next-token prediction over text, object locations and FAST-encoded action tokens, then a post-training stage that adds the flow-matching action expert. Its paper describes keeping the two action representations apart with an attention mask - “we use the attention matrix to ensure that the different action representations do not attend to each other” - and does not describe a stop-gradient. Knowledge insulation is the follow-up (arXiv:2505.23705, 28 May 2025, Driess et al.), which in its own words “formalizes the approach of pi0.5 and extends it to develop a single-stage training recipe”, and that is where the stop-gradient appears.

Here is the insulated recipe. The backbone is trained on FAST discrete tokens, using the cross-entropy objective it was pretrained with, which is fast, stable and preserves semantics. Simultaneously, in the same training step, a continuous flow-matching action expert is trained on the same data, and it is what actually runs on the robot. The crucial detail is the wire that is deliberately cut: gradients from the action expert are blocked from flowing back into the backbone by an explicit stop-gradient operator.

Knowledge insulation: one backbone trained by discrete FAST tokens, a flow-matching action expert trained alongside it, and a blocked gradient path so the expert cannot damage the backbone's pretrained representations

Wider than the screen; scroll it sideways.

The reason is worth stating precisely, because it generalises far past robotics. A freshly initialised action expert produces essentially random outputs, so early in training its gradients are noise. Let that noise propagate into a carefully pretrained vision-language backbone and it degrades exactly the semantic representations you paid for. Insulating the backbone means it only ever receives a clean learning signal, from the discrete-token objective it already understands.

The whole choice, on one page

How the action leaves the model	Training objective	What inference costs	Where you meet it
One bin index per dimension per timestep	cross-entropy, identical to the backbone’s own	one sequential forward pass per token; 350 for a 50-step chunk on a 7-joint arm	RT-1, RT-2, OpenVLA
FAST: frequency transform, then byte-pair encoding	cross-entropy, roughly ten times fewer tokens	still sequential, still the slow branch	pi0-FAST
L1 regression head	mean absolute error against the demonstrated chunk	one forward pass	OpenVLA-OFT
Flow-matching action expert	match the straight-line velocity from noise to the real chunk	a small fixed number of integration steps	pi0, pi0.5, SmolVLA
Flow-matching diffusion transformer	the same target, in a deeper stacked head	four integration steps in GR00T N1.7’s published figure	GR00T N1.7 (16 diffusion layers; N1.6 had 32)
Discrete for the backbone, flow matching for the robot	both, with the gradient path cut between them	the flow-matching cost only	knowledge insulation (pi0.5 does the same split across two stages, without the stop-gradient)

Two things to carry out of this table. First, ACT - the policy you already trained - is a continuous chunk regressor, so you have shipped route two already and the unfamiliar branch is the token one. Second, when you read that a model “runs at 30 Hz”, the number is almost entirely a statement about this table row and almost not at all about the size of the backbone.

Check yourself

1. Why does predicting actions as discrete tokens preserve a pretrained model’s language ability better than bolting on a regression head?

Because the objective does not change. The backbone was pretrained with cross-entropy over a vocabulary, and token-based actions keep exactly that loss, so the gradients arriving during robot training have the same character as the ones that built the representation. A regression head introduces a different loss, computed by a module that starts out random, and the gradients it sends backwards early in training are noise with respect to everything the backbone knows.

2. A 7-joint arm, a 50-step chunk, one token per joint per timestep. Why can this not run a 50 Hz control loop, regardless of hardware?

Because 350 tokens means 350 sequential forward passes, and sequential means they cannot be batched or parallelized away. A 50 Hz loop gives you 20 milliseconds per chunk, so each pass would have to complete in about 57 microseconds, which no multi-billion-parameter forward pass does. Action chunking softens this, because you predict 50 steps and then execute them while the next chunk is computed, but the decode itself is still the wall.

3. FAST applies a discrete cosine transform before tokenizing. Why does that particular transform help for robot actions specifically?

Because robot trajectories are smooth, so almost all of their energy sits in the lowest few frequency components and the high components are near zero. A frequency representation therefore concentrates the signal into a few large numbers and a long tail of nearly-identical small ones, which is exactly the shape byte-pair encoding compresses well. The same transform applied to genuinely high-frequency data would save nothing, which is why this is a claim about robot motion rather than about transforms.

4. State the flow-matching training target in one sentence, without using the word “diffusion”.

Mix a real action chunk with a random noise chunk in some random proportion, then train the network to output the direction and speed that would carry that mixture the rest of the way to the real chunk. Because the path is defined as a straight line, the target velocity is simply the real chunk minus the noise chunk, and it is the same at every point along the path.

5. Why does flow matching need fewer inference steps than a diffusion sampler, and why does the robot care?

The training target is a straight-line path from noise to data, so the learned velocity field is close to constant along each path and a coarse integrator follows it accurately. Curved paths need small steps to stay on them; straight ones do not. The robot cares because each integration step is a forward pass, and the number of forward passes per chunk is what sets the achievable control rate. Fewer steps is directly more hertz.

6. Under knowledge insulation, gradients from the action expert are blocked from reaching the backbone. What breaks if you connect them, and what would you lose if you removed the action expert instead?

Connect them and the randomly initialised expert’s early gradients propagate into the pretrained vision-language backbone and degrade the semantic representations that were the reason for using a pretrained model at all. Remove the expert and you are back to autoregressive token decoding at inference, which is too slow for a fast control loop. The two halves answer different questions - what teaches the backbone cleanly, and what runs fast on the robot - so the design keeps both and cuts the wire between them.

Do this

Twenty minutes, numpy only, no GPU.

1. Measure the compression yourself. This builds one plausible action chunk, tokenizes it both ways and reports the error in degrees, which is the unit your arm actually cares about.

import numpy as np

H, D, BINS = 50, 6, 256           # 50 timesteps, 6 joints, 256 bins per number

t = np.linspace(0, 1, H)
chunk = np.zeros((H, D))          # a plausible smooth reach, in radians
for j in range(D):
    chunk[:, j] = (0.6 * np.sin(2 * np.pi * (0.5 + 0.2 * j) * t + j)
                   + 0.2 * np.sin(2 * np.pi * 1.0 * t + 2 * j))

def rms_deg(err):
    return np.degrees(np.sqrt(np.mean(err ** 2)))

def dct_matrix(n):                # orthonormal DCT-II: M @ M.T is the identity
    k, i = np.arange(n)[:, None], np.arange(n)[None, :]
    m = np.cos(np.pi * (i + 0.5) * k / n)
    m[0] *= np.sqrt(1 / n)
    m[1:] *= np.sqrt(2 / n)
    return m

tokens = np.round((chunk + 1) / 2 * (BINS - 1))
print(f"one bin per sample   {tokens.size:3d} tokens   rms "
      f"{rms_deg(tokens / (BINS - 1) * 2 - 1 - chunk):6.3f} deg")

M = dct_matrix(H)
coeffs = M @ chunk                # row 0 is the slowest component
scale = np.abs(coeffs).max()
for K in (4, 6, 8, 12):
    kept = np.round(coeffs[:K] / scale * 127)
    back = np.zeros_like(coeffs)
    back[:K] = kept / 127 * scale
    print(f"lowest {K:2d} frequencies {kept.size:3d} tokens   rms "
          f"{rms_deg(M.T @ back - chunk):6.3f} deg")

It prints 300 tokens at 0.127 degrees for naive binning, and 48 tokens at 0.891 degrees keeping the lowest 8 frequencies - a sixth of the tokens for an error under a degree. Now decide whether that trade is good, which is a hardware question rather than a maths one: your arm’s own backlash eats the first fraction of a degree after every reversal, so a token budget spent buying 0.127 degrees is buying precision the mechanism cannot deliver.

2. Break it on purpose. Add a fast tremor to the chunk with chunk += 0.02 * np.sin(2 * np.pi * 12 * t)[:, None] and re-run. Naive binning barely notices, moving from 0.127 to 0.131 degrees, because it never assumed the signal was smooth. Every frequency-domain reconstruction gets worse - the 8-coefficient one goes from 0.891 to 1.201 degrees - and adding coefficients now buys less than it did. You have reproduced, at toy scale, the reason FAST is described as prioritising low frequencies rather than as lossless.

3. Note what this exercise leaves out. It implements only the first of FAST’s two stages. The byte-pair encoding that follows is where a large part of the reported tenfold compression comes from, because runs of near-zero high-frequency coefficients merge into single tokens. Write one sentence predicting how much extra compression BPE should buy on your coefficient array, then look at how many of your coefficients rounded to zero. The prediction is the exercise.

What you can now do

You can name the two ways an action leaves a model and say what each costs at training time and at inference time. You can explain why discrete tokens protect a pretrained backbone’s language ability, why frequency-domain tokenization compresses robot trajectories specifically, and why flow matching needs few enough integration steps to hold a 50 Hz control loop. And you can describe knowledge insulation - discrete supervision for the backbone, a continuous expert for the robot, and a deliberately cut gradient path between them - and say which paper it actually comes from, which is not the one most write-ups credit.