Where you are. You have a policy trained on a hundred million simulated steps. This lesson is the only step that matters to anyone outside the simulator, and the one where the honest evidence is much narrower than the enthusiasm.
It worked. Then it fell over.
The quadruped trots beautifully in simulation. Ten thousand evaluation episodes, every command, every terrain. You flash it onto the robot, put it on the floor and let go.
It takes one and a half steps and slams into the ground.
You check the obvious things. The joint order is right. The units are right. The control frequency is right. The policy is producing sensible-looking numbers. The robot is simply not doing what the simulation said it would.
Here is what was different. On the real machine, the joint angles the policy reads are about thirty milliseconds old by the time they arrive, and the commanded torque takes a few more milliseconds to actually appear at the joint. In simulation both were instant. The policy learned a fast, tight, high-gain reaction because in its world a correction landed immediately. On hardware, every correction arrives late, so it corrects the situation the robot was in a moment ago, overshoots, corrects harder, and resonates itself into the floor.
Nothing about the physics was wrong. One number that nobody modelled was.
The idea in one paragraph
The instinct is to fix the simulator: measure the latency, measure the friction, measure the inertia, and make the model right. That work never finishes, because there are always more parameters and each is only right on the day you measured it. Domain randomisation takes the opposite route. Do not try to hit reality with one simulator. Randomise the simulator’s parameters - friction, mass, motor gains, latency, sensor noise, terrain - over ranges wide enough that reality is plausibly one draw from that distribution. Then train across the whole family. The policy cannot rely on any particular value of anything, because that value changes every episode, so it learns behaviour that works for the whole family. If your ranges were wide enough, reality is inside the family and the policy has already seen something like it.
Wider than the screen; scroll it sideways.
Watch it happen, and watch what it costs
The mechanism is easier to believe once you have measured it, and it fits on the Module 1 pendulum.
code/domain_randomization.py takes that gravity-loaded joint and turns it into a family: link mass, joint damping, and how many control steps late the measurement arrives. It tunes two PD controllers over exactly the same grid of gains. One is tuned on the nominal plant alone, which is what you do when you trust your simulator. The other is tuned on the average over sixteen randomly drawn plants. Then both are deployed across a sweep of plants neither of them was tuned for.
Here is what came out on my machine, in radians of settling error, lower being better:
| on the nominal plant | averaged over the randomised family | |
|---|---|---|
| tuned on nominal only | 0.0245 | 0.9764 |
| tuned on the randomised family | 0.0850 | 0.1313 |
Read both diagonals. The nominal-tuned controller is the best thing there is on the plant it was tuned for, and it is catastrophic off it - 0.98 radians is not “degraded”, it is not controlling anything. The randomised controller is roughly seven times better across the family.
The latency sweep is the clearest view. The nominal-tuned controller picked the stiffest gain on the grid, , because with instant measurements stiffer is always better. Add 20 ms of delay and its error is 1.49 radians: it is oscillating, exactly like the quadruped in the hook. The randomised controller picked with more damping, and holds 0.085 radians flat out to 40 ms.
That much I measured on a toy plant, and it demonstrates the mechanism. Whether the same conservatism cost is large in practice on real robot policies is a claim I would want evidence for before asserting; I could not find a single canonical paper that quantifies it. The indirect evidence is that an entire literature exists on choosing randomisation ranges automatically - work that uses a language model to propose the distributions, and benchmarks comparing online and offline adaptive schemes - and that literature would have no reason to exist if the ranges did not matter.
The other half: the family has to contain reality
Follow the randomised controller past 40 ms in the figure and it degrades too: 0.27 at 50 ms, 0.53 at 60, 1.55 at 80. That is not a failure of the method. The family it trained on had latencies from 0 to 60 ms in it, and past that edge the policy is extrapolating, which nothing has promised it can do.
Four gaps, not one
“The reality gap” is a single phrase covering at least four different problems with four different answers, and conflating them is how projects lose months.
Wider than the screen; scroll it sideways.
Appearance is the best-understood one and the one Tobin’s paper solved. Randomise textures, lighting, camera pose. This is also the only gap that image augmentation touches: colour jitter and sharpness jitter shift what the images look like, and they do nothing whatever about how the robot moves. LeRobot ships those transforms and turns them off by default, with a note that strong augmentation can hurt if it is too aggressive - a reminder that even the well-understood gap has a wrong setting.
Dynamics is where randomisation has genuinely won. Mass, inertia, friction, motor gains, torque limits, external pushes: randomise them and zero-shot transfer of quadruped and humanoid gaits is now routine rather than remarkable. The Playground paper randomises “sensor noise, dynamics properties and task uncertainties”, explicitly including ground friction and payload mass, and reports that all four of its Go1 behaviours transfer to hardware and cope with uneven terrain.
Latency is the one people forget, and it is the one in this lesson’s hook and in the measurement above. A simulator with zero delay does not have a small latency error; it has a structurally different system, and it will teach gains that oscillate the moment they meet a real bus. Latency has to be in the model before it can be randomised.
Contact is the open one, and it is why manipulation lags locomotion. How a solver resolves a squeeze, a slip or a jam is an approximation, and randomising the parameters of a wrong contact model gives you a family of wrong contact models. Recent work on contact-rich transfer says exactly this: subtle differences in contact dynamics can change the outcome of a task entirely. Rigid-body locomotion dynamics are the thing simulators are best at, which is not a coincidence for where sim-to-real works.
Check yourself
1. State domain randomisation’s mechanism in one sentence, without using the word “robust”.
Train across a distribution of simulators wide enough that the real robot is plausibly a sample from it, so the policy cannot depend on any single parameter value and has effectively already been trained on something close to reality.
2. The randomised controller scores 0.085 on the nominal plant where the nominal-tuned one scores 0.0245. Why is that the expected result rather than a sign of a bad experiment?
Because they were optimised for different objectives. The nominal-tuned controller minimises error on exactly one plant, so nothing can beat it there. The randomised controller minimises the average over sixteen plants, so it accepts a worse score on any individual one in exchange for not failing on the others. Being beaten on the single plant is the definition of the trade, not evidence against it.
3. Your simulator has no latency at all. Why is randomising mass and friction more widely not a substitute?
Because latency is not a parameter of the same system; it changes the structure of the loop. With a delay, feedback acts on a stale measurement, which is what turns a high gain from good into unstable. No amount of variation in mass or friction ever produces that behaviour, so a policy trained without delay never sees the failure mode and never learns to avoid it. A missing phenomenon cannot be randomised.
4. Your vision-based grasping policy fails on the real robot. Someone suggests turning up the colour jitter. When is that the right call and when is it a distraction?
It is right when the failure is that the policy does not recognise the scene: wrong lighting, different table colour, unfamiliar textures. It is a distraction when the policy locates the object correctly and then mishandles the contact - closing too early, slipping, jamming. Augmentation shifts appearance only. If the perception is fine and the interaction is wrong, no amount of image augmentation will touch it.
5. Why is zero-shot transfer routine for quadruped gaits and unreliable for contact-rich manipulation?
Because the gap that randomisation closes is not the gap that matters in each case. Locomotion depends mostly on rigid-body dynamics and intermittent foot contact, which simulators model well, so the real robot lands inside a randomised family of plausible mass, friction and gain values. Manipulation depends on how a contact patch actually behaves under squeeze, slip and deformation, which is approximated rather than modelled, so a randomised family of a wrong model does not contain reality no matter how wide you make it.
6. You have two weeks and a real robot. What do you measure before choosing randomisation ranges, and why measure at all if you are not trying to match?
Measure the parameters you can cheaply pin down - control-loop latency, actual mass with payload, roughly what friction the surface gives - not to match them but to check they are comfortably enclosed by your ranges, and to catch anything you forgot to model at all. A range that does not contain the truth gives you a policy that is confidently robust in the wrong neighbourhood, which is worse than not randomising, because it also costs you performance.
Do this
1. Run the transfer experiment. Fill in randomised_family and deployment_sweep in code/domain_randomization.py and run it. It takes about ten seconds. Confirm the two diagonals: nominal-tuned wins on nominal and collapses off it; randomised-tuned loses on nominal and holds across the family.
2. Find the edge of the box, then find the price of widening it. Two experiments, five minutes:
- Narrow the delay range in your family to
0..2steps and re-run the latency sweep. The randomised controller should now break not far past where the nominal one does. That is the family failing to contain reality. - Widen it to
0..40steps, far past anything real, and look at what happens to the nominal-plant column. That is over-randomisation, measured on your own machine. Write down both numbers; the pair of them is the argument for choosing ranges rather than maximising them.
3. Write the deployment checklist for a robot you might actually own. For the SO-101 arm coming in Module 4, list every parameter where your simulator and the real arm will differ, and mark each one: randomise it, measure it, or accept it. Then mark which of the four gaps each one belongs to. You will find that the ones you are least sure how to classify are contact-related, which is exactly where the field is least sure too.
What you can now do
You can state what domain randomisation actually does - convert a fidelity problem into a coverage problem - and explain why deliberately unrealistic simulators can transfer better than carefully identified ones. You have measured both halves of the trade on your own machine: the robustness a randomised family buys, and the performance it costs on the nominal plant. You can separate the appearance, dynamics, latency and contact gaps, say which fix belongs to each, and give a defensible answer to why legs cross the gap reliably and hands do not.
Next, the decision itself: given a task in front of you, which of the three approaches you have now built deserves it.