Where you are. You can describe a robot as a sense-decide-act loop, you know how the field got here, and you know roughly who is building what. This lesson is about the decide step: who writes it, and the five genuinely different answers in use today.
Somebody else has to make the coffee
You are away for a week, so you leave a note for whoever is covering: mug from the second shelf, under the left spout, press the big button, wait for the beep.
It works on Monday. On Tuesday somebody unloaded the dishwasher wrong and the mugs are stacked on the counter instead. The note is now wrong, and it is wrong in a way that no amount of extra precision fixes. “Second shelf” was never really an instruction. It was a bet about the world.
So you rewrite it: find a mug, put it under the spout. That version survives Tuesday. It has also quietly handed the hard part back, because “find a mug” is not a step. It is an entire field.
And there is a third version you would never write down at all. If the person were standing next to you, you would just make the coffee twice while they watched, then hand them the mug and let them try.
Those are three of the five ways a robot gets programmed. The choice between them has almost nothing to do with how good your code is. It is a decision about how much you are willing to bet on the world holding still.
The idea in one paragraph
There are five distinct ways to fill in the decide step, and all five are in use right now, though the last one is only barely out of the lab. They form a progression: at one end you write out the motion itself; at the other you write down a goal, hand over some data, and let an optimiser work out the motion. Every step along that progression buys tolerance for a world that varies, and pays for it in data, in compute, and in how much you can still say about what the machine will do. Choosing correctly is the first real robotics skill, because the paradigm decides what you build, what you have to collect, and how the thing will fail.
Wider than the screen; scroll it sideways.
Paradigm 1: scripted motion
Hardcode the motion. Move to pose A, close the gripper, move to pose B, open. Perception is usually nothing at all, or a beam sensor that says “a part is present.”
This sounds primitive, and it is what most of the world’s installed robots are doing right now. The trick is that the factory does not make the robot smart; it removes the need. Parts arrive in fixtures that hold them at an exact pose, so there is nothing left to perceive. What you get back is a machine that is deterministic, fast, auditable and cheap to certify, which matters enormously when it is swinging a steel arm near people.
It dies the instant the world varies. A cup three centimetres left of where the script expects is a failure, and there is no graceful degradation on the way there.
You will write one in Module 1, a pick-and-place built out of pure geometry, so that you feel both the power and the brittleness in one afternoon.
Paradigm 2: the classical pipeline
Perception, then planning, then control, each stage hand-designed. Estimate the state of the world from sensors. Compute a path to the goal that avoids collisions. Track that path with a controller. This is textbook robotics, and it is where the famous machinery lives: Kalman filters and SLAM for estimating state, planners such as RRT for paths, PID and MPC for tracking them.
Four acronyms in one sentence, so take them at survey depth now. Two of the four you build yourself later; the other two you only need to recognise.
It wins wherever you can write down what state means. A warehouse robot’s state is a position and a heading on a known floor plan. A drone’s is a pose and a velocity. A surgical robot works inside a body that was imaged in advance. In all of those, hand-designed maths is not merely adequate; it is better than a learned policy, because you can analyse it and bound what it will do.
It struggles with manipulating diverse objects. Try writing down the state of a pile of laundry, then write a planner that folds it. People tried for about three decades. This is Shakey’s architecture, grown up and industrialised, and it inherits Shakey’s boundary: the pipeline is complete exactly as far as the state you wrote down, and not one object further.
Paradigm 3: imitation learning
Show, do not program. A human teleoperates the robot through the task a few dozen times while the system records camera images, joint angles and the commands sent. A neural network then learns the mapping from observation to action. Nothing models the world, nothing plans; the whole policy is “in situations that look like this, do things like that.”
This is what took over arm tasks, and the reason is that it sidesteps the impossible part. Folding laundry stops requiring a model of laundry and starts requiring fifty demonstrations of folding. The representation the network invents for itself never has to be nameable by you, which is precisely why it can cover cases you could not have enumerated.
A few hundred dollars of hardware and an afternoon of demonstrations gets you a working policy for one task. That loop is where you will live during Modules 3 and 4, training the two workhorse algorithms of this paradigm, ACT and Diffusion Policy.
Paradigm 4: reinforcement learning
Demonstrate nothing. Write down a reward instead - plus one for moving forward, minus a lot for falling over - and let the robot discover the behaviour by trial and error. It needs millions of trials, which is why this happens in simulation: thousands of virtual robots practising in parallel, faster than real time. The resulting policy is then moved onto the physical machine, a step with its own name, sim-to-real, and its own long list of problems.
It owns walking. Essentially every modern legged robot learned to walk this way. Rewards for walking are easy to write, rigid-body walking physics simulates well, and falling over in simulation costs nothing but electricity.
It struggles with arm tasks for the mirror-image reasons. What is the reward function for “neatly folded”? And contact, the moment two surfaces touch and deform and slip, is the part of physics that simulators model worst.
Paradigm 5: foundation models
Take the recipe that produced large language models, one big generalist model trained on an enormous and diverse dataset, and point it at robotics. A vision-language-action model, or VLA, takes camera images plus an instruction in English (“pick up the red cup”) and outputs robot actions. It is trained on data pooled from many robots, many tasks and many labs.
A VLA is imitation learning at scale, and the scale is the entire point. Single-task imitation gives you a policy for the objects you demonstrated. Broad pretraining is a bet that a model which has seen a million objects will cope with object number eleven. You then fine-tune it on your own task with a comparatively tiny dataset, which is the same economics as fine-tuning a language model: somebody else already paid for the expensive part.
That fine-tuning workflow, which you meet in Module 5, is probably the single most valuable practical skill in this course.
Choosing between them
The five coexist; a 2026 factory line and a 2026 research humanoid use different ones, correctly. Picking is three questions asked in order, and the first yes decides.
Wider than the screen; scroll it sideways.
The loop the last three share
Whatever the model, learning-based robotics runs one operating loop, and it looks like MLOps with physics bolted on.
Wider than the screen; scroll it sideways.
The word carrying the weight there is evaluate.
Evaluation means a success rate over at least twenty scripted trials, with starting conditions varied on purpose and every failure counted, including the embarrassing ones. It does not mean a video. A demo is an existence proof; a success rate is a measurement, and almost every overstated robotics claim lives in the gap between the two. The teams that ship are the ones that spin this loop fastest and score themselves most honestly.
Check yourself
1. Why does a car factory still use paradigm 1 in 2026, and why is that not a failure of imagination?
Because the factory removed the need for intelligence rather than supplying it: parts arrive at exact poses in fixtures, so there is nothing to perceive and nothing to decide. Scripted motion there is deterministic, auditable, fast and cheap to certify. Learning only adds value where the variability is irreducible. Matching the paradigm to the amount of variation is good engineering, not conservatism.
2. Why does reinforcement learning dominate walking while imitation learning dominates arm tasks? Two reasons each.
Walking: the reward is easy to write down (forward velocity, stay upright) and rigid-body walking physics simulates well, so millions of cheap simulated trials transfer to the real robot. Arm tasks: the reward is hard to specify (what number is “neatly folded”?) and contact with deformable objects simulates badly. But arm tasks are easy to demonstrate by teleoperation, so demonstrations win where rewards cannot be written.
3. Your imitation policy works in the demonstrated region but fails when the cube starts five centimetres outside it. Name the failure and give the cheapest fix.
Distribution shift: the observation is outside the training distribution, and an imitation policy has no recovery behaviour, so the first small error moves it somewhere even less familiar and the error compounds. The cheapest fix is more demonstrations covering that region, including recoveries from deliberate mistakes. Data coverage beats architecture almost every time.
4. Why is “sim-to-real” a constant preoccupation in reinforcement learning and barely mentioned in imitation learning?
Because imitation learning collects its data on the real robot, so there is no reality gap to cross; the data is already in the target distribution. Reinforcement learning needs millions of trials, which are only affordable in simulation, so the policy is necessarily born in a physics approximation and has to survive the move to a world with different friction, latency, backlash and sensor noise. The gap is where most of the difficulty concentrates.
5. A VLA is “imitation learning at scale.” What does the scale actually buy, and what does it cost you compared with training your own single-task policy?
It buys breadth: behaviour on objects, scenes and phrasings that appear nowhere in your own data, because the pretraining distribution is vastly broader than any dataset you could collect. It costs you control and legibility. You inherit somebody else’s data, biases and failure modes, you cannot inspect what the model learned, and when it fails oddly your only real levers are more fine-tuning data or a different base model.
Do this
Twenty minutes, in your notes, no code.
-
Pick three robots you have seen in the last month: a robot vacuum, a humanoid demo video, a factory arm clip. For each, write four lines - which paradigm or mix of paradigms, what the observation is, what the action is, and the first thing that would break it.
-
Go back to the coffee note at the top of this lesson and write the two versions that were missing. What would the paradigm 4 version be, if you could only leave your colleague a scoring rule and a practice kitchen? What would the paradigm 5 version be, if your colleague had already made coffee in a thousand other kitchens? Getting these two into plain English is the fastest way to check that the ladder actually landed.
Anything you are unsure about is a good first grilling session with an AI tutor: paste your four lines and ask it to argue the opposite classification.
What you can now do
You can place any robot system in one of five programming paradigms, name the artifact its builders had to author, predict the way it will fail, and argue for the cheapest paradigm a problem allows rather than the most impressive one it tolerates.