Feedback control: measuring the mistake instead of predicting it

Where you are. You can work out which joint angles put the hand where you want it. Nothing so far makes a motor actually arrive there. This lesson is the bridge from geometry to motion.

The shower you cannot set once

Your shower has one lever. Yesterday, three quarters of the way round was perfect, so this morning you set it to three quarters of the way round and step in.

It scalds you. Somebody ran the dishwasher, or the boiler is cold, or it is February. So you do what everyone does: you put a hand in the stream, feel, nudge the lever, feel again, nudge less, and after a few seconds it is right.

Notice what you never learn. You do not, over years of showering, converge on the correct lever angle. There isn’t one. What you learn instead is the procedure: put a hand in the water, compare it with what you wanted, move the lever in the direction that helps, repeat. That procedure works on a shower you have never used before, in a hotel, in the dark.

Now the same shape in a machine. Set your car’s cruise control to 70 and drive at a hill. If the car held the throttle where it was, it would slow down; you would feel the hill. It does not slow down, because it is not holding the throttle. It is holding the speed, and it does that by continuously changing the throttle. The number the driver cares about is held constant by refusing to hold anything else constant.

That is the whole idea, and it is the thing that separates a robot from a machine that repeats a motion.

The idea in one paragraph

There are two ways to make a machine do what you want. You can compute the command in advance from everything you know about the world and send it, which is open loop, or you can measure what actually happened, compare it against what you wanted, and correct, which is feedback. Open loop is precisely as good as your model of the world and no better; the moment the load changes or somebody knocks the table, the error is permanent, because nothing in the system is watching. Feedback swaps the requirement “know the world exactly” for the much cheaper requirement “be able to measure your own mistake.” That trade is why almost every actuator in every robot runs a feedback loop, and the rest of this lesson is about what it buys, what it costs, and why the most obvious version of it is not good enough.

The joint that fights back

The test rig for the next three lessons is the simplest thing that still has real physics in it: one motor, one rigid link, one mass on the end. A pendulum.

Its equation of motion, stated rather than derived:

In words: the joint’s resistance to being accelerated, times how fast you are accelerating it, equals your motor’s torque, minus the torque gravity applies through the link, minus friction that grows with speed. Gravity’s share carries a $\sin\theta$, so it is zero when the link hangs straight down and largest when the link is horizontal. Every joint on every arm is this equation plus coupling to its neighbours.

Put numbers on it. The link is 0.4 m long with 0.5 kg on the end, so holding it horizontal takes 0.5 × 9.81 × 0.4 = 1.962 N·m, forever, with no motion to show for it. The motor’s job is never just to move the joint; it is to move the joint and to keep replacing the torque gravity is taking away, every millisecond, for as long as you want the arm to stay up.

Plan it perfectly, and it works

Here is the open-loop version, done properly. You know the equation, so you can turn it round: pick the motion you want, work out the acceleration it needs at every instant, and compute the torque that produces it. Then play that torque out like a tape, blind.

Do that for a smooth two-second move from hanging down to horizontal and the joint lands within 0.04° of 90 and stays there. Open loop is not a straw man. When the model is right, it is excellent: no sensor, no delay, no loop to make unstable.

Then change one number

Now put something in the gripper. Five percent of 500 g is 25 g, roughly a AA battery. Nothing else changes; the same torque tape plays out.

The joint settles at 72.25°, almost 18 degrees low. Ten percent heavier, and it settles at 65.38°.

That number is not mysterious, which is what makes it damning. The plan ends up supplying a constant 1.962 N·m, and the joint comes to rest wherever the real gravity torque happens to equal it: $m_{\mathrm{real}}\,g\,l\sin\theta = 1.962$. Solve it and you get 72.25 degrees. The arm is not broken, or badly tuned, or noisy. It is sitting exactly where the arithmetic says it must, and it will sit there all day.

And a disturbance is worse than a load. Nudge the arm while it holds at 90 degrees, and it does not sag back. It goes over the top and keeps turning; in the two seconds after the shove it makes more than four full revolutions. The reason is worth sitting with: $\sin\theta$ peaks at 90 degrees, so the horizontal hold is the hardest gravity ever pulls. Push past it and gravity’s pull falls while the motor’s push stays exactly where the tape says. Past that point the motor is helping the arm run away.

One wire

The fix is one signal, and the diagram is the whole lesson.

Read the encoder. Subtract. Act on the difference. Repeat.

error = target - measured        # every millisecond, forever
torque = some_rule(error)

The dumbest rule that works

You do not need PID to get the benefit. Try the crudest control law there is: if the joint is below the target, full torque up; if it is above, full torque down.

torque = TAU_MAX * sign(error)

That is a thermostat. It is called bang-bang control, and on this plant it works: the joint swings up, swings for a while, and settles down around 89.72°, averaged over the last half second of a twenty-second run.

Then make it ten percent heavier, the change that cost open loop 25 degrees, and run it again. The joint holds at 89.91°. The extra mass is worth under two tenths of a degree, and both figures are averages taken inside a buzz wider than the gap between them.

That is the trade, and it is the reason feedback runs the world. You gave up needing an accurate model of the plant, and in exchange you took on needing a measurement and a loop.

Two bills arrive

Bang-bang is not the end of the story, because it fails in two ways that both matter on real hardware.

The first is that it never sits still. Holding at 90 degrees, the motor slams from full torque one way to full torque the other, for ever, and the reversals get faster as the swing decays: about 4 a second five seconds in, 21 a second at twenty seconds, and 107 a second by forty, climbing towards half the loop rate. It is never applying 1.962 N·m; it is applying +3 and −3 in a blur that averages out to roughly the right thing. On a real servo that is heat in the windings, wear in the gear teeth, and an audible buzz.

The second is subtler and more important. How accurately bang-bang holds depends entirely on how often it looks.

At 1000 Hz the joint does eventually hold within ±0.24° of target, but the decay is slow: it is still swinging ±1.6° at eleven seconds and does not stay inside a quarter of a degree until about 38 s. At 200 Hz it settles into a permanent ±12.4° wobble. At 50 Hz it swings by forty degrees and more, erratically, and never stops. Same rule. Same code. Same plant. Only the interval between looks changed.

What is actually missing

Look at what bang-bang throws away. It uses the sign of the error and discards the size. Forty degrees off and four tenths of a degree off produce exactly the same command: everything you have got.

That single fact explains both failures. It buzzes because near the target it cannot push gently, so it can only overshoot and come back. It depends on loop rate because the only thing limiting the overshoot is how quickly you notice.

The repair suggests itself. Push in proportion to how wrong you are: hard when far, gently when close. That one change is the next lesson, and it turns out to be both a genuine fix and the source of a brand new problem, which is where the other two letters come from.

Check yourself

Do this

Before you write a real controller, write the worst one that could possibly work. About fifteen minutes.

Open code/pid_pendulum.py. Ignore the TODO(you) for now and read simulate() instead. Find the four lines that are the entire loop: read the state, compute the command, apply the physics, advance time. Everything in this module’s control lessons happens inside those four lines.

Now set T_END = 20.0 at the top of the file, because this controller takes its time, and replace the body of PID.update with the dumbest rule there is. The every counter lets you slow the loop down without touching the physics step:

def update(self, err):
    if self.n % self.every == 0:                # only look every `every` ticks
        self.held = TAU_MAX * np.sign(err)      # full torque, whichever way helps
    self.n += 1
    return self.held

Add self.n, self.held, self.every = 0, 0.0, 1 to __init__ and to reset. Run it. All three curves are identical, because the gains no longer do anything, and all three end up holding near 90 degrees; the script should print about 89.72°. Then:

1. Change M from 0.5 to 0.55 and run again. It prints about 89.91°. Do not read anything into which of the two is nearer 90: the difference between them is smaller than the buzz the controller is sitting in, and the printed figure is an average over half a second of that buzz. The point is that a ten percent mass change moved it by a fraction of a degree, when open loop lost 25 degrees to exactly the same change, and your controller has never been told what M is.
2. Put M back and set self.every = 20, which makes it a 50 Hz loop on a 1 kHz simulation. It now prints around 51° and the plot never settles. Try every = 5 for 200 Hz and you get about 80°. Write one sentence explaining why looking less often makes the same rule hold worse.
3. Predict, before you run it, what happens if you replace TAU_MAX * np.sign(err) with err * 8.0. Then run it and see whether the joint reaches 90 degrees at all. Whatever you find is the subject of the next lesson.

Put the file back to raise NotImplementedError and T_END = 5.0 when you are done, or keep a copy; you will implement it properly next.

What you can now do

You can say what open-loop control is, demonstrate that it works when the model is right, and predict exactly where it lands when the model is wrong. You can write the sense-compare-correct loop from memory, explain why it replaces a requirement to know the world with a requirement to measure a mistake, and name the two prices a crude version pays: a command that never settles, and accuracy that is hostage to loop rate. Most usefully, you can say what a control law is missing when it uses only the sign of the error, which is exactly the gap the next lesson fills.