Where you are. Your follower moves under its own power and you have set its limits. This lesson puts the leader arm in your hand and connects the two.
Push the follower and watch your other hand
Both arms are on the desk, a hand’s width apart, powered, and the program is running.
Take the leader by its handle and move it slowly left. The follower goes left. Lift the leader’s elbow and the follower’s elbow lifts. It is uncanny for about ninety seconds and then it feels completely ordinary, which is its own kind of remarkable.
Now do one more thing. With your free hand, take the follower’s forearm and push it gently off course. A second, not a wrestling match.
Two things happen at once. The follower pushes back at you, because it is a position-controlled arm being told to be somewhere it is not. And the leader, still in your other hand, does not move and does not resist. It reports nothing. Your hand cannot tell the difference between a follower swinging through empty air and a follower shoving against your fingers.
That gap is the whole lesson.
The idea in one paragraph
Teleoperation on these arms is a one-way copy of numbers. Tens of times a second the software reads six joint angles off the leader you are pushing around by hand, converts them through both arms’ calibrations into a shared scale, and writes them to the follower as six goal positions. The follower’s servos then do what servos do: drive to the angle, using whatever force that takes. Nothing returns. Contact force, gripper load, a motor running hot - none of it reaches your hand, because there is no wire for it and no sensor producing it. That absence is why teleoperation is cheap to build, why it produces beautifully clean demonstrations, and why the first thing you break will be the gripper.
Two arms that are not the same arm
They look like twins and they are not. The follower carries six identical servos geared 345:1, which is a lot of reduction and a lot of holding torque, and it is the arm that does work. The leader carries a mix: one joint geared like the follower’s, and the other five at the lower ratios of 191:1 and 147:1. Less reduction means less friction and far less reflected inertia at the output, which from Module 0 you already know is the definition of backdrivable. You can push the leader around with two fingers precisely because it was built to be pushed.
While you teleoperate, the leader is not driving anything. You supply the motion; its servos are there to report where you put them. It is a six-axis input device that happens to be shaped exactly like the thing it commands, which is the entire trick: you never solve inverse kinematics, because your arm does it.
What crosses the gap
Wider than the screen; scroll it sideways.
One tick of the loop is four steps: read the six leader positions over its serial bus, map them through the leader’s calibration into a shared scale, map them through the follower’s calibration into the follower’s raw units, write them as goal positions. Then again. Then again.
Five of the six joints travel as angles, in degrees by default. The gripper does not: it is always normalised to a 0 to 100 scale, whatever units the other joints are using. That is a deliberate choice rather than an inconsistency. “Gripper at 34 degrees” tells you nothing useful, because what you care about is how far through its travel it is - which is exactly what a percentage says.
You are the only loop
Wider than the screen; scroll it sideways.
Count the feedback loops on that desk. Inside each servo there is one, closed against its own encoder, running far faster than anything you wrote. Around that sits the host program, which reads numbers from one arm and writes them to another and never looks at the object at all. It closes nothing.
Then there is you. Your eyes are on the brick, your hand is on the leader, and when the gripper comes down slightly off-centre you shift it. That is the only loop in the room that has both the arm and the task inside it.
This matters more than it sounds, and it is the reason this module sits where it does in the course. When you record demonstrations in the next few lessons, the thing being recorded is not “the robot doing the task”. It is the output of a controller - you - that is about to be replaced by a neural network with none of your eyes, none of your priors and none of your ability to notice that the brick moved. Everything the network gets to imitate has to be visible in the frames and the joint angles. Nothing you knew and did not act on survives the recording.
The follower is always a little behind
Wider than the screen; scroll it sideways.
Move the leader quickly and watch the follower carefully. It trails. Stop, and it arrives a moment later. The faster you move, the further behind it sits.
This is the steady-state error from the PID lesson in Module 1, in the room with you rather than on a plot. The follower’s servos are running a fixed position loop against a joint that has gravity pulling on it, a gearbox rubbing against itself, and real inertia to accelerate. A moving target it can never quite catch produces a standing gap, and the gap grows with the speed of the target and the load on the joint.
You cannot tune it from your side, and you should not want to. What you can do is stay inside it. Sweep a joint slowly and the follower is where the leader is. Sweep it fast and it is somewhere else, which means the arm in the camera frame is not the arm you think you are driving.
How wide the gap gets on your arm, at your speeds, with your payload, is not a number anyone can hand you. Measure it. It becomes your speed budget for every recording session in this module.
Bringing it up
Two commands, and the second one is the one you will run a hundred times.
lerobot-teleoperate \
--robot.type=so101_follower --robot.port=/dev/ttyACM0 --robot.id=my_follower \
--teleop.type=so101_leader --teleop.port=/dev/ttyACM1 --teleop.id=my_leaderAdd cameras and a live view only once bare teleoperation works:
lerobot-teleoperate \
--robot.type=so101_follower --robot.port=/dev/ttyACM0 --robot.id=my_follower \
--robot.cameras="{ top: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30} }" \
--teleop.type=so101_leader --teleop.port=/dev/ttyACM1 --teleop.id=my_leader \
--display_data=trueThat last flag streams the camera feeds and the joint traces to a viewer, which is the first time you get to see what the robot sees rather than what you see. Cameras and viewpoints is where that view gets designed properly; for now it is a sanity check.
If nothing moves at all, the fault is almost never in the flags. Go and look at the arm: the motor LEDs, the three-pin cables, the board’s jumpers and the power supply. Servo IDs and wiring has the triage sequence, and it is worth running before you read a single line of the stack trace.
One version note, because this surface moves. The commands above are written against the 0.6.x line of LeRobot. Pin the version you install, write it in your notes, and be suspicious of any tutorial older than a few months: several command names and one calibration directory have already been renamed under people’s feet.
Without hardware
Every LeRobot frame stores both halves of the teleoperation loop: action is where the
leader was, observation.state is where the follower actually got to.
- Instead of two arms, download
lerobot/svla_so101_pickplaceand computeaction - observation.stateper joint, row by row, across one episode. - Measure this: the median and 95th percentile of that gap per joint, and where in the episode it peaks. That is the follower falling behind the hand, measured on somebody else’s bench, and it is the quantity this lesson asks you to feel.
- What you lose: the direction of the asymmetry as an experience. Nothing the follower touches travels back to your hand, and reading that in a column of numbers is not the same as pushing a leader arm into a table and feeling nothing.
Check yourself
1. You hold the follower’s gripper closed with your fingers and squeeze the leader’s trigger anyway. Trace what each of the two arms is doing.
The leader is doing nothing at all: it is unpowered as far as motion goes, so your trigger finger meets only its own gearbox friction, exactly as it would in free air. Its servo simply reports a smaller gripper value each tick. The follower receives that value as a goal position, cannot reach it because your fingers are in the way, and does what any position-controlled servo does with a standing error: pushes harder. It will keep pushing until it hits whatever torque or current limit is configured. This is the position-control-on-contact failure from Module 0, and teleoperation walks you into it every single episode.
2. Why is the gripper normalised to 0-100 while the other five joints are in degrees?
Because a gripper angle is not a quantity you ever want to reason about. What matters is how far through its travel the jaws are, which is a fraction, not an angle. Normalising to a percentage also makes the value survive mechanical differences: a jaw with a slightly different range still means “half closed” at 50. The five arm joints are different, because their angles correspond to physical geometry you compute with.
3. Name every feedback loop that is running while you teleoperate, and say which one is watching the task.
Three levels. Inside each servo, a position loop compares the goal against its own encoder and drives current until they agree; six of these on each arm. Around that, the host program reads six numbers and writes six numbers, which is not a feedback loop at all - it never observes the outcome of anything it wrote. And around everything, you: eyes on the object, hand on the leader, correcting. Only the outermost loop is watching the task. That is the loop a policy has to replace.
4. Your follower visibly trails the leader during fast sweeps. Is this a bug, and what should you change?
Not a bug. It is standing error in a position loop chasing a moving target under gravity and gearbox friction, and it grows with speed and load. Nothing on your side of the wire tunes it away. Change your own speed instead: find the fastest sweep where you cannot see a gap, and treat that as the ceiling for recording. The reason this matters is that the camera records the follower, so a lagging follower puts lag into your dataset.
5. You start lerobot-teleoperate and it asks you to move the arm to the middle of its range. You calibrated this arm yesterday. What happened, and what is the wrong thing to do next?
The identifier you passed did not match the identifier you calibrated with, so no calibration file was found, and a missing calibration looks exactly like a new arm. The wrong thing to do is comply. Sweeping the joints now writes a fresh calibration under the new name, and you will end up with two calibrations for one arm and no idea which one your data was recorded against. Quit, fix the string, run it again.
6. Teleoperation feels like the robot is doing the task. In what sense is that false, and why does the distinction matter for the next lesson?
Nothing on the desk is deciding anything. The arm is executing a stream of joint angles that your hand produced, and no software in the loop has any representation of the brick, the bowl or success. It matters because a recording of teleoperation is a recording of your control policy, and the network that later replaces you sees only the camera frames and the joint angles. Anything you knew but did not express as motion is simply absent from the data.
Do this
About forty minutes, with your own hardware. Nothing here has been tested against a physical arm by the author, so treat every step as something you verify rather than something you trust.
1. Store your identifiers once. From module-04-hardware/code/:
python teleop_preflight.py init
python teleop_preflight.py commandsThe first writes so101_setup.json with your ports, your identifiers and your camera indices. The second prints your calibrate and teleoperate commands with those values substituted. Paste from that output for the rest of the module and the strings cannot drift.
2. Run the physical checklist before you open a port.
python teleop_preflight.py checkSix questions, in order, stopping at the first “no” with the remedy. If you find yourself annoyed at answering them, that feeling is the reason the script exists.
3. Teleoperate with no cameras. Fewer moving parts, and if this does not work nothing downstream will. Move every joint through a slow, deliberate sweep and confirm the follower goes where you send it. Then quit and add the cameras and --display_data=true.
4. Measure your speed budget. This is the number you are actually here for.
Pick one joint, say the shoulder. Sweep it through a wide arc as slowly as you comfortably can, and watch the follower. Speed up a little. Repeat until you can clearly see the follower sitting behind the leader mid-sweep. Now come back down until the gap disappears again.
Write down what that speed feels like in seconds: “a full shoulder sweep in about three seconds.” Then do it again with something heavy in the gripper, because payload widens the gap, and write that down too. Those two numbers are your recording speed limit, and the next lesson spends them.
5. Find your own version of the asymmetry. Push the follower’s forearm off course while holding the leader. Notice, deliberately, that your hand learns nothing. Then put the arm somewhere it can safely stall for a second against a soft object and watch how completely unremarkable that feels from the leader’s side. That absence is what you are compensating for with your eyes for the rest of this module.
What you can now do
You can bring up a leader-follower pair from a stored configuration, explain exactly which six numbers cross the gap between the arms and in which direction, and say why contact force is not among them. You can name the three nested loops running during teleoperation and identify the only one that is watching the task. And you have measured, on your own hardware, the speed above which your follower stops keeping up - which is the constraint every demonstration you record from here has to live inside.