35 min

Reading somebody else's robot

The SO-101 model file is the complete specification of the arm, and four minutes of reading gets you its degrees of freedom, its limits, its servo gains and one thing it can never do.

Where you are. You can read MJCF: bodies nest, joints connect, geoms have shape, actuators drive. This lesson points that skill at a real robot and gets the whole specification out of it.

The folder on your desk

Someone drops a folder in front of you. Inside: one text file about two hundred and fifty lines long, and nineteen binary blobs of geometry. She says, “that’s the arm we’re using.” You have never seen the arm. It lives in a building you have never visited.

Before you write a single line of code you want three answers. How many motors does it have? How far does each one turn? And can it hold a pen pointing in any direction you like, or are some directions simply not on the menu?

All three answers are in the folder. Getting them out takes about four minutes and no guessing, because for everything downstream of this moment the file is not a description of the robot. It is the robot.

The idea in one paragraph

The SO-101 model from MuJoCo Menagerie is a few hundred lines of MJCF, and reading it hands you the entire vocabulary from the very first lesson of this course, in order: <body> elements are the links, their nesting is the kinematic chain, each <joint> states how one link may move relative to its parent, each <geom> is a piece of shape, and each <actuator> is the servo driving one joint. What you get out of that read is not trivia. You get the degree-of-freedom count, the reachable angle of every joint, the exact gains the simulated servos will use to fight gravity, and one honest surprise that appears in no README: this arm has five degrees of freedom for placing its hand, not six, so there are poses it can never hold. Every later lesson in this module - control, teleoperation, grasping, recording - is downstream of numbers that live in this one file.

Getting the file

Menagerie is DeepMind’s collection of robot models tuned to work well in MuJoCo, all under permissive licences. Clone it once and you have the arm.

git clone https://github.com/google-deepmind/mujoco_menagerie ~/mujoco_menagerie

There are two SO-ARM models in there, and the difference matters more than it looks.

Nesting is the chain, on a real robot

MJCF anatomy established the rule: a <body> inside another <body> is a link hanging off the one above it. Here is that rule applied to two hundred and fifty lines of somebody else’s XML.

world no joint: the base is bolted down base 147 g shoulder_pan hinge, -110° to 110° shoulder 100 g shoulder_lift hinge, -100° to 100° upper_arm 103 g elbow_flex hinge, -96.8° to 96.8° lower_arm 104 g wrist_flex hinge, -95° to 95° wrist 79 g wrist_roll hinge, -157.2° to 157.2° gripper 87 g, carries the fingertip frame no joint gripper camera_mount 12 g, welded to the gripper moving_jaw_so101_v1 12 g, hinge -10° to 100° indentation in the XML is the chain
The SO-101 body hierarchy: world, base, shoulder, upper arm, lower arm, wrist and gripper in a single chain, each joined by one named hinge, with the camera mount welded on and the moving jaw branching off the gripper

Wider than the screen; scroll it sideways.

Three things in that picture are worth stopping on.

The base is welded to the world. It has no joint, so it cannot move relative to the ground. That single absence is the difference between an arm and a mobile robot, and it is why every position you compute this module is measured from a fixed origin sitting under the base.

A body is not always a link. camera_mount is a body with no joint. It carries 12 grams and a camera and it moves exactly as the gripper moves, forever. MuJoCo still gives it its own body because a body is the unit that owns mass and geometry, not the unit that owns motion. Bodies are the nouns; joints are the verbs.

The chain forks at the hand. gripper has two children. One is the camera mount, welded. The other is moving_jaw_so101_v1, on a hinge. That fork is what makes this a gripper rather than a stick.

Six joints, five of which matter

Every joint on this arm is a hinge. There is not a single sliding joint anywhere, which is typical: rotation is cheap to build and easy to seal.

JointRangeWhat it does
shoulder_pan-110° to 110°swings the whole arm left and right
shoulder_lift-100° to 100°raises and lowers the upper arm
elbow_flex-96.8° to 96.8°folds the forearm
wrist_flex-95° to 95°tilts the hand
wrist_roll-157.2° to 157.2°spins the hand about its own axis
gripper-10° to 100°opens and closes the jaw

Six joints, and Module 0 told you six is the magic number: three numbers of position and three of orientation are what it takes to put an object anywhere in any attitude. So the arm should reach everything. It does not, and the reason is on the last row of the table.

The gripper joint drives the jaw. Nudge it and the fingertip does not move at all; the exercise measures 0.0 mm of travel, because the jaw hangs off the hand rather than carrying it. That leaves five joints to place the hand, against six numbers that a pose demands.

what a pose asks for six independent numbers: three of position, three of orientation 1 2 3 4 5 6 what the arm supplies five joints move the fingertip; the sixth only opens the jaw shoulder_pan shoulder_lift elbow_flex wrist_flex wrist_roll 105 mm 104 mm 88 mm 48 mm 4 mm none nothing supplies a sixth direction which one is missing depends on where the arm is fingertip Jacobian rank = 5, at all 200 random poses tested the numbers are fingertip travel for a 0.3 rad nudge
Six independent numbers define a pose, but the SO-101 supplies only five directions of motion at the fingertip, with the sixth left unfilled

Wider than the screen; scroll it sideways.

You do not have to take that on faith. Build the fingertip’s Jacobian - the matrix from Lesson 1.13 that says how the hand moves per unit of joint speed - stack its position and orientation halves into a six-row matrix, and ask for its rank. Over two hundred random configurations the answer is 55, every time.

rank(J(q))=5<6for every q\mathrm{rank}\left(J(\mathbf{q})\right) = 5 < 6 \qquad \text{for every } \mathbf{q}

In words: at every configuration there is some combination of “move a bit this way and tilt a bit that way” that no joint speeds can produce. Which combination it is changes as the arm moves, which is why you cannot simply declare “it loses yaw” and route around it.

The servos, spelled out

Six <position> actuators, one per joint, each named identically to the joint it drives. Every one of them inherits from a <default class="sts3215"> block, so a single edit changes all six.

ParameterValueMeaning
kp998.22how hard the servo pulls toward the commanded angle
kv2.731how much it resists moving fast, which damps the pull
forcerange±2.94 N·mthe hard ceiling on what any joint can produce

This is the servo from the machine itself, written down. You send an angle; a controller you did not write pushes until the angle is reached; and the only thing standing between “push” and “push forever” is that force ceiling. The whole position-control contract is those three numbers.

The file is honest about its own uncertainty here, in a comment: these gains are not a one-to-one mapping of the gains the real SO-101 servos use. The XML also defines a backlash joint class worth ±0.5 degrees and then, deliberately, applies it to no joint. Both of those are the sim-to-real gap sitting in plain sight with its name on it, and you will come back to them in domain randomisation.

The mismatch that only shows up if you read

Every actuator has a ctrlrange, meaning the span of numbers you are allowed to write as a command. Every joint has a range, meaning the span of angles it can physically occupy. Nothing forces those two to agree, and on this arm they do not.

wrist_roll   ctrl [-157.21, 162.79]   joint [-157.21, 157.21]

The actuator will accept a command 5.58 degrees past where the joint stops. Ask for it and the servo pushes against a hard limit forever, drawing force it can never convert into motion; the same unbounded retry loop from Lesson 0.3, now in simulation where it costs you nothing but a confusing log.

Count the geoms and the number looks absurd: forty-eight shapes on the arm, plus the floor, for a machine with six moving parts. They fall into two piles that never mix.

what you look at class="visual", group 2, contype=0 conaffinity=0 STL mesh hundreds of triangles 18 geoms in this model, 0 of them collide what the solver touches class="collision_gripper", groups 3 and 4, condim=6 boxes, not meshes three spheres, radius 0.75 mm 30 geoms in this model, all of them collide
The same gripper finger drawn twice: the smooth visual mesh that is rendered but never collides, and the boxes and tiny spheres that do the colliding

Wider than the screen; scroll it sideways.

Eighteen geoms are meshes with contype="0" conaffinity="0", which is MJCF for “draw me, but I am not here.” They are what you see, and they take no part in physics at all. The other thirty sit in the collision classes and are all the solver ever touches: twenty boxes, six spheres, one capsule, and three convex hulls where a primitive would not do.

The temptation is to ask why the collision shapes are not just the meshes. The answer is the whole of contact and grasping in one sentence: contact between two curved meshes produces few and badly conditioned contact points, while contact between primitives produces stable, well-separated ones. Look at the fingertips in that figure. Three spheres of radius 0.75 mm, hand-placed, on each jaw. Somebody sat down and decided that a grasp needs three clean points of contact per finger, and put them there by hand. That is not an optimisation. It is the difference between a cube that stays in the gripper and a cube that squirts out of it.

Review

The model file is the specification

Treat a robot model file the way you would treat an interface definition rather than documentation. Documentation describes intent and drifts; an interface definition is what the machinery actually executes, so it cannot drift from behaviour without breaking. If the file says a joint stops at ninety-six point eight degrees, the joint stops there, and no comment anywhere can overrule it. Reading a few hundred lines hands you the whole vocabulary in order: body elements are the links, their nesting is the kinematic chain, each joint states how one link may move relative to its parent, each geom is a piece of shape, and each actuator is the servo driving one joint. What you get out is the degree-of-freedom count, the reachable angle of every joint, and the exact gains the simulated servos will use to fight gravity.

Six joints, five degrees of freedom

Six joints, and six is the magic number: three numbers of position and three of orientation are what it takes to put an object anywhere in any attitude. So the arm should reach everything. It does not. The gripper joint drives the jaw, and nudging it moves the fingertip zero millimetres, because the jaw hangs off the hand rather than carrying it. That leaves five joints to place the hand against six numbers a pose demands. You do not have to take it on faith: stack the fingertip Jacobian’s position and orientation halves into a six-row matrix and ask for its rank, and over two hundred random configurations the answer is five, every time. At every configuration there is some combination of move a bit this way and tilt a bit that way that no joint speeds can produce, and which combination changes as the arm moves.

Control range is not joint range

Every actuator has a control range, the span of numbers you are allowed to command. Every joint has a range, the span of angles it can physically occupy. Nothing forces those two to agree, and on this arm they do not: the actuator will accept a command five and a half degrees past where the joint stops. Ask for it and the servo pushes against a hard limit forever, drawing force it can never convert into motion. Never assume the two spans are the same. Clip your commands to the joint range rather than the control range, and clip them yourself rather than trusting the model. The audit is three lines of code, and this is exactly the class of bug that survives a code review and dies in a demo.

Check yourself

1. camera_mount is a <body> but is not a link. What makes the difference, and how do you tell from the file?

A body owns mass, inertia and geometry. A link is a body that can move relative to its parent, which requires a joint. camera_mount has no <joint> child, so it is welded: it moves exactly as the gripper body moves and adds no degree of freedom. In the compiled model, model.body_jntnum for that body is zero. Bodies are the nouns, joints are the verbs, and a body with no verb never moves on its own.

2. The arm has six joints. Why does it still fail to reach some pose-and-orientation combinations?

One of the six drives the jaw, not the hand. Nudge gripper and the fingertip travels zero millimetres, so only five joints position and orient the hand. A full pose needs six independent numbers, three of position and three of orientation, so at every configuration there is a direction in that six-dimensional space with no joint motion behind it. The mechanical check is the rank of the six-row fingertip Jacobian: it is 55 at every configuration tested, never 66.

3. You ask the model for wrist_roll = 160° and the joint settles at 157.2°. Is this a bug in MuJoCo?

No. The actuator’s ctrlrange runs to 162.79 degrees while the joint’s own range stops at 157.21, so the command is legal and the angle is not. The servo pushes against the joint limit indefinitely and the limit wins. The fix is on your side: clip commands to model.jnt_range rather than to model.actuator_ctrlrange, and audit the two against each other whenever you pick up an unfamiliar model.

4. Eighteen geoms in this model never collide with anything. Why keep them?

They are the visual meshes, carrying contype="0" conaffinity="0". They exist so the robot looks like the robot in a viewer and in rendered camera images, which matters a great deal once you start training policies on pixels. They are excluded from contact because mesh-on-mesh collision is slow and produces poorly conditioned contact points; the thirty primitive shapes underneath do the physical work. One skin to be seen, one skin to be touched.

5. The XML comments say the sts3215 gains are not a one-to-one match for the real servo’s gains, and defines a backlash class it never applies. What are those two facts telling you?

That the model’s author knew exactly where the model stops matching reality and wrote it down. The simulated servo will respond a bit differently from the physical one, and the real joints have roughly half a degree of free play that the model does not reproduce. Both are named handles for later: they are among the first things worth randomising in Lesson 2.16, precisely because you already know they are wrong.

6. Why does the fingertip move only about 4 mm when you nudge wrist_roll, while shoulder_pan moves it over 100 mm?

Because wrist_roll spins the hand about an axis that passes very close to the fingertip. Distance travelled is roughly the rotation angle times the distance from the axis, so a joint near the tool moves it barely at all while a joint at the shoulder swings it through a wide arc. That does not make wrist_roll useless; it contributes orientation rather than position, which is exactly what the last joints of an arm are for. It is also the reason a shoulder-angle error hurts far more than a wrist-angle error of the same size.

Do this

Open module-02-simulation/code/inspect_arm.py. It has three functions marked # TODO(you); the rest is written for you as worked examples of the model-reading API.

cd module-02-simulation/code
python inspect_arm.py            # the whole report
python inspect_arm.py --limits   # just the range audit
python inspect_arm.py --dof      # just the degree-of-freedom probe
  1. body_tree - walk model.body_parentid down from the world body and print the hierarchy with each body’s mass and the joint that attaches it. Your output should reproduce the figure above, including camera_mount showing as welded.

  2. limit_audit - compare every actuator’s ctrlrange against the range of the joint it drives, and flag any that reach past it. Exactly one actuator on this arm fails. Find it without scrolling back up.

  3. dof_report - for each joint, zero qpos, set that one joint to 0.30 rad, call mujoco.mj_forward, and measure how far the site named gripperframe travelled. Then build the six-row Jacobian with mujoco.mj_jacSite at a few hundred random configurations and print its rank.

Then one thing the script does not do. Open so101.xml in an editor, find the <default class="sts3215"> block, change kp from 998.22 to 50, and re-run --dof. Nothing changes, and the reason is worth saying out loud: mj_forward places bodies from joint angles, and gains have no say in geometry. Gains decide how the arm gets to an angle, never where that angle puts the hand. Put the number back.

What you can now do

You can open an unfamiliar robot model and read out its skeleton, its joint limits, its actuator gains and its force ceiling, without trusting a README. You can tell a link from a body, spot a welded body, and follow a chain that forks. You can count the degrees of freedom that actually place the hand and prove the count with a rank test rather than by counting motors, and you know that this particular arm is one short of universal. And you know that the pretty meshes and the shapes that collide are two different sets of objects, which is the first fact you will need when the grasping stops working.

What you can now do

You can open an unfamiliar robot model, map every element onto links, joints, actuators and degrees of freedom, and extract facts about the machine that no README states.