From a model output to a hand command
Trace the boundary that makes or breaks an integration: observation, action representation, controller and feedback.
Write a complete observation/action contract and locate missing finger-control fields.
Write the interface before training
A policy can produce a correctly shaped array that is wrong for a robot. Define an interface contract: camera names and calibration, state fields, joint ordering, units, reference frames, control mode, normalization, timestamps and missing-data behavior. Record the hardware and firmware revision. An array with 16 numbers does not tell you whether those numbers mean absolute positions, position increments, torques or a lower-dimensional grasp representation.
Make observation and action time explicit
The image exposure time, host receive time and command execution time are different events. Store them separately when available and document clock synchronization. Training pairs must represent the intended causal relationship: what the policy could observe before choosing an action. Pairing an image with a future state can leak information; pairing it with a delayed command without accounting for latency can teach an inconsistent response.
Decode actions without changing their meaning
Unnormalize using the statistics associated with the checkpoint and dataset. For a joint-position target, specify radians or degrees and which joints it controls. For a delta target, state whether the increment is relative to the measured position or last target. For a Cartesian target, identify the frame and inverse-kinematics mapping. Do not convert position values into torque commands by simply changing the API call.
A grasp coordinate is not an independent finger
A synergy compresses hand motion into fewer coordinates. That can simplify learning, but limits the configurations directly expressible through that mapping. Retargeting a human hand likewise needs geometry and constraints; copying human joint angles is insufficient. Inspect thumb reach, coupled joints, wrist pose and laterality. A dataset of human poses can help with geometry, but it does not automatically contain robot motor targets or contact forces.
Keep the execution controller separate
The policy proposes a target. A lower-level controller tracks it under the hardware interface and limits. Record controller configuration, command rate, interpolation and what happens when commands stop arriving. Rate limiting or clipping can change the action distribution, so log both the proposed command and the accepted command. A policy evaluation that silently changes these behaviors is a different experiment.
Validate an offline trace first
Select one permitted episode. Decode it without connecting motors, inspect the values and verify the joint map. Plot targets next to measured state with timestamps; check saturation, discontinuities and missing observations. Only use the exact manufacturer-supported setup for subsequent physical testing. This lesson produces an interface audit, not a ready-to-run device adapter.
An illustrative interface contract
| Field | Record explicitly | Failure to catch |
|---|---|---|
| Observation | camera_front at exposure time; q in named joint order | A new camera crop changes the input |
| Target | q_target in radians, absolute position | Degrees interpreted as radians |
| Timing | capture → infer → queue → accept → execute | Inference-only timing hides transport delay |
| Feedback | q_measured, contact signals and timestamps | A commanded grasp is counted as a measured grasp |
Work through the mechanics
Create a versioned data contract and reject a mismatched action before execution.
Prerequisites: Joint angles, radians, arrays and coordinate frames.
The schema is part of the model
Two vectors of the same length can mean different things. One may contain absolute joint angles in radians; another may encode per-step joint increments; a third may encode tendon displacements. The checkpoint, dataset adapter and device interface must agree on the meaning of every coordinate.
Keep an immutable contract beside each recording: hand revision and laterality, joint order, control mode, units, clock basis, sample rate, reference frames, calibration and normalizer version. Store commanded and measured state separately. Missing tactile data must have a validity mask; zeros can mean real zero load and should not silently mean “sensor missing”.
The fragment below records metadata only. It is an original interchange sketch, not RLDS, LeRobot or a vendor SDK schema. Actual samples also need per-stream acquisition timestamps, named arrays and episode boundaries. A receiver must validate the complete agreed contract.
{
"schema_version": "teaching-1",
"hand_revision": "toy-hand-A",
"laterality": "right",
"control_mode": "absolute_joint_position",
"angle_unit": "rad",
"clock": "monotonic_host_A",
"normalizer_id": "train-split-v1",
"joint_names": ["thumb_curl", "index_curl", "middle_curl",
"ring_curl", "little_curl"]
}This five-coordinate teaching hand uses one abstract curl coordinate per finger. It does not claim independent control of all anatomical joints or compatibility with a real product.
Read the primary work: robosuite: controller interfaces and conventions ↗
Decode values before reasoning about motion
For a simple bounded normalization, map a physical coordinate q between lower bound l and upper bound u to x between −1 and 1. In the example l = −0.2 rad, u = 1.4 rad and x = −0.5 decode to q = 0.2 rad. The bounds are invented training scales, not certified mechanical limits.
Do not confuse normalization bounds with safety limits. Some releases use quantiles, standard deviations or masks instead. Use the exact statistics expected by the checkpoint. A changed normalizer can change physical commands without changing the network weights. Handle a constant coordinate separately rather than dividing by zero.
x = 2(q − l)/(u − l) − 1 q = l + (x + 1)(u − l)/2 −0.2 + (−0.5 + 1) × 1.6/2 = 0.2 rad
Decode an action value
Synthetic teaching example · calculations only. No inference, simulation or hardware execution.
Decoded teaching coordinate · -0.2 + (-0.5 + 1) × (1.4 − -0.2) / 2. No hardware command is sent.
Read the primary work: OpenVLA: embodiment-specific unnormalization ↗
Retargeting is a constrained problem
Human fingertips can supply geometric targets, but they do not specify robot joint torques or prove a feasible grasp. A useful retargeting objective balances fingertip error against a posture prior and temporal smoothness, subject to joint and collision constraints. Different hand proportions and missing degrees of freedom can prevent an exact match.
Let fᵢ(q) be forward kinematics for fingertip i and pᵢ its target in the same frame. A position term has squared-length units, while a joint prior has squared-angle units; weighting and scaling must be explicit. Adding a smoothness penalty does not replace velocity-limit checks. Even a collision-free kinematic solution can fail under contact, friction or tendon coupling.
q* = argminq Σᵢ wᵢ‖fᵢ(q) − pᵢ‖²
+ λ‖q − qprior‖² + μ‖q − qprevious‖²
subject to joint limits and collision constraintsRead the primary work: MIT Manipulation: geometric pose and frames ↗ · Dex Retargeting: supported optimization workflows ↗
Find the silent contract failures
A recording is in degrees, the controller expects radians, and two joint names have been swapped. All arrays are finite and the same length. Why does a shape check pass? What tests would you add?
Reveal the worked answer
Shape and finiteness do not encode semantics. Compare the contract’s units, exact joint order, laterality and controller mode; reject mismatches rather than guessing. Run numeric round-trip tests with known values and verify transforms with known poses. Start with recorded data or simulation; this assignment provides no hardware actuation command.
Self-review checklist
- Rejects unit and joint-order mismatches explicitly.
- Distinguishes normalization scales from mechanical limits.
- Tests named coordinates and laterality, not only array length.
- Preserves raw command, decoded target and measured response separately.
Original Dexhands teaching examples. This is a self-study rubric, not automated grading or certification. Research sources reviewed September 25, 2026; examples do not report experiments run by Dexhands.
Try it yourself
Follow one action through the system
Interactive concept exercise · no model inference, hardware commands or measured performance.
Record what the system actually receives
Instruction: “Place the card in the slot.” Images and named joint states carry timestamps. Touch is an input only if the trained interface includes it.
Inspect: Check camera calibration, synchronization and missing fields.
Your practical task
- Trace each stage in the interactive example.
- Write joint names, units, reference frames and normalization for your own setup.
- Identify missing inputs, action coordinates and feedback before using a checkpoint.
What to produce: An interface sheet with exact fields and unresolved mismatches.
Check your understanding
Original sources & next steps
OpenVLA action decoding and normalization ↗Diffusion Policy data and action interfaces ↗Original Dexhands teaching material. Lesson and linked references reviewed 2026-09-25. Research links are not endorsements or evidence of hardware compatibility.