DEXTEROUS MANIPULATION

The world of dexterous hands.

Compare hardware. Discover applications. Share what works. An open reference and community for dexterous manipulation.

DexMachina

MandiZhao

Physics-based retargeting of human demonstrations into coordinated robot-hand manipulation of articulated objects.

What you can explore

Explore the functional-retargeting pipeline, Genesis simulation setup and released training examples.

Before you use it

The evaluation script is present at the reviewed revision, although the README still lists it as a TODO. Use a matching checkpoint and saved environment; customized Genesis and rl-games forks are required. Simulation is separate from hardware validation.

Original contributors

Mandi Zhao, Yifan Hou, Dieter Fox, Yashraj Narang, Shuran Song and Ajay Mandlekar

License / access: Code: MIT · data, models and dependencies may have separate terms

Original code & documentation

Hand and object examples

Training and evaluation

FROM REFERENCE TO FIRST EXPERIMENT

DexMachina: inspect the simulation before training a policy

INPUTHand/object assets + processed human demonstrations
OUTPUTSimulation rollouts and trained policies
Environment
Custom Genesis + rl-games forks
Embodiment
Released bimanual hand models; verify the chosen asset

What you need

The repository recommends Python 3.10 and provides its own Genesis and rl-games forks plus an environment file. Match those instructions instead of assuming a stock Genesis install is equivalent. Data and model assets have separate access terms.

First useful check

Load one hand in simulation before downloading every demonstration or starting a long training run.

A small first experiment

  1. Pin the environment and assets

    Record commits for DexMachina and both custom forks. Follow the linked installation instructions and check which hand assets are included. Keep asset permissions with the experiment log.

    Author documentation →
  2. Inspect a single simulated hand

    After the upstream installation, run the documented Allegro inspection example. The example defaults to the left hand. Keep robot hardware disconnected; this opens a simulated scene.

    python examples/inspect_hand.py --hand allegro_hand --vis
    Hand and object examples →
  3. Separate setup from policy evaluation

    Inspect the object and demonstration before training. The source includes evaluation, but its default checkpoint value is not a usable file. Supply your own matching run and inspect --help in the installed environment. A single rollout does not establish a benchmark success rate.

    Training and evaluation instructions →
  4. Check the checkpoint bundle before evaluation

    Save the code below as check_dexmachina.py and run python check_dexmachina.py /path/to/trusted/run/nn/best.pth. The evaluator looks for params/env.pkl two directory levels above the .pth file. This check only examines filenames and existence; it never loads pickle or checkpoint contents. Keep the original run directory together. After a PASS, use the upstream evaluation instructions in the matching environment. Run one evaluation environment first with the documented -B 1 option; retain the full checkpoint path rather than copying only its filename. A failed import should be resolved against the custom rl-games fork before changing task settings.

    from pathlib import Path
    import sys
    
    if len(sys.argv) != 2:
        raise SystemExit("Usage: python check_dexmachina.py /path/to/run/nn/best.pth")
    checkpoint = Path(sys.argv[1]).expanduser().resolve()
    if checkpoint.suffix != ".pth":
        raise SystemExit("Expected a .pth checkpoint")
    config = checkpoint.parent.parent / "params" / "env.pkl"
    missing = [str(p) for p in (checkpoint, config) if not p.is_file()]
    if missing:
        raise SystemExit("Missing: " + ", ".join(missing))
    print("PASS: checkpoint and saved environment exist")
    print("Next: verify their provenance and matching code revision")
    Reviewed evaluation source →

What to record

  • Use an experiment sheet with columns for task/object, hand and side, checkpoint, code revisions, seed, trial count, success criterion and failures. Define success before watching the rollout; report successful trials over all attempted trials, including resets and failures.
  • Record model identity, handedness, missing assets and a scene screenshot.
  • For a later policy test, record the checkpoint, task, reset conditions and number of trials separately from this setup check.
  • Retain the checkpoint, params/env.pkl, code/fork commits, task and observation/action conventions. Do not execute untrusted serialized files.
Common problems and what to check
A checkpoint loads but the result differs
First compare hand side, object/task, saved environment and code revisions. Then record reset conditions and seeds. Do not tune the success definition after seeing results or report one selected video as a success rate.
Imports fail with a stock simulator
Compare your simulator fork and dependencies with the project environment.
A rendered scene looks successful
Rendering only confirms a scene loads. It does not establish task success, accurate contact physics or hardware transfer.
Evaluation reports a missing env.pkl
A .pth file alone is not the run bundle. Preserve its parent run directory and params/env.pkl; do not substitute a configuration from a different hand or task.

Where it stops

At the reviewed source revision, the evaluation script is present even though the README still lists it as a TODO. It expects the matching saved environment beside the checkpoint. The workflow below is source-reviewed, not a reproduced benchmark or real-hand deployment.

Public documentation reviewed; not independently reproduced · reviewed 2026-10-30. Record the exact source revision when reproducing it.

Keep exploring

ManipTrans

Transfers dexterous bimanual human demonstrations to robots with residual learning.

ARCTIC

Two human hands interacting with articulated objects, with motion and contact information for reconstruction research.

DexMimicGen

Generated demonstrations and simulation environments for learning coordinated dexterous manipulation with two arms.

Editorial reference · checked 2026-10-24

Start a discussion

Published posts are public. First contributions are reviewed. Only share material you have permission to disclose.

Report a post