DexMachina
MandiZhao
Physics-based retargeting of human demonstrations into coordinated robot-hand manipulation of articulated objects.
What you can explore
Explore the functional-retargeting pipeline, Genesis simulation setup and released training examples.
Before you use it
The evaluation script is present at the reviewed revision, although the README still lists it as a TODO. Use a matching checkpoint and saved environment; customized Genesis and rl-games forks are required. Simulation is separate from hardware validation.
Original contributors
Mandi Zhao, Yifan Hou, Dieter Fox, Yashraj Narang, Shuran Song and Ajay Mandlekar
License / access: Code: MIT · data, models and dependencies may have separate terms
DexMachina: inspect the simulation before training a policy
- Environment
- Custom Genesis + rl-games forks
- Embodiment
- Released bimanual hand models; verify the chosen asset
What you need
The repository recommends Python 3.10 and provides its own Genesis and rl-games forks plus an environment file. Match those instructions instead of assuming a stock Genesis install is equivalent. Data and model assets have separate access terms.
First useful check
Load one hand in simulation before downloading every demonstration or starting a long training run.
A small first experiment
Pin the environment and assets
Record commits for DexMachina and both custom forks. Follow the linked installation instructions and check which hand assets are included. Keep asset permissions with the experiment log.
Author documentation →Inspect a single simulated hand
After the upstream installation, run the documented Allegro inspection example. The example defaults to the left hand. Keep robot hardware disconnected; this opens a simulated scene.
Hand and object examples →python examples/inspect_hand.py --hand allegro_hand --visSeparate setup from policy evaluation
Inspect the object and demonstration before training. The source includes evaluation, but its default checkpoint value is not a usable file. Supply your own matching run and inspect --help in the installed environment. A single rollout does not establish a benchmark success rate.
Training and evaluation instructions →Check the checkpoint bundle before evaluation
Save the code below as check_dexmachina.py and run python check_dexmachina.py /path/to/trusted/run/nn/best.pth. The evaluator looks for params/env.pkl two directory levels above the .pth file. This check only examines filenames and existence; it never loads pickle or checkpoint contents. Keep the original run directory together. After a PASS, use the upstream evaluation instructions in the matching environment. Run one evaluation environment first with the documented -B 1 option; retain the full checkpoint path rather than copying only its filename. A failed import should be resolved against the custom rl-games fork before changing task settings.
Reviewed evaluation source →from pathlib import Path import sys if len(sys.argv) != 2: raise SystemExit("Usage: python check_dexmachina.py /path/to/run/nn/best.pth") checkpoint = Path(sys.argv[1]).expanduser().resolve() if checkpoint.suffix != ".pth": raise SystemExit("Expected a .pth checkpoint") config = checkpoint.parent.parent / "params" / "env.pkl" missing = [str(p) for p in (checkpoint, config) if not p.is_file()] if missing: raise SystemExit("Missing: " + ", ".join(missing)) print("PASS: checkpoint and saved environment exist") print("Next: verify their provenance and matching code revision")
What to record
- Use an experiment sheet with columns for task/object, hand and side, checkpoint, code revisions, seed, trial count, success criterion and failures. Define success before watching the rollout; report successful trials over all attempted trials, including resets and failures.
- Record model identity, handedness, missing assets and a scene screenshot.
- For a later policy test, record the checkpoint, task, reset conditions and number of trials separately from this setup check.
- Retain the checkpoint, params/env.pkl, code/fork commits, task and observation/action conventions. Do not execute untrusted serialized files.
Common problems and what to check
- A checkpoint loads but the result differs
- First compare hand side, object/task, saved environment and code revisions. Then record reset conditions and seeds. Do not tune the success definition after seeing results or report one selected video as a success rate.
- Imports fail with a stock simulator
- Compare your simulator fork and dependencies with the project environment.
- A rendered scene looks successful
- Rendering only confirms a scene loads. It does not establish task success, accurate contact physics or hardware transfer.
- Evaluation reports a missing env.pkl
- A .pth file alone is not the run bundle. Preserve its parent run directory and params/env.pkl; do not substitute a configuration from a different hand or task.
Where it stops
At the reviewed source revision, the evaluation script is present even though the README still lists it as a TODO. It expects the matching saved environment beside the checkpoint. The workflow below is source-reviewed, not a reproduced benchmark or real-hand deployment.
Keep exploring
ManipTrans
Transfers dexterous bimanual human demonstrations to robots with residual learning.
ARCTIC
Two human hands interacting with articulated objects, with motion and contact information for reconstruction research.
DexMimicGen
Generated demonstrations and simulation environments for learning coordinated dexterous manipulation with two arms.
Editorial reference · checked 2026-10-24