DEXTEROUS MANIPULATION

The world of dexterous hands.

Compare hardware. Discover applications. Share what works. An open reference and community for dexterous manipulation.

DexDeform

DexDeform research team

Dexterous deformable-object manipulation with human demonstrations and differentiable physics.

What you can explore

Explore simulated shaping tasks, demonstration loading and trajectory optimization.

Before you use it

The README includes incomplete release tasks. Do not assume every paper component is turnkey or that simulated deformable materials match real ones.

Original contributors

Sizhe Li, Zhiao Huang and collaborators

License / access: MIT · check model, data and dependency terms separately

Original code & documentation

FROM REFERENCE TO FIRST EXPERIMENT

DexDeform: load a soft-object scene, replay a demonstration and inspect the score

INPUTTask configuration, material state and demonstration actions
OUTPUTRendered trajectories and shape-distance measurements
Environment
CUDA simulator + PyTorch / Jupyter
Embodiment
Released Shadow-hand soft-object simulation

What you need

Use the project environment.yml in a separate environment and record the commit, GPU, CUDA and PyTorch versions. The reviewed file includes Linux-specific package builds and CUDA dependencies; it is not a portable Mac installation recipe. Install the editable package and the documented shape-distance dependencies before opening a notebook.

First useful check

Load the folding environment and save one initial render before replaying actions or optimizing a trajectory.

A small first experiment

  1. Open the files that actually exist

    At the reviewed commit, environment loading is tutorials/2_environment_loading.ipynb and trajectory optimization is tutorials/1_trajectory_optimization.ipynb. The README lists the opposite numbering. Start Jupyter from the repository root and use the file tree links below. Before running cells, replace author-specific /home/... paths with your own data and output locations.

    Environment-loading notebook →
  2. Inspect the scene before changing its dynamics

    Create the folding task, render the initial material shape and inspect the hand from multiple views. The notebook obtains the action width from env.simulator.torch_action_scale; use that value instead of assuming it matches the number of visible joints. Record the task configuration, initial state and random seed. Run random actions only in the simulator.

    Folding material and scene configuration →
  3. Replay one trusted demonstration

    Use tutorials/3_demonstration_loading.ipynb. Set demo_dir to your downloaded task folder and choose an existing file explicitly: the example indexes demo_files[5], which fails with fewer than six files. The loader uses serialized demonstration files; only load files from a source you trust. Check states and actions are non-empty, use the demonstration’s initial state, and match env_name to that demonstration before stepping through its actions.

    Demonstration replay notebook →
  4. Separate a shape score from task success

    The scoring notebook selects CUDA device 4; change this before importing the GPU stack if your machine has a different visible device set. Its random target cloud demonstrates an API, not a task goal. Evaluate against the intended target shape in the same coordinates and units. Keep the Sinkhorn settings, particle sampling and task protocol fixed when comparing runs; also inspect whether the object stays grasped and reaches the intended shape.

    Shape-distance example →
  5. Optimize only after replay is understood

    The trajectory notebook uses lift_box and optimizes an action sequence through differentiable physics. Save the starting state, loss definition and before/after trajectory. Replaying a saved demonstration, optimizing one trajectory and training the full learned method are three different results. For a useful report, name which one you completed and retain failed rollouts as well as the best render.

    Trajectory-optimization notebook →
  6. Make the result reproducible

    Attach your task, code revision, environment, input-file provenance, rendered output and exact changes to paths or devices. Keep setup errors separate from manipulation failures. Use the linked checklist to plan a controlled comparison rather than compare videos with different materials or initial states.

    Reproduction checklist →

What to record

  • A scene loads and renders before any optimization is attempted.
  • A replay starts from the demonstration state and uses the matching task and action shape.
  • The target shape, metric settings and failure cases accompany the result.
Common problems and what to check
A notebook link is missing
Use the numbered files at the pinned commit; the README numbering is stale.
No demonstrations or an index error
Confirm the local folder and available filenames before selecting a demonstration; do not assume a sixth file exists.
CUDA device ordinal error
Review the notebook’s device selection and the devices visible to your process before importing CUDA libraries.
A low loss looks like success
Check the real task goal and the rendered trajectory. The scoring tutorial’s random target is illustrative, not a benchmark target.

Where it stops

This is a simulation workflow, not a physical Shadow Hand controller. The release still notes unfinished data-loading cleanup and teleoperation synchronization. A rendered example does not reproduce the full paper or establish material fidelity.

Public source and notebook cells reviewed; GPU simulation and paper results not independently reproduced · reviewed 2026-10-23. Record the exact source revision when reproducing it.

Keep exploring

DexMV

Simulation and retargeting tools for learning dexterous manipulation from human videos.

ManiSkill

SAPIEN-based manipulation simulation with GPU-parallel environments and robot-learning tools.

Editorial reference · checked 2026-09-22

Start a discussion

Published posts are public. First contributions are reviewed. Only share material you have permission to disclose.

Report a post