Choose a method by the experiment you can run
VLA, imitation learning, reinforcement learning and world models overlap. Start with the task, observations and evidence—not a model leaderboard.
Choose a baseline and a falsifiable experiment for the data and environment you have.
Separate the learning signal from the architecture
Imitation learning describes learning from demonstrated behavior. Reinforcement learning describes optimizing a reward through interaction or related experience. VLA describes an interface and model family using vision and language to produce actions; many VLAs learn from demonstrations. A world model predicts transitions and can support learning or planning. These are not mutually exclusive products to rank on one scale.
Choose the smallest informative baseline
For a fixed, narrow task, compare a repeatable scripted or conventional controller with a task-specific imitation policy when appropriate. Keep observations and evaluation conditions comparable. A more general model is useful only if the added capabilities address your task or generalization requirements. Count demonstration collection, reset effort and integration work as part of the experiment, rather than reporting inference time alone.
Use a VLA when language variation matters
Define how instructions change the desired behavior. Holding out paraphrases tests a different claim from holding out objects or tasks. Check whether the language distinguishes the target or merely repeats one constant command. A model that succeeds with the instruction removed may be using the scene rather than instruction understanding. Test this with an appropriate ablation rather than inferring causality from a successful clip.
Use imitation when you can show the behavior
ACT and Diffusion Policy are useful method references for learning action sequences from demonstrations. For your hand, demonstrations must match the deployed observation and action schema. Ask whether the dataset contains recovery from slips and partial grasps, not just clean successes. Record demonstrator interventions and episode boundaries. If distribution shift causes repeated failure, collecting more identical episodes may not solve the missing coverage.
Use reinforcement learning when interaction is meaningful
A simulator can provide many trials, but its reward and contact model define what the agent learns. A policy may exploit a simulator artifact or obtain reward without completing the intended task. Inspect rollouts, randomization ranges and failure cases. Keep simulator scores separate from real-world results. A world model can provide imagined experience, but model errors then become another source of policy error.
Use a world model only with an explicit downstream question
Prediction can be the research objective itself. If the objective is planning, define candidate actions, a cost or reward, uncertainty handling and the point at which a fresh observation corrects the rollout. Compare with the same planner or policy without the learned prediction. A video generator that accepts hand motion does not automatically provide a useful action search space or calibrated contact dynamics.
Different claims require different comparisons
| Claim | Useful comparison | Keep fixed |
|---|---|---|
| Language helps | Correct instruction versus controlled instruction ablation | Scene and available sensors |
| Touch helps | Policy with and without tactile input | Task split and collection procedure |
| Planning helps | Same action budget with and without learned prediction | Observations and success rule |
| Transfer works | Held-out embodiment or environment evaluation | Report adaptations and new training data |
Work through the mechanics
Write a controlled, reproducible experiment rather than a model popularity contest.
Prerequisites: Training/validation/test splits and one precise manipulation task.
Compare the question, not just the model name
Imitation learning describes the learning signal. VLA describes a language-and-vision-conditioned action system. A world model describes a predictive component. They are not mutually exclusive categories. A language-conditioned policy may be trained with imitation; a planner may use both a policy proposal and a world model.
For a fixed card-placement task, first establish a deterministic or imitation baseline that uses the same hand, cameras and reset conditions. Add language only if changing instructions is part of the research claim. Add a predictive model if anticipating outcomes is the proposed mechanism. Every extra component should earn its place through a specific test.
| Comparison | Keep fixed | Question it answers |
|---|---|---|
| Baseline → VLA | Task data, controller, cameras, evaluation starts | Does the chosen pretrained/conditioned system help here? |
| Vision → vision + touch | Other inputs, training protocol, task distribution | Does measured contact information help this task? |
| Policy → policy + prediction | Execution budget and success rule | Does prediction improve downstream decisions? |
| Horizon 1 → horizon 8 | Representation and evaluation conditions | What changes with chunk length and update behavior? |
Read the primary work: ACT: an action-chunk imitation baseline ↗ · Diffusion Policy: configurations and evaluation ↗
Split episodes before making windows
Suppose a recorded episode has 100 timesteps and training uses 8-step action windows at stride 1. It yields 93 windows before boundary padding. Adjacent windows share seven actions. Randomly splitting those windows can put near-identical observations and targets in training and test, producing misleading validation scores.
Assign whole episodes or sessions to splits first. For a claim about new objects, hold out object identities too; for a claim about new operators or sites, group by those units. A session split alone does not establish generalization to new object categories. Declare which variations are in distribution and which are intentionally held out.
Fit preprocessing statistics on training data only. Exclude future frames from observation windows, and keep action targets aligned with the command timestamps used in deployment. Mask padded targets so padding is not learned as genuine behavior.
Number of unpadded windows = T − H + 1 For T = 100 and H = 8: 93 windows These are correlated windows, not 93 independent episodes.
Read the primary work: LeRobot: dataset design and episode structure ↗
Write a small ablation plan before training
Start with a recorded-data audit and a tiny-subset overfit check. Then compare a baseline and one proposed change using fixed splits and evaluation rules. Tune on validation only. Keep the final test set untouched until the comparison is ready; repeated testing followed by tweaks turns it into another validation set.
Record total updates, batch size, data mixture, random seeds, wall-clock time and the selected checkpoint rule. Equal updates do not imply equal compute, and equal compute does not imply equal data exposure. Choose the fairness criterion that answers your question and show the remaining differences.
If you change sensing, architecture and data simultaneously, you can evaluate the combined system, but cannot attribute the improvement to one factor. A clean failure is still informative when it identifies the missing observation, action label or physical capability.
claim: "touch improves held-out card placement"
comparison: [vision_state, vision_state_touch]
split_unit: recording_session
held_out: [object_identity, session]
checkpoint_selection: validation_only
report: [all_trials, success_rule, interventions, latency, failures]
artifacts: [source_commit, config, checkpoint, split_manifest]
# Add actual seeds, budgets and artifact versions before a run.Read the primary work: openpi: record the exact training configuration ↗
Repair a confounded experiment
Model A gets 20 demonstrations and one camera. Model B gets 200 demonstrations, two cameras and touch. B wins. What can you claim, and what experiment is missing?
Reveal the worked answer
You can describe the performance difference between those complete systems under the stated test, not an isolated model-architecture advantage. Match inputs and data for a model comparison, or run a documented ablation that changes one factor at a time. Retain an untouched test set and report the budget differences.
Self-review checklist
- States a system-level claim without attributing causality to architecture.
- Specifies the controlled factor and what stays fixed.
- Groups correlated samples before splitting.
- Records selection rules and compute/data differences.
Original Dexhands teaching examples. This is a self-study rubric, not automated grading or certification. Research sources reviewed September 25, 2026; examples do not report experiments run by Dexhands.
Try it yourself
Compare methods for your starting point
Interactive concept exercise · no model inference, hardware commands or measured performance.
Start with an imitation baseline, then test whether extra conditioning helps.
Imitation / ACT or diffusion
Directly studies observation-to-action learning.
Evidence needed: Need synchronized robot actions, held-out episodes and a matching controller.
VLA
Useful if language or broader transfer is part of the claim.
Evidence needed: Compare to the same-data baseline; instruction following needs its own test.
Reinforcement learning
May refine behavior if a useful interaction environment and reward exist.
Evidence needed: Demonstrations alone do not supply a validated simulator or reward.
World model
Useful if predicting outcomes answers a defined research question.
Evidence needed: Prediction improvement needs a separate downstream control test.
Editorial guidance, not a model ranking. These approaches can be combined.
Your practical task
- Choose a research situation in the comparison tool.
- List the data and interfaces available in your own project.
- Write one baseline and one controlled comparison, including what would count as failure.
What to produce: A method choice justified by evidence and an experiment that could disprove its advantage.
Check your understanding
Original sources & next steps
ACT: action-chunking study ↗DreamerV3: learning through imagined trajectories ↗OpenVLA: comparisons and scope ↗Original Dexhands teaching material. Lesson and linked references reviewed 2026-09-25. Research links are not endorsements or evidence of hardware compatibility.