Robot learning · Deep dive
A human shows the robot how; the robot turns those demonstrations into its own skill. This is imitation learning — and it's why Mission 2 works. We'll also see why "just copy the human" quietly falls apart, and the three fixes the 2026 field actually uses.
The simplest way to teach a robot a skill is to record a person doing it, then train the robot to predict "what would the human do here?"
In Ground Control you met the two families: imitation learning (a human shows how) and reinforcement learning (trial and error for a reward). For robot arms we almost always start with imitation — it needs no reward to hand-design, and no million crashes. You just demonstrate.
The most basic form has a blunt name: behavioral cloning (BC). It's ordinary supervised learning. You collect a pile of (observation, action) pairs from a human expert, and you train a network to map one to the other — exactly like image classification maps (picture → label), except here it's (what the robot sees → what to do next).
Key idea
behavioral cloning "BC"
Train a policy — the robot's brain, written πθ(action | observation) — to reproduce the expert's action for each observation it saw. No reward, no exploration. It's "supervised learning on demonstrations."
Remember from Ground Control: policy = the brain, observation = what it sees, action = what it does. BC just fits that brain to a human's recorded behavior.
flowchart LR H["human expert
demonstrates"] --> D["dataset of pairs
(observation, action)"] D --> T["train policy π
(supervised learning)"] T --> P["policy acts
on its own"]
That's the whole idea, and it's genuinely how Mission 2 works: you tele-operate the arm to pick and place a few dozen times, then train. When the demos are good and the situation stays close to what you showed, BC is remarkably strong for how simple it is.
Words you'll keep seeing
expert — the human (or scripted) source of good demonstrations. demonstration / demo — one recorded attempt. rollout — the policy running on its own, start to finish.
Behavioral cloning is, underneath, which kind of machine learning?
Answer: b. BC is plain supervised learning: the "label" for each observation is the action the human took. No reward is involved — that's the RL family.
BC can only copy what you showed. If your demonstrations are sloppy, inconsistent, or all from the exact same starting spot, the policy inherits every flaw.
Fix
✓ Chapter 1 checklist
Before any learning, you need demonstrations in a standard shape. That shape is the LeRobotDataset — the root of the whole ladder.
Where do the (observation, action) pairs come from? From tele-operation. Recall ALOHA's leader–follower trick from Ground Control: you move the small leader arm; the follower copies it and does the real task; and while you puppeteer, the system records everything.
"Everything" is several synchronized streams, sampled ~30–50 times a second:
One recorded attempt is an episode. A few dozen episodes is a dataset.
What is it
LeRobotDataset
Hugging Face's standard format for robot demos — video + state + action + timestamps, packaged so any LeRobot policy can train on it, and shareable on the HF Hub. The docs call it "the fuel of all learning": no dataset, no policy.
# conceptually, what Mission 2 does — record demos into a dataset lerobot-record --robot.type=so101_follower --dataset.repo_id=me/pick_place # then a policy trains on that dataset (next chapters explain HOW)
Two details that matter later. First, the actions are normalized — remember the −1…1 ruler from Ground Control? Datasets store statistics so every joint is on a comparable scale. Second, the observation is multimodal: pixels and numbers together. A good policy has to read both.
Rule of thumb · how many demos?
For a single, well-scoped task (one object, one bin), useful policies often start around ~50 episodes; robustness climbs with more and more varied ones. There's no magic number — it scales with how much the world can vary.
What is stored in a single demonstration episode?
Answer: c. An episode is the whole time-series of what the robot saw and what it did — that's exactly the (observation, action) pairs BC needs, plus the video/state that richer policies use.
If every episode starts with the cube in the same spot, the policy never learns to handle it elsewhere — it memorizes one trajectory.
Fix
✓ Chapter 2 checklist
Naive behavioral cloning has one deep flaw. Understanding it is the key that makes every modern method make sense.
Here's the trap. BC trains only on the states the human visited. But the moment the policy drives on its own, it makes a tiny mistake — turns a joint a hair too far. Now the robot is in a state slightly different from anything in the training data. The policy has never seen this exact situation, so its next guess is a little worse… which lands it somewhere even more unfamiliar… and the error snowballs.
Key idea
compounding error a.k.a. covariate shift / distribution shift
The policy's own small mistakes move it off the distribution of states it was trained on. Off-distribution, it's guessing — so mistakes feed on themselves. This is the classic failure of naive imitation.
flowchart LR S["start"] --> A["expert path
(in the data)"] S --> B["policy path
tiny error → new state"] B --> C["never-seen state
bigger error"] C --> F["failure"]
Why it's worse than it sounds
If the policy makes a mistake with small probability ε at each of T steps, naive BC's total error can grow like ε·T² — quadratically with the length of the task — because each mistake also pushes you into worse states (Ross & Bagnell, 2010). A method that stays on-distribution grows only like ε·T. That gap is the whole game.
The textbook fix is DAgger (Dataset Aggregation): let the policy drive, and wherever it wanders, ask the expert "what should you have done here?" — then add those corrections to the data and retrain. It directly patches the off-distribution holes.
The catch with DAgger
DAgger needs an expert available during training to label the policy's weird states — expensive and awkward for real robots. So in practice the field mostly attacks compounding error a different way: make the policy itself more robust so it slips less and recovers better. That's Chapters 4 and 5.
Why does naive behavioral cloning tend to fail on long tasks?
Answer: a. That's compounding error / covariate shift. Trained only on the expert's states, the policy is lost once its own mistakes take it off that distribution — and each mistake makes the next one likelier.
A policy that starts fine and then diverges partway through is the signature of compounding error — not a bug in your code.
Fix
✓ Chapter 3 checklist
Instead of deciding one tiny step at a time, predict a short sequence of future actions. Fewer decision points means fewer chances to compound.
The first big fix came from the ALOHA team's ACT — Action Chunking with Transformers (Zhao et al., 2023). The idea is simple to state: rather than predicting the single next action, the policy predicts a chunk — the next k actions at once (often k ≈ 50–100, about a second of motion) — then executes them.
Key idea
action chunk
One prediction = a short sequence of future actions, not a single step. If the policy only "decides" every k steps instead of every step, it has roughly k× fewer moments where a small error can throw it off — directly shrinking compounding error.
flowchart TD
subgraph ONE["single-step BC"]
O1["obs"] --> a1["1 action"] --> O2["obs"] --> a2["1 action"] --> O3["obs → …"]
end
subgraph CH["action chunking (ACT)"]
P1["obs"] --> K1["k actions at once"] --> P2["obs (k steps later)"] --> K2["k actions"]
end
There's a bonus. Humans aren't perfectly consistent moment-to-moment (we pause, we jitter). Predicting a whole chunk lets the policy commit to a coherent motion instead of chasing that noise. To keep the seams smooth, ACT uses temporal ensembling: chunks overlap, and overlapping predictions get averaged, so there's no jerk when one plan hands off to the next.
Under the hood (just the shape)
ACT is a Transformer that reads the images + joint state and outputs the next chunk. It's trained as a small CVAE (a variational auto-encoder) so it can absorb the natural variation in human demos. You don't need the math for Mission 2 — just the shape: see → emit a chunk.
Why does predicting an action chunk reduce compounding error?
Answer: b. Deciding every k steps instead of every step cuts the number of error-prone decision moments, and committing to a coherent plan also smooths out human noise. Temporal ensembling keeps the chunk hand-offs seamless.
Too long a chunk and the robot can't react to a change (the cube moved!) until the plan ends. Too short and you're back to twitchy single-step BC.
Fix
✓ Chapter 4 checklist
Often there are many correct actions for the same situation. Averaging them is wrong. Diffusion policies learn the whole set of good options.
Picture the arm facing a cube with an obstacle in front. Going left around it is fine; going right is fine. Both are in your demos. A plain regression policy tries to output one action, so it splits the difference and drives straight into the obstacle — the average of two good answers is a bad answer.
Key idea
multimodality
For one observation there can be several equally-valid actions. A policy that predicts a single number is forced to average them, which can land between the good options. You need a policy that can represent "either A or B," not "the mean of A and B."
Diffusion Policy (Chi et al., 2023) borrows the trick behind AI image generators. Instead of outputting one action, it starts from pure noise and denoises it, step by step, into a clean action sequence — conditioned on what the robot sees. Because denoising can settle into different valid answers, it naturally captures the whole multimodal set, and it trains stably on messy human data.
flowchart LR N["random noise"] --> D1["denoise"] --> D2["denoise"] --> D3["denoise"] --> A["a coherent
action sequence"] O["observation"] -.conditions every step.-> D2
Diffusion Policy and ACT aren't rivals so much as two answers to different parts of the same problem: ACT commits to coherent chunks; diffusion captures multimodal action distributions. Modern systems freely mix the ideas (chunked diffusion heads are common).
Why can't we just train a policy to regress to the single "average" action?
Answer: c. When several actions are valid, their mean may be a bad action. Diffusion policies represent the whole distribution of good options instead of collapsing to one average.
Denoising takes several passes per action, so naive diffusion inference can be too slow for a fast control loop.
Fix
✓ Chapter 5 checklist
Add a language instruction and internet-scale pretraining on top of these policy ideas, and you get the models heading toward "grab that."
Everything so far learns one task from your demos. The frontier adds two things: language ("pick the red cup") and pretraining on huge vision-language data before ever seeing a robot. That's a VLA.
What is it
VLA Vision-Language-Action model
One model that takes camera + a text instruction and outputs actions. It inherits common sense from vision-language pretraining, then is fine-tuned on robot demonstrations — so it can follow instructions and generalize to objects it never saw in your data.
The key realization: VLAs are built from the pieces in this course. Under the hood most of them still chunk actions (Chapter 4) and use a diffusion or flow-matching action head (Chapter 5) — trained on LeRobotDataset-style demos (Chapter 2). You already understand their engine.
flowchart LR IMG["camera"] --> V["VLA
(pretrained vision-language + action head)"] TXT["'pick the red cup'"] --> V V --> ACT["action chunk"]
The 2026 names worth knowing: SmolVLA (450M — small enough to run on a MacBook, the one Physical Spark's ladder ends on), π0 / π0.5 (Physical Intelligence), GR00T (NVIDIA, fine-tunes inside LeRobot), plus OpenVLA and RT-2 as the lineage.
The bridge — what's next in the ladder
You now have the brain. The next missions give it eyes and reach: Mission 4 · Eyes (depth, point clouds, hand-eye calibration) → Mission 5 · Grasp Anything (6-DoF grasping of unseen objects) → Mission 6 · "pick that up" (or "저거 집어" — the mission is language-agnostic) (language → grasp → execute) — a VLA-shaped pipeline end to end.
What does a VLA add over a plain ACT or Diffusion policy?
Answer: b. VLAs still learn from demos (often still chunked + diffusion-headed), but they add language conditioning and internet-scale pretraining — which is what lets them follow "pick the red one" and handle unseen objects.
✓ Chapter 6 checklist
We link and reframe; we never copy. Everything here is explained in our own words — go read the primary work.