smolvla-so101-pickplace-v2 — 90% language-conditioned pick-and-place

SmolVLA (450M) fine-tuned on so101-sim-pickplace-v2 (160 SO-ARM100 episodes incl. 12.5% verified-retry recovery demonstrations):

"Pick up the red block and place it in the blue tray."

Results (20 eval episodes, fixed seed disjoint from training)

Policy	Success
SmolVLA base, zero-shot	0%
SmolVLA on 100 nominal eps	55%	all failures = right-side non-engagements (coverage)
SmolVLA on the 160-ep -v2 set (this model)	90%	coverage fixed; one eval episode shows a live miss→retry→success (165 steps vs the stereotyped ~105)
ACT (~52M specialist BC) on the same -v2 data	50%	hurt by the same data — see below

The cross-architecture finding

The identical -v2 dataset moved two architectures in opposite directions:

Trained on -v2 (160 eps, incl. recovery branches)	vs. its nominal-data baseline
ACT (chunked L1 regression)	65% → 50% (and temporal ensembling flips from +10 to −15)
SmolVLA (flow-matching action expert)	55% → 90%

Recovery demonstrations make the data behaviorally multimodal: from near-identical states, some demonstrations proceed to lift while others re-approach. Chunk regression averages incompatible continuations into compromise actions; a flow-matching generative head simply learns both modes. Full analysis: the ACT side is documented as a negative-result model card.

Fine-tuning recipe (single Kaggle T4, free tier)

LeRobot 0.6.0 lerobot-train: batch 8, 12k steps (~8 h), AMP, frozen VLM / action-expert-only (defaults), cameras renamed to the base's naming (front→camera1, wrist→camera2). Same recipe as the 55% v1 model — only the data changed.

Provenance

Base lerobot/smolvla_base (Apache-2.0); author-recorded scripted-expert demonstrations in MuJoCo; robot model from MuJoCo Menagerie (trs_so_arm100).