Ngseo/basevla-libero-noaug-nostate

BaseVLA — LIBERO 4-suite (Qwen2.5-VL-3B), vision-only, no augmentation

Plain behaviour cloning: no V-JEPA, no latent alignment, no decorrelation — the objective is L1(action) alone. Trained without proprioception and without augmentation, starting from a ManiSkill-pretrained VLA rather than from Qwen directly.

Initialised from Ngseo/basevla-maniskill-aug (step 40,000), then fine-tuned for 100,000 steps on LIBERO.

Data

The four standard LIBERO suites, equal sampling weight per suite (weight / n_frames, so suite size does not bias the mix):

suite	episodes	frames
libero-spatial	432	52,970
libero-object	454	66,984
libero-goal	428	52,042
libero-10	379	101,469
total	1,693	273,465

Front camera (observation.images.image) only; wrist camera dropped. 10 fps.

Architecture

Qwen2.5-VL-3B + LoRA ─ forward_full_hidden ─┬─ Head A → z_a ─┐
                                            └─ Head B → z_b ─┴─ concat → ResNetActionHead → 30×7

L = L1(action)

LoRA	r=32, LLM q_proj/v_proj only — the vision tower is frozen
Heads	dual AttentiveLatentHead, proj 4096, 8 queries, depth 2
Inputs	1 RGB frame @224 + task string — no proprioceptive state
Output	30-step action chunk
Optimiser	AdamW, lr 5e-5, batch 32, bf16
Augmentation	none

Two choices worth flagging:

• chunk_size 30, not the LIBERO-standard 10. The ManiSkill checkpoint was trained at 30, and the action head's output layer is chunk × action_dim, so 30 is required for its weights to load. At LIBERO's 10 fps that is a 3-second horizon rather than 1 second — evaluation must use the same chunk length.
• --no_state, so the state token collapses to a constant learned bias (state_proj(0)) and no proprioception reaches the model.

Training curve

step	loss
10,000	0.0781
20,000	0.0636
30,000	0.0523
50,000	0.0407
70,000	0.0354
100,000	0.0340

Related

repo	data	state	aug	init
basevla-libero-aug	LIBERO	yes	yes	scratch, chunk 10
this	LIBERO	no	no	ManiSkill 40k, chunk 30
basevla-maniskill-aug	ManiSkill	no	yes	scratch

Note that this model and basevla-libero-aug differ in more than augmentation (state input, chunk length and initialisation all differ), so they are not a controlled augmentation ablation.

Not evaluated in simulation or on a robot. The losses above are training-set L1 on normalised actions; no LIBERO rollout success rate is reported here.

Contents

import torch
ck = torch.load("checkpoint.pt", map_location="cpu", weights_only=False)
ck["step"]        # 100000
ck["args"]        # full training config
ck["agg_stats"]   # action normalisation used at train time

checkpoint.pt holds policy (full VLM incl. LoRA), latent_head, free_latent_head, action_head, optimizer, agg_stats and args.