Team-RAMEN/ramen-g1-ikea-assembly-ramen-ori-all-skills

RAMEN-Ori: all skills

RAMEN-Ori is the imitation-learning policy that Team RAMEN built for the IROS 2026 Humanoid IKEA Assembly Challenge (Unitree G1 assembling a children's table). This repository holds one model trained on all six assembly skills. It was also the starting point of the per-skill models we ran at the competition:

skill	per-skill model	started from
rotate_table_base	ramen-g1-ikea-assembly-ramen-ori-rotate-table-base	this model, 300k
pick_table_leg	ramen-g1-ikea-assembly-ramen-ori-pick-table-leg	this model, 300k
insert_table_leg	ramen-g1-ikea-assembly-ramen-ori-insert-table-leg	this model, 400k
flip_table	ramen-g1-ikea-assembly-ramen-ori-flip-table	this model, 150k

Code: ramen-g1-ikea-assembly (training in model/ramen_ori/, inference in inference/desktop/).

Model

• Robot: Unitree G1 (29 DoF) with Dex1-1 grippers. The policy drives both arms (7 + 7 joints) and both grippers. It does not output the waist or the legs.
• Inputs
  • 3 cameras at 224 x 224: the left eye of the head camera with the YOLO-OBB detections drawn on it ("overlay"), the left wrist camera and the right wrist camera.
  • A 71-D state (below).
  • A skill id (below).
• Output: a chunk of 32 steps of absolute joint targets, 16-D per step (arms 14 + grippers 2).
• Architecture: a frozen LingBot ViT-B/16 image encoder (lingbot-vision), a fusion transformer (6 layers, width 512) over the image, state and skill tokens, and a flow-matching DiT action expert (12 layers, width 768) that generates the chunk in 6 steps. The attention layers use QK-norm. State and action are normalized inside the model.

skill id	skill
0	insert_table_leg
1	flip_table
2	rotate_leg_to_tighten
3	pick_table_leg
4	rotate_table_base
5	move_table_base

State (71-D, definition version 2). "Upper body" is the waist (3), both arms (7 + 7) and both grippers (2).

dims	name	content
0–18	joint	current upper-body joint angles
19–37	tracking_err	previous command − current angles (waist always 0; 0 on the first step)
38–56	velocity	current − previous angles (0 on the first step)
57–58	hand_state	gripper positions
59–70	ee_pose	left and right hand position (xyz) and orientation (euler xyz)

Training

• Data: the six RAMEN-Ori datasets listed in the metadata. Each is one skill cut out of BitRobot/G1_WBT_Dex1_Building-Children-Table, using the segment timestamps of BitRobot/2026-humanoid-ikea-assembly-challenge. The head images were rendered ahead of time with the YOLO-OBB overlay (confidence 0.30, line width 2, JPEG 4:4:4), in 10 variants per frame (the original and 9 augmented).
• Recipe
  • Batch 32; the six skills are sampled with equal probability.
  • AdamW, learning rate 1e-4, weight decay 0.01, gradient clipping at 1.0, bf16 autocast, EMA 0.9999.
  • 2k warmup steps, then cosine decay to 1e-5 over 150k steps. We then continued at a fixed 1e-5 up to 600k steps.
  • Loss: flow-matching behaviour cloning, plus a forward-kinematics loss. The predicted joints are passed through the G1 URDF and compared with the teacher on elbow position, hand position, hand orientation and the distance between the hands.
  • State dropout: during training, each state group is hidden with a fixed probability (joint 0.1, hand 0.1, tracking_err 0.3, velocity 0.3, ee_pose 0.3), so that the model does not simply copy its previous command.
  • The image encoder stays frozen.
• Compute: one NVIDIA H100 SXM5 80GB. The first 150k steps took about 4.9 hours.
• Config: model/ramen_ori/configs/retrain/all6_state_dropout.yaml (it inherits c32_state_dropout.yaml, common.yaml and base.yaml).

Checkpoints

ckpt_step_010000.pt to ckpt_step_600000.pt, every 10k steps (60 files, about 3.5 GB each), at commit 17b5658e33f98e264a6e05481fc9d069e9fde546. Each file holds model_state_dict (the raw weights), ema_state_dict (the EMA weights), cfg (used to rebuild the model) and contract (the input/output layout, checked at start-up). Our inference code currently loads the raw weights: it reads ema_state_dict only in the shadow_params format, which these checkpoints do not use, so it falls back to model_state_dict and prints a warning.

We used three of them: 150k, 300k and 400k (see below).

Evaluation

Offline, on held-out samples (224 per skill), generating the action the same way as inference does (6 flow steps, the chunk averaged with the temporal-ensemble weights), with the EMA weights. Hand position error in mm:

step	insert	flip	rotate_leg	pick	rotate_table	move	mean
50k	15.00	25.80	15.12	19.11	24.26	17.46	19.46
150k	11.37	20.39	11.80	14.36	19.75	14.21	15.32
300k	10.89	20.78	11.61	14.15	19.61	14.79	15.30
400k	10.74	20.91	11.54	14.29	19.91	15.23	15.44
600k	10.74	21.25	11.76	14.47	20.13	15.71	15.68

• 150k is the best for flip_table and move_table_base.
• 300k is within the best range on the six-skill mean, and the best for rotate_table_base.
• 400k is the first step where insert_table_leg reaches its plateau.

Each step was evaluated with a different random choice of the augmented image variant, so small differences between steps include that noise (for insert_table_leg, 400k–600k stay within 0.16 mm).

Usage

The inference code selects a checkpoint by a "slot" name in inference/desktop/lower_policy/configs/policy_config.yaml: <skill>_ramen_ori_all6_{150k,300k,400k}. For example, to run one skill on the robot without moving it (add --actuate to move):

pixi run -e runtime python -m evaluate.model_evaluation.runners.run_skill \
    --interface eth0 --skill insert_table_leg --variant insert_table_leg_ramen_ori_all6_300k

Limitations

• Lab data only. The training data was recorded in one lab setting (black floor mat, white furniture). The competition scene looked different.
• The ee_pose input does not match inference. This model was trained on the dataset's recorded observation.state.ee_state. That value uses a different frame and tool offset from our forward kinematics and is 10–18 cm away from it. Our inference computes ee_pose with forward kinematics. The per-skill models were retrained with forward-kinematics ee_pose to remove this gap.
• The overlay must match training. The head image has to be drawn with the same YOLO-OBB weights (commit 8221d0a), confidence threshold, line width and JPEG chroma subsampling (4:4:4). The checkpoint records the YOLO weights under their former name (Team-RAMEN/IROS2026_RAMEN_Hara_yoloobb_upperpolicy@8221d0a…); the start-up check compares the commit only.

License

This model is released under the Apache License 2.0.

The checkpoints include, unchanged, the frozen LingBot ViT-B/16 image encoder weights from robbyant/lingbot-vision-vit-base (lingbot-vision), which are also licensed under Apache-2.0.

Data attribution

The training data is derived from BitRobot's G1_WBT_Dex1_Building-Children-Table and 2026-humanoid-ikea-assembly-challenge, both licensed under CC BY 4.0. We cut the episodes into skills and rendered the YOLO-OBB overlay onto the head images.