susanping nielsr HF Staff commited on
Commit
fc04bbd
·
1 Parent(s): bae545e

Add model card for Xiaomi-Robotics-0 (#1)

Browse files

- Add model card for Xiaomi-Robotics-0 (859fe96217edc9c75de160a1a1b11d52c2b94d79)
- Resolve merge conflict with main (122e70b09180fbe9b668e94757f2a1907b82d91d)


Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +92 -1
README.md CHANGED
@@ -1,7 +1,98 @@
1
  ---
2
  license: apache-2.0
 
3
  pipeline_tag: robotics
4
  tags:
5
  - vision-language-action
6
  - vla
7
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: transformers
4
  pipeline_tag: robotics
5
  tags:
6
  - vision-language-action
7
  - vla
8
+ - robotic-manipulation
9
+ ---
10
+
11
+ # Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution
12
+
13
+ Xiaomi-Robotics-0 is an advanced vision-language-action (VLA) model with 4.7B parameters, specifically engineered for high-performance robotic reasoning and seamless real-time execution. It is pre-trained on large-scale cross-embodiment robot trajectories and vision-language data, enabling broad and generalizable action-generation capabilities.
14
+
15
+ - **Paper:** [Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution](https://huggingface.co/papers/2602.12684)
16
+ - **Project Page:** [https://xiaomi-robotics-0.github.io/](https://xiaomi-robotics-0.github.io/)
17
+ - **Repository:** [https://github.com/XiaomiRobotics/Xiaomi-Robotics-0](https://github.com/XiaomiRobotics/Xiaomi-Robotics-0)
18
+
19
+ ## Key Features
20
+
21
+ * **🧠 Strong Generalization**: Pre-trained on diverse cross-embodiment trajectories and VL data to handle complex, unseen tasks.
22
+ * **🚀 Real-Time Ready**: Optimized with asynchronous execution to minimize inference latency.
23
+ * **🛠️ Flexible Deployment**: Fully compatible with the Hugging Face `transformers` ecosystem and optimized for consumer GPUs.
24
+
25
+ ## Sample Usage
26
+
27
+ Xiaomi-Robotics-0 is integrated into the Hugging Face `transformers` ecosystem. Below is an example of how to load the model and generate actions for a robotic task.
28
+
29
+ ```python
30
+ import torch
31
+ from transformers import AutoModel, AutoProcessor
32
+
33
+ # 1. Load model and processor
34
+ model_path = "XiaomiRobotics/Xiaomi-Robotics-0-LIBERO"
35
+ model = AutoModel.from_pretrained(
36
+ model_path,
37
+ trust_remote_code=True,
38
+ attn_implementation="flash_attention_2",
39
+ dtype=torch.bfloat16
40
+ ).cuda().eval()
41
+ processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, use_fast=False)
42
+
43
+
44
+ # 2. Construct the prompt with multi-view inputs
45
+ language_instruction = "Pick up the red block."
46
+ instruction = (
47
+ f"<|im_start|>user
48
+ The following observations are captured from multiple views.
49
+ "
50
+ f"# Base View
51
+ <|vision_start|><|image_pad|><|vision_end|>
52
+ "
53
+ f"# Left-Wrist View
54
+ <|vision_start|><|image_pad|><|vision_end|>
55
+ "
56
+ f"Generate robot actions for the task:
57
+ {language_instruction} /no_cot<|im_end|>
58
+ "
59
+ f"<|im_start|>assistant
60
+ <cot></cot><|im_end|>
61
+ "
62
+ )
63
+
64
+ # 3. Prepare inputs
65
+ # Assuming `image_base`, `image_wrist`, and `proprio_state` are already loaded
66
+ inputs = processor(
67
+ text=[instruction],
68
+ images=[image_base, image_wrist], # [PIL.Image, PIL.Image]
69
+ videos=None,
70
+ padding=True,
71
+ return_tensors="pt",
72
+ ).to(model.device)
73
+
74
+ # Add proprioceptive state and action mask
75
+ robot_type = "libero"
76
+ inputs["state"] = torch.from_numpy(proprio_state).to(model.device, model.dtype).view(1, 1, -1)
77
+ inputs["action_mask"] = processor.get_action_mask(robot_type).to(model.device, model.dtype)
78
+
79
+ # 4. Generate action
80
+ with torch.no_grad():
81
+ outputs = model(**inputs)
82
+
83
+ # Decode raw outputs into actionable control commands
84
+ action_chunk = processor.decode_action(outputs.actions, robot_type=robot_type)
85
+ print(f"Generated Action Chunk Shape: {action_chunk.shape}")
86
+ ```
87
+
88
+ ## Citation
89
+
90
+ ```bibtex
91
+ @misc{robotics2026xiaomi,
92
+ title = {Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution},
93
+ author = {Xiaomi Robotics},
94
+ howpublished={\url{https://xiaomi-robotics-0.github.io}},
95
+ year = {2026},
96
+ note={Project Website}
97
+ }
98
+ ```