nielsr HF Staff commited on
Commit
b17717a
·
verified ·
1 Parent(s): 204898d

Improve model card with metadata, paper links and usage

Browse files

Hi! I'm Niels, part of the community team at Hugging Face. I'm opening this PR to improve the model card for RealWonder. The updates include:
- Adding the `image-to-video` pipeline tag to the YAML metadata.
- Including links to the paper, project page, and GitHub repository.
- Adding a sample usage section for offline inference based on the repository's instructions.
- Adding the BibTeX citation.

Files changed (1) hide show
  1. README.md +50 -0
README.md CHANGED
@@ -1 +1,51 @@
 
 
 
 
1
  # RealWonder: Real-Time Physical Action-Conditioned Video Generation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: image-to-video
3
+ ---
4
+
5
  # RealWonder: Real-Time Physical Action-Conditioned Video Generation
6
+
7
+ RealWonder is the first real-time system for action-conditioned video generation from a single image. It uses physics simulation as an intermediate bridge to translate 3D actions into visual representations (optical flow and RGB) that a distilled video generator can process efficiently.
8
+
9
+ [**Project Page**](https://liuwei283.github.io/RealWonder/) | [**Paper (arXiv)**](https://arxiv.org/abs/2603.05449) | [**GitHub Repository**](https://github.com/liuwei283/RealWonder)
10
+
11
+ ## About RealWonder
12
+
13
+ Current video generation models often lack a structural understanding of how actions affect 3D scenes. RealWonder addresses this by integrating three components:
14
+ 1. **3D reconstruction** from single images.
15
+ 2. **Physics simulation** for structural grounding.
16
+ 3. **Distilled video generator** requiring only 4 diffusion steps.
17
+
18
+ The system achieves 13.2 FPS at 480x832 resolution, enabling interactive exploration of forces, robot actions, and camera controls on various materials.
19
+
20
+ ## Usage
21
+
22
+ ### Offline Inference
23
+
24
+ To generate a video from a single image and action-conditioned simulation results, follow these steps as described in the official implementation:
25
+
26
+ 1. **Run physics simulation**:
27
+ ```bash
28
+ python case_simulation.py --config_path demo_data/lamp/config.yaml
29
+ ```
30
+
31
+ 2. **Run video generation**:
32
+ ```bash
33
+ python infer_sim.py \
34
+ --checkpoint_path ckpts/Realwonder-Distilled-AR-I2V-Flow/sink_size=1-attn_size=21-frame_per_block=3-denoising_steps=4/step=000800.pt \
35
+ --sim_data_path result/lamp/final_sim \
36
+ --output_path result/lamp/final_sim/final.mp4
37
+ ```
38
+
39
+ ## Citation
40
+
41
+ ```bibtex
42
+ @misc{realwonder2026,
43
+ title={RealWonder: Real-Time Physical Action-Conditioned Video Generation},
44
+ author={Liu, Wei and Chen, Ziyu and Li, Zizhang and Wang, Yue and Yu, Hong-Xing and Wu, Jiajun},
45
+ year={2026},
46
+ eprint={2603.05449},
47
+ archivePrefix={arXiv},
48
+ primaryClass={cs.CV},
49
+ url={https://arxiv.org/abs/2603.05449},
50
+ }
51
+ ```