Image-to-Video
Diffusers
Safetensors
nielsr HF Staff commited on
Commit
59eef53
Β·
verified Β·
1 Parent(s): 75b4cc9

Improve model card with metadata, links, and usage example

Browse files

This PR significantly improves the model card for "One-to-All Animation: Alignment-Free Character Animation and Image Pose Transfer" by:

- Integrating the `pipeline_tag: image-to-video` to correctly categorize the model for better discoverability on the Hugging Face Hub.
- Adding `library_name: diffusers`, which enables the automatic "how to use" widget on the model page due to its compatibility with the Diffusers library.
- Including direct links to the paper ([https://huggingface.co/papers/2511.22940](https://huggingface.co/papers/2511.22940)), project page ([https://ssj9596.github.io/one-to-all-animation-project/](https://ssj9596.github.io/one-to-all-animation-project/)), and the GitHub repository ([https://github.com/ssj9596/One-to-All-Animation](https://github.com/ssj9596/One-to-All-Animation)).
- Adding a "Quick Inference" section with a code snippet directly sourced from the official GitHub README to provide immediate usability instructions.
- Incorporating "Highlights", "Showcase" with visual examples, and "Dependencies and Installation" sections for comprehensive information.

Please review and merge this PR if everything looks good.

Files changed (1) hide show
  1. README.md +101 -3
README.md CHANGED
@@ -1,3 +1,101 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: image-to-video
4
+ library_name: diffusers
5
+ ---
6
+
7
+ # One-to-All Animation: Alignment-Free Character Animation and Image Pose Transfer
8
+
9
+ [🌐 Project Page](https://ssj9596.github.io/one-to-all-animation-project/) | [πŸ“„ Paper](https://huggingface.co/papers/2511.22940) | [πŸ’» Code](https://github.com/ssj9596/One-to-All-Animation)
10
+
11
+ This repository contains the model and code for "One-to-All Animation: Alignment-Free Character Animation and Image Pose Transfer." We present a unified framework for high-fidelity character animation and image pose transfer for references with arbitrary layouts. This work reformulates training as a self-supervised outpainting task, designs a reference extractor for comprehensive identity feature extraction, integrates hybrid reference fusion attention, and introduces identity-robust pose control and a token replace strategy for coherent long-video generation.
12
+
13
+ ## 🌟 Highlights
14
+
15
+ We provide a **complete and reproducible** training and evaluation pipeline:
16
+
17
+ - βœ… **Full Training Code**: Three-stage progressive training from scratch
18
+ - βœ… **Complete Benchmarks**: Reproduction code and pre-trained checkpoints
19
+ - βœ… **Flexible Training Codebase**: Multi-resolution, multi-aspect-ratio, and multi-frame training codebase
20
+ - βœ… **Datasets**: Pre-processed open-source datasets + self-collected cartoon data
21
+
22
+ ## 🎭 Showcase - 1.3B Model Results
23
+
24
+ <p align="center">
25
+ <img src="https://github.com/ssj9596/One-to-All-Animation/raw/main/assets/combined_video1.gif" height="300"/> &nbsp;&nbsp; <img src="https://github.com/ssj9596/One-to-All-Animation/raw/main/assets/combined_video2.gif" height="300"/>
26
+ </p>
27
+
28
+ ## πŸ”§ Dependencies and Installation
29
+
30
+ 1. Clone Repo
31
+ ```bash
32
+ git clone https://github.com/ssj9596/One-to-All-Animation.git
33
+ cd One-to-All-Animation
34
+ ```
35
+
36
+ 2. Create Conda Environment and Install Dependencies
37
+ ```bash
38
+ # create new conda env
39
+ conda create -n one-to-all python=3.12
40
+ conda activate one-to-all
41
+
42
+ # install pytorch
43
+ pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124
44
+ # or
45
+ pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 -i https://mirrors.aliyun.com/pypi/simple/
46
+
47
+ # install python dependencies
48
+ pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
49
+
50
+
51
+ # (Recommended) install flash attention 3 (or 2) from source:
52
+ # https://github.com/Dao-AILab/flash-attention
53
+ ```
54
+
55
+ 3. Download Models
56
+
57
+ - Download pretrained models
58
+ ```bash
59
+ cd ./pretrained_models
60
+ bash download_pretrained_models.py
61
+ ```
62
+
63
+ - Download checkpoints
64
+ ```bash
65
+ cd ./checkpoints
66
+ bash download_checkpoints.py
67
+ ```
68
+
69
+ > πŸ’‘ **Tip**: Edit the script and uncomment the specific models you want to download.
70
+ > - **1.3B_1**: Best performance on video benchmark among 1.3B models (paper results).
71
+ > - **1.3B_2**: Further trained on v1 with large camera movement data and increased image ratio. Better for dynamic video generation. Best on image benchmark (paper results).
72
+ > - **14B**: Best overall performance among 14B models (paper results).
73
+
74
+ ## β˜•οΈ Quick Inference
75
+
76
+ We provide several examples in the [`examples`](https://github.com/ssj9596/One-to-All-Animation/tree/main/examples) folder.
77
+ Run the following commands to try it out:
78
+
79
+ ```bash
80
+ # Step 1: Prepare model input
81
+ cd video-generation
82
+ python infer_preprocess.py
83
+
84
+ # Step 2: Run inference with your preferred model
85
+ python inference_1.3b.py # For 1.3B model
86
+ # or
87
+ python inference_14b.py # For 14B model
88
+ ```
89
+ You can enter the script to modify the input path.
90
+
91
+ ## πŸ“š Citation
92
+ If you find **One-to-All Animation** useful, consider citing our work:
93
+
94
+ ```bibtex
95
+ @inproceedings{shi2025onetoall,
96
+ title={One-to-All Animation: Alignment-Free Character Animation and Image Pose Transfer},
97
+ author={Shi, Shijun and Xu, Jing and Li, Zhihang and Peng, Chunli and Yang, Xiaoda and Lu, Lijing and Hu, Kai and Zhang, Jiangning},
98
+ booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
99
+ year={2025}
100
+ }
101
+ ```