Instructions to use YilaiLiu-HKU/SlotMem with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Wan2.2
How to use YilaiLiu-HKU/SlotMem with Wan2.2:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -44,22 +44,6 @@ Download all checkpoints:
|
|
| 44 |
huggingface-cli download YilaiLiu-HKU/SlotMem --local-dir . --include "ckpt/*"
|
| 45 |
```
|
| 46 |
|
| 47 |
-
## Installation
|
| 48 |
-
|
| 49 |
-
```bash
|
| 50 |
-
git clone https://github.com/YilaiLiu-HKU/SlotMem.git
|
| 51 |
-
cd SlotMem
|
| 52 |
-
|
| 53 |
-
conda create -n slotmem python=3.10 -y
|
| 54 |
-
conda activate slotmem
|
| 55 |
-
|
| 56 |
-
pip install torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/cu128
|
| 57 |
-
pip install -e .
|
| 58 |
-
pip install -r requirements_slotmem.txt
|
| 59 |
-
```
|
| 60 |
-
|
| 61 |
-
Set `CKPT_DIR` to your local Wan2.2 I2V base model directory.
|
| 62 |
-
|
| 63 |
## Usage
|
| 64 |
|
| 65 |
Download the SlotMem checkpoints into the code repository:
|
|
@@ -68,38 +52,8 @@ Download the SlotMem checkpoints into the code repository:
|
|
| 68 |
huggingface-cli download YilaiLiu-HKU/SlotMem --local-dir . --include "ckpt/*"
|
| 69 |
```
|
| 70 |
|
| 71 |
-
Stage-1 inference:
|
| 72 |
-
|
| 73 |
-
```bash
|
| 74 |
-
CUDA_VISIBLE_DEVICES=0 \
|
| 75 |
-
CKPT_DIR=/path/to/Wan2.2-I2V-A14B \
|
| 76 |
-
JSON_PATH=./sample/test/3_271/rewrite_caption.json \
|
| 77 |
-
REF_IMAGE_PATH=./sample/test/3_271/frame.jpg \
|
| 78 |
-
LOW_EXPERT_CKPT_PATH=./ckpt/stage1/stage1_low.pt \
|
| 79 |
-
HIGH_EXPERT_CKPT_PATH=./ckpt/stage1/stage1_high.pt \
|
| 80 |
-
OUTPUT_ROOT=./inference_outputs/slotmem_stage1 \
|
| 81 |
-
bash test_slotmem_stage1.sh
|
| 82 |
-
```
|
| 83 |
-
|
| 84 |
-
Stage-2 inference:
|
| 85 |
-
|
| 86 |
-
```bash
|
| 87 |
-
CUDA_VISIBLE_DEVICES=0 \
|
| 88 |
-
CKPT_DIR=/path/to/Wan2.2-I2V-A14B \
|
| 89 |
-
JSON_PATH=./sample/test/3_271/rewrite_caption.json \
|
| 90 |
-
REF_IMAGE_PATH=./sample/test/3_271/frame.jpg \
|
| 91 |
-
LOW_EXPERT_CKPT_PATH=./ckpt/stage1/stage1_low.pt,./ckpt/stage2/stage2_low.pt \
|
| 92 |
-
HIGH_EXPERT_CKPT_PATH=./ckpt/stage1/stage1_high.pt,./ckpt/stage2/stage2_high.pt \
|
| 93 |
-
OUTPUT_ROOT=./inference_outputs/slotmem_stage2 \
|
| 94 |
-
bash test_slotmem_stage2.sh
|
| 95 |
-
```
|
| 96 |
-
|
| 97 |
For training, data curation, benchmarking, and sample data format, please refer to the [GitHub repository](https://github.com/YilaiLiu-HKU/SlotMem).
|
| 98 |
|
| 99 |
-
## Limitations
|
| 100 |
-
|
| 101 |
-
SlotMem is designed for research on narrative long video generation and character consistency. The released checkpoints are trained with limited data scaling and may not generalize to every visual domain, character style, or deployment setting. Users should validate outputs carefully and disclose synthetic content in real-world use.
|
| 102 |
-
|
| 103 |
## Citation
|
| 104 |
|
| 105 |
The BibTeX entry will be updated when the paper is public.
|
|
|
|
| 44 |
huggingface-cli download YilaiLiu-HKU/SlotMem --local-dir . --include "ckpt/*"
|
| 45 |
```
|
| 46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
## Usage
|
| 48 |
|
| 49 |
Download the SlotMem checkpoints into the code repository:
|
|
|
|
| 52 |
huggingface-cli download YilaiLiu-HKU/SlotMem --local-dir . --include "ckpt/*"
|
| 53 |
```
|
| 54 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
For training, data curation, benchmarking, and sample data format, please refer to the [GitHub repository](https://github.com/YilaiLiu-HKU/SlotMem).
|
| 56 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
## Citation
|
| 58 |
|
| 59 |
The BibTeX entry will be updated when the paper is public.
|