---
license: other
base_model: alibaba-pai/Wan2.2-Fun-5B-Control
base_model_relation: adapter
library_name: peft
pipeline_tag: video-to-video
model_name: WAN Fun LoRA - 5B Control - Train CFG4 (RGS3) - step1600 - 4-step inference CFG1
language:
  - en
  - zh
datasets:
  - Yanbin99/Depth-Normal-Videos-42K
  - fal/cosmos-openvid-1m
tags:
  - wan2.2
  - wan-fun
  - controlnet
  - depth-control
  - lora
  - dmd
  - non-ar
  - four-step
  - cfg-free
  - cfg1
  - train-cfg4
  - rgs3
  - step1600
  - rank64
---

# WAN Fun LoRA: Train CFG4 (RGS3), step 1600, 4-step inference CFG1

This is the **training-step-1600 generator LoRA** distilled directly on
[alibaba-pai/Wan2.2-Fun-5B-Control](https://huggingface.co/alibaba-pai/Wan2.2-Fun-5B-Control)
using prompt + dynamic Depth video pairs. It is intended for **four denoising
steps with `guidance_scale=1.0`**, which disables classifier-free guidance.
The training teacher uses **CFG=4 (`real_guidance_scale=3`, RGS3)**;
`TrainCFG4` in the repository name records that training setting.

中文：这是专门在 **Wan2.2-Fun-5B-Control** 上训练的 DMD LoRA。发布训练
**1600 step** 检查点；推理 **4 steps、CFG=1（关闭 CFG）**。它就是本次
PAI-Bench-C Depth 评测使用的权重。训练时 generator、teacher、critic 均加载
完整 Fun-Control 模型；发布文件仅包含推理所需的 generator LoRA。

| Setting | Value |
| --- | --- |
| Training checkpoint | **step 1600** |
| Training teacher CFG / RGS | **CFG 4 / RGS 3** |
| Inference denoising steps | **4** |
| Inference CFG | **1.0, conditional-only, no CFG branch** |
| LoRA strength | **1.0** |
| Rank / alpha | 64 / 64 |
| Scheduler | VideoX-Fun `FlowUniPCMultistepScheduler` |
| Scheduler configuration shift / runtime shift | 1.0 / 5.0 |
| Observed four timesteps | `[999, 937, 833, 624]` |
| TeaCache | Disabled for four-step inference |
| Base revision used | `b8bc1a65ab71d054ba4636dc0dac104aa4df2686` |

`step1600` describes the saved training iteration. `4Step` describes inference.
CFG and LoRA strength are separate settings. The LoRA residual is
`strength * (alpha/rank) * (B @ A)`.

During DMD training the teacher used `real_guidance_scale=3`, with the formula
`conditional + 3 * (conditional - unconditional)`. This corresponds to a
conventional teacher CFG scale of **4**, while the student/critic branch and
the released inference recipe use conventional **CFG 1**.

## Training and provenance

The run used two nodes with eight H100 GPUs each, microbatch one per GPU,
gradient accumulation two, and effective global batch 32. It reused the exact
5,000 prompt + dynamic-depth pairs from the preceding ControlNet Depth run:

- Depth videos: [Yanbin99/Depth-Normal-Videos-42K](https://huggingface.co/datasets/Yanbin99/Depth-Normal-Videos-42K), revision `9c5d205edd71077bbab7d96cae971a25a1fbf91f`.
- Corresponding captions: [fal/cosmos-openvid-1m](https://huggingface.co/datasets/fal/cosmos-openvid-1m), revision `10b41fc29006eff62ff64b8795b8ae8ef7ff9cde`.
- Training geometry: 832 x 480, 93 raw video frames, 24 latent frames.
- Generator, frozen teacher, and trainable critic each load the full Fun-Control DiT.
- Generator and critic have independent rank-64 adapters; the teacher is frozen.

The published adapter contains 600 FP32 tensors over 300 Linear layers in
30 blocks: self-attention q/k/v/o, cross-attention q/k/v/o, and FFN 0/2.
There are 161,218,560 adapter parameters. The full Fun-Control backbone and
its learned control conditioning remain required.

The source training bundle SHA256 is
`47f94f7fc85ed53b5480b28c2bfa3a05659bcf4aeb1278004c0860224787e78a`.
Both exported formats were compared tensor-by-tensor to its `generator_lora`
state. All 3,600 LoRA video inference receipts from this evaluation record
the same source checksum, complete loading of 600 tensors / 300 target layers,
four denoising steps and CFG 1.

## Loading

Use the native VideoX-Fun Wan2.2-Fun-5B-Control pipeline. The evaluated
VideoX-Fun source revision was `1fd9ed9208ba280f6b48c542a7739e636f4e2a14`.
Start from its [5B Control example](https://github.com/aigc-apps/VideoX-Fun/blob/1fd9ed9208ba280f6b48c542a7739e636f4e2a14/examples/wan2.2_fun/predict_v2v_control_5b.py).

Download the adapter and loader:

```python
from huggingface_hub import snapshot_download

release = snapshot_download(
    "Perflow-Shuai/WAN-Fun-LoRA-5B-Control-TrainCFG4-4Step-CFG1-step1600",
    allow_patterns=["generator_lora.pt", "perflow_lora.py", "load_wan_fun_lora.py"],
)
```

After the example loads the full Control transformer, and **before** applying
CPU/group offloading, attach this adapter once:

```python
import sys
from pathlib import Path

sys.path.insert(0, release)
from load_wan_fun_lora import load_wan_fun_lora

receipt = load_wan_fun_lora(
    transformer,
    Path(release) / "generator_lora.pt",
    strength=1.0,
)
assert receipt["loaded_targets"] == 300
assert receipt["loaded_tensors"] == 600
```

In that example, set `sampler_name="Flow_Unipc"`, `num_inference_steps=4`,
`guidance_scale=1.0`, `shift=5`, and `enable_teacache=False`. Its UniPC branch
sets the scheduler constructor's shift to 1 and the pipeline call applies
runtime shift 5. Supply your prompt and dynamic depth video via its existing
control-video inputs. Preserve the model's depth/inpainting/reference
conditioning preprocessing, including the learned `ref_conv` behavior.

`adapter_model.safetensors` plus `adapter_config.json` is the equivalent
PEFT/native-Wan export. Keys use `blocks.*.self_attn.*`, `cross_attn.*`, and
`ffn.*`. Diffusers-specific key conversion and generic ComfyUI LoRA loading
were not validated for this release. The included loader takes
`generator_lora.pt`; its shared implementation is in `perflow_lora.py`.

## PAI-Bench-C Depth evaluation

We evaluated 600 Depth cases, with six prompt variants per control for
diversity (3,600 generated videos per local model). Local outputs use 121
frames at 24 FPS and seed 43. The four-step variants use identical inputs,
resolution, seed and scheduler; their difference is this adapter at strength 1.

| Model | Blur SSIM ↑ | Edge F1 ↑ | Depth si-RMSE ↓ | Mask mIoU ↑ | Quality ↑ | Diversity ↑ |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Official reported Wan2.2-Fun-5B-Control | 0.556 | 0.106 | 1.819 | 0.615 | 9.317 | 0.481 |
| Local base, 40 steps, CFG 6, TeaCache | 0.557 | 0.099 | 4.836 | 0.600 | 8.962 | 0.474 |
| Local base, 4 steps, CFG 1 | 0.682 | 0.089 | 2.654 | 0.637 | 8.896 | 0.248 |
| **WAN Fun LoRA step1600, 4 steps, CFG 1** | **0.544** | **0.092** | **1.805** | **0.589** | **9.290** | **0.458** |

The official row is quoted from [PAI-Bench](https://github.com/SHI-Labs/physical-ai-bench)
and was not run by us. Local main metrics use its contemporaneous legacy
evaluator; diversity uses the current implementation, whose source is identical
across the two snapshots. Exact revisions and values are in `evaluation.json`.
The official run's private configuration is unavailable, so the local multistep
row is a best-effort reproduction. Legacy depth RMSE has outliers; these rows
should not be read as proof that the LoRA surpasses the official base overall.

Compared with direct four-step inference, this LoRA improves Depth, Quality,
Diversity and slightly Edge F1, while Blur SSIM and Mask mIoU decrease.
Blur SSIM measures similarity after blurring and is not a sharpness metric.
The adapter was trained for Depth; other control modalities and transfer to
unmodified Wan backbones are not established by these results.

## Files and terms

- `generator_lora.pt`: generator-only payload with `generator_lora` and `step`.
- `adapter_model.safetensors`, `adapter_config.json`: equivalent FP32 adapter.
- `load_wan_fun_lora.py`, `perflow_lora.py`: strict native-Wan loader.
- `training_config.yaml`, `inference_config.yaml`: training and inference settings.
- `release_metadata.json`, `provenance.json`, `evaluation.json`, `SHA256SUMS`: provenance, measured results and checksums.

The upstream [Fun-Control model card](https://huggingface.co/alibaba-pai/Wan2.2-Fun-5B-Control)
declares Apache-2.0; the linked depth dataset declares MIT and the caption
dataset declares Apache-2.0. This release follows the preceding depth-conditioned
adapter's `license: other` designation because upstream source-video rights
are not established by those dataset labels alone. Consult the respective
upstream terms. Original training videos, captions, base-model weights and
critic weights are not redistributed here.