maximsobolev275/PinkCherry-b6-H3-lora-r768-convrot

PinkCherry MiniMax-H3 LoRA (beta-0.6, rank 256, bf16)

LoRA adapter extracted from PinkCherry MiniMax H3 bf16 beta-0.6 finetune relative to the MiniMax-H3 base model. Captures the full model delta in ComfyUI-native LoRA format (bf16, single .safetensors file with diffusion_model. prefix).

LoRA specifications

Spec	Value
Rank	256
Dtype	BF16
Format	ComfyUI LoRA (single .safetensors, diffusion_model. prefixed keys)
Source FT	PinkCherry_MiniMax_H3_bf16_beta-0.6.safetensors (~66 GB)
Base	minimax_h3_fl2va_bf16.safetensors (Comfy-Org/MiniMax-H3, 66.3 GB)
Method	Direct ΔW = W_ft − W_base → randomized SVD (niter=4)
Tensor coverage	All 266 weight layers + biases/norms (1D via rank-1)

What's inside the LoRA

The adapter contains every tensor present in both FT and base with matching shapes — no layers are skipped, even identical ones:

Category	Method	Count
2D+ weight layers (attn projections, MLP, etc.)	Randomized SVD at rank min(256, M, K)	266
1D tensors (biases, layer norms, scale_shift_table)	Rank-1 approximation: A=[[1.0]], B=ΔW.reshape(N,1)	full coverage
Identical layers (ΔW = 0)	Zero-filled placeholder	included

This guarantees ComfyUI finds every expected key when applying the LoRA — no missing-key errors.

Finetune analysis (bf16 vs bf16)

Comparison run across all 266 weight layers in the model.

Summary

Metric	Value
Total layers compared	266
Identical (ΔW = 0)	66 (24.8%)
Changed (ΔW > 0)	200 (75.2%)
Significant (ΔW > 0.5%)	1 (0.4%)
Strong (ΔW > 5%)	0 (0.0%)
**Avg ΔW /
**Max ΔW /
Avg SNR (changed layers)	2.46× (signal vs bf16 noise floor)
Avg cosine similarity (changed)	1.0220 (≈ same direction)
Std ratio (FT / base)	1.000004 (dequantization bit-identical)

Verdict: very light finetune — the delta is well above bf16 quantization noise (SNR ≈ 2.5) but very small in absolute terms (avg 0.12%). The model is mostly identical to base, with localized changes in attention projections and MLP layers.

Top-5 most changed layers

| Layer | ΔW / ||W|| | SNR | cos | |---|---|---|---| | blocks.49.mlp.fc2.weight | 2.785% | 55.70 | 1.01769 | | blocks.28.attn.out_proj.weight | 0.301% | 6.01 | 1.00501 | | blocks.29.attn.out_proj.weight | 0.292% | 5.85 | 1.00504 | | blocks.27.attn.out_proj.weight | 0.288% | 5.76 | 1.00533 | | blocks.26.attn.out_proj.weight | 0.263% | 5.26 | 1.00569 |

Aggregate by layer type

| Layer type | N | Avg ΔW/||W|| | Avg SNR | Signal | |---|---|---|---|---| | mlp.fc2 | 50 | 0.001522 | 3.04 | ✓ good | | attn.out_proj | 50 | 0.001434 | 2.87 | ✓ good | | attn.qkv_proj | 50 | 0.001140 | 2.28 | ✓ good | | mlp.fc1 | 50 | 0.000829 | 1.66 | ~ medium | | adaln_proj.linear | 50 | 0.000000 | 0.00 | ✗ unchanged | | patch_proj | 2 | 0.000000 | 0.00 | ✗ unchanged | | time_embedder.proj_in/out | 2 | 0.000000 | 0.00 | ✗ unchanged | | token_refiner.* | 8 | 0.000000 | 0.00 | ✗ unchanged | | condition_proj | 1 | 0.000000 | 0.00 | ✗ unchanged | | final_layer.* | 3 | 0.000000 | 0.00 | ✗ unchanged |

What was changed by the finetune

• Attention projections (attn.out_proj, attn.qkv_proj) — light changes across all 50 blocks
• MLP layers (mlp.fc1, mlp.fc2) — light changes across all 50 blocks
• Block 49 mlp.fc2 — the only layer with substantial change (2.8%)

What was NOT changed

• All adaln_proj.linear layers (50)
• All patch_proj, audio_patch_proj layers
• All time_embedder.proj_in/out layers
• All token_refiner.* layers (8)
• All final_layer.* layers
• All biases (1D) and condition_proj

Rank recommendation

The diagnostic suggests rank 64 would be sufficient for this finetune magnitude. The released version is rank 256 (over-provisioned, captures noise too). If you want a smaller file with the same effective signal, re-extract at rank 64.

How to use in ComfyUI

1. Drop the .safetensors file into ComfyUI/models/loras/
2. In your workflow, add a Load LoRA node
3. Set strength to 1.0 (or lower — try 0.7–0.9 for subtler effect)
4. Connect to your MiniMax-H3 model loader

The LoRA keys are prefixed with diffusion_model. (ComfyUI convention for MiniMax-H3). No additional setup needed.

Files

• PinkCherry-b06_H3_LoRA_r256_bf16_comfy.safetensors — the LoRA adapter (bf16, rank 256)

Extraction pipeline

1. Stream base + FT headers via HTTP Range (~60 KB each)
2. For each common weight tensor:
   • Stream base tensor via HTTP Range (no full-file download)
   • Stream FT tensor via HTTP Range
   • Compute ΔW = W_ft − W_base directly in bf16
   • Randomized SVD (torch.svd_lowrank, niter=4) → lora_A = V.t(), lora_B = U @ diag(S)
   • For 1D tensors: rank-1 approximation (A=[[1.0]], B=ΔW.reshape(N,1))
3. Write single .safetensors file with pre-computed header (streaming, low RAM)

Disk footprint during extraction: ~7 GB (LoRA + buffer). Runtime: ~30 min on Colab Free T4 (HTTP bound).

Acknowledgements

• Base model: Comfy-Org/MiniMax-H3
• Finetune: SexGod1979/PinkCherry_MiniMax-H3 (beta-0.6)
• LoRA format: ComfyUI-native safetensors with diffusion_model. prefix

License

Same as the underlying MiniMax-H3 model. See the base model card for details.