--- license: mit language: - en library_name: pytorch pipeline_tag: text-to-image tags: - model-glue - sana - stable-diffusion - model-stitching - safetensors base_model: - stable-diffusion-v1-5/stable-diffusion-v1-5 - Efficient-Large-Model/Sana_600M_512px_diffusers inference: false --- # SD1.5 CLIP → SANA text bridges Three trained bridges translate **77 × 768 SD1.5 CLIP text states** into **77 × 2304 SANA conditioning states and a mask**. The bridge runs as an ordinary deterministic network forward, followed by normal frozen SANA sampling. There is no fitting, retrieval, particle sampling, or candidate selection during inference. These are bridge weights for the [model-glue implementation](https://github.com/255BITS/model-glue), not a complete Diffusers pipeline. The donor text encoder, SANA transformer, and DC-AE decoder are loaded separately at pinned revisions. SANA's native empty conditioning is stored in each checkpoint, so deployment needs no Gemma encoder. ## Choose a checkpoint | File | Role | Selected training update | |---|---|---:| | [model.safetensors](https://huggingface.co/ntc-ai/model-glue-sd15-sana-text/resolve/main/model.safetensors) | Default: validation-selected linear continuation with fixed mask calibration | 1250 | | [alternatives/linear-uncalibrated.safetensors](https://huggingface.co/ntc-ai/model-glue-sd15-sana-text/resolve/main/alternatives/linear-uncalibrated.safetensors) | Same trained head before calibration; retained for its mask tradeoff | 1250 | | [alternatives/expanded-source.safetensors](https://huggingface.co/ntc-ai/model-glue-sd15-sana-text/resolve/main/alternatives/expanded-source.safetensors) | Expanded output head with full-source access and fixed mask calibration | 1000 | The expanded-source architecture repairs a verified blind space in the old source projection, but did not win the combined validation ranking. The default therefore retains the linear head. All three checkpoints include the complete bridge, cloud, calibration buffers, architecture metadata, and pinned source/recipient identities. They load directly without the parent checkpoint or training caches. ## Run Use the implementation revision that was tested for this release: ```bash git clone https://github.com/255BITS/model-glue.git cd model-glue git checkout f67054d329fe5246510a094dd791d485526e3dd6 python3 -m venv .venv .venv/bin/python -m pip install -e . .venv/bin/python - <<'PY' from huggingface_hub import snapshot_download snapshot_download( repo_id="ntc-ai/model-glue-sd15-sana-text", local_dir="artifacts/huggingface/sd15-sana-text", allow_patterns=["model.safetensors", "manifest.json", "SHA256SUMS"], ) PY CUDA_VISIBLE_DEVICES=1 .venv/bin/python -m model_glue.sana_infer \ --checkpoint artifacts/huggingface/sd15-sana-text/model.safetensors \ --prompt 'A park bench on a beach of white sand looking out to the ocean.' \ --seed 87654 --steps 20 --height 512 --width 512 --guidance 4.5 \ --out artifacts/sana-text-example ``` Choose an available CUDA device for your machine. The output directory must be new. The command saves `image.png` and `render.json`, including the checkpoint hash and deterministic-forward verification. To use an alternative, download its path in `allow_patterns` and pass that checkpoint to the same CLI. For a reproducible download, set `revision` to the desired Hugging Face commit hash. The tested deployment uses CLIP FP16 with a fixed batch padded to 16, a FP32 bridge, SANA BF16, and the pinned 512-pixel sampler recipe. Input is limited to 77 tokens. The source skip uses matching sequence lengths; it does not establish word-level alignment between tokenizers. `config.json` describes the default bridge, while the loader treats checkpoint metadata as authoritative. ## Measured quality Selection used **58 validation prompts**, averaging seeds within each prompt. The final test was held out until selection was frozen: **16 prompts × 3 seeds**, 20 sampling steps, 512 × 512, guidance 4.5. Every declared output was scored. | Method | Native-image LPIPS ↓ | Pixel MAE ↓ | CLIP cosine ↑ | |---|---:|---:|---:| | Original parent, comparison baseline | 0.38347 | 0.14984 | 0.30620 | | Uncalibrated linear continuation | 0.37711 | 0.14652 | 0.30744 | | Validation-selected default | 0.37802 | 0.14609 | 0.30757 | | Expanded-source alternative | 0.37778 | 0.14716 | 0.30815 | | Native SANA reference | 0.00000 | 0.00000 | 0.30820 | The default improves LPIPS by **1.42%** and pixel MAE by **2.50%** against the original parent. LPIPS improves on 11/16 prompt averages, but the paired difference of −0.00545 has a 95% prompt-bootstrap interval of **[−0.01769, +0.00817]**. The interval includes no improvement. These are modest observed fidelity gains, not established gains in general prompt accuracy or aesthetics. The test ranking did not replace the frozen validation choice. Mask calibration reduces validation errors from 6 to 4, but increases missed test positions from **1 to 3**; exact test masks decline from 15/16 to 14/16 prompts. The uncalibrated control has slightly better test LPIPS, while the calibrated default has better pixel MAE and CLIP. Composition and attribute errors remain. In this illustrative beach-bench case, the default's orientation is closer to native, while the expanded-source alternative changes the bench color. Columns are native SANA, original parent, selected default, and expanded-source alternative. ![Bench comparison](https://huggingface.co/ntc-ai/model-glue-sd15-sana-text/resolve/main/figures/bench-comparison.jpg) The first four test prompts include failures: the small car between trucks is not recovered, and the swan remains goose-like. These panels show seed 87654; all three seeds contribute to the table. Both displayed panels were visually reviewed; neither contains people. ![First four test prompts and failures](https://huggingface.co/ntc-ai/model-glue-sd15-sana-text/resolve/main/figures/first-four-test-prompts.jpg) ## Training and provenance Seven continuations completed **8,500 updates on GPU 1**, starting from the same trained parent. The original stem, cloud, and masks stayed fixed during training; linear runs trained the output projection, and source-head runs trained the added branch. Mask calibration later folded one validation-fitted scalar into the mask bias. All new cloud controls are fixed; this study establishes no particle-movement advantage. SANA and its decoder were frozen throughout. Training used 8,192 paired captions, paired embedding supervision, and frozen-SANA functional distillation. Full-trajectory and repaired b_cap arms were also tested but did not win selection. These released winners were trained with the supervised distillation recipe. There is no new Anima result in this release. - [Complete study and reproduction details](https://github.com/255BITS/model-glue/blob/f67054d329fe5246510a094dd791d485526e3dd6/docs/results/formulation-retraining-20260905.md) - [Release manifest and per-file hashes](https://huggingface.co/ntc-ai/model-glue-sd15-sana-text/blob/main/manifest.json) - [Checksums](https://huggingface.co/ntc-ai/model-glue-sd15-sana-text/blob/main/SHA256SUMS) - [Training configurations](https://huggingface.co/ntc-ai/model-glue-sd15-sana-text/tree/main/configs) - [Selection and measured results](https://huggingface.co/ntc-ai/model-glue-sd15-sana-text/tree/main/reports) The implementation passed 82 tests, including checkpoint migration, resume, source reachability, mask geometry, and differentiable rollout checks. The Gaussian reference audit has zero cap-parameter-gradient discrepancy. Exact upstream revisions and checkpoint SHA256 values are in the manifest. The release contains learned bridge weights under the repository's MIT license; the separate upstream models retain their own licenses. No donor or recipient model weights, training datasets, optimizer states, or credentials are included.