---
license: mit
base_model: inclusionAI/Ming-Image-0.1-Design
base_model_relation: quantized
pipeline_tag: text-to-image
tags:
- text-to-image
- rocm
- amd
- strix-halo
- gfx1151
- int8
- design
- rgba
---

# Ming-Image-0.1-Design — ROCm build (AMD Strix Halo, gfx1151) · INT8 MLLM · paired with Ling-3.0-flash-VL

[inclusionAI/Ming-Image-0.1-Design](https://huggingface.co/inclusionAI/Ming-Image-0.1-Design) (text-to-image for
UI, posters and infographics, RGBA output) made to run on AMD ROCm, with the routed experts of its 17B-parameter
MoE language model stored as weight-only INT8, and wired to the prompt enhancer its model card names: **Ling-3.0-flash-VL**
(served from our [ROCmFP4 build](https://huggingface.co/kingjones777/Ling-3.0-flash-VL-MTP-ROCmFP4-GGUF)).

Everything below was measured on one AMD Ryzen AI Max+ 395 (Radeon 8060S, gfx1151) — see
[Reproduction](#reproduction). Nothing here was run on CUDA.

## Results (1024 × 1024, 12 steps, cfg 1.0, seed 42, four prompts)

| prompt (1024², 12 steps) | BF16 · upstream code | BF16 · contiguous attention | INT8 · contiguous attention | INT8 · fast attention |
|---|---:|---:|---:|---:|
| four-seasons cabin (upstream's example prompt) | 339.9 s | 211.1 s | 181.1 s | 77.8 s |
| banking-app screen (Ling rewrite) | 501.8 s | 270.0 s | 259.7 s | 88.5 s |
| jazz-night poster (Ling rewrite) | 355.5 s | 221.4 s | 219.9 s | 79.4 s |
| water-cycle infographic (Ling rewrite) | 606.7 s | 312.0 s | 250.0 s | 100.3 s |
| **mean** | 451.0 s | 253.6 s | 227.7 s | 86.5 s |
| speed-up vs upstream code | 1.00× | 1.78× | 1.98× † | 5.21× |

| configuration | PyTorch peak allocated | Ming's own GTT peak | box |
|---|---:|---:|---|
| BF16 · upstream code, all components resident | 58.7 GiB | 74.5 GiB | Ling stopped |
| BF16 · contiguous attention | 58.9 GiB | 74.5 GiB | Ling stopped |
| BF16 · contiguous · `--release-mllm-after-conditioning` (cabin) | 47.2 GiB | — (baseline not settled) | Ling stopped |
| **INT8 · contiguous · `--release-mllm-after-conditioning`** | **33.6 GiB** | **35.8 GiB** | Ling-3.0-flash-VL resident (67.5 GiB) → box peak 103.3 GiB of 128 GiB GTT |
| INT8 · fast attention · release | 33.6 GiB | 35.1 GiB | Ling resident |

- **Contiguous attention is the big one:** 1.78× faster end to end, and the images are **byte-identical** to the upstream code path (4/4 prompts plus the `--release` run, compared with `cmp`).
- † **INT8** runs the same BF16 diffusion transformer, so it does not change the step time: its 227.7 s mean vs 253.6 s for BF16 with the same attention reflects when the runs happened (later, with Ling resident), not the quantization. What INT8 does cost is the conditioning pass, where the weights are dequantized on the fly: 2.4–4.7 s vs 1.1–2.0 s in BF16. INT8 exists for memory: with `--release-mllm-after-conditioning` Ming needs 35.8 GiB at its peak, which fits next to the resident Ling-3.0-flash-VL (67.5 GiB) on a 128 GiB box — BF16 with the upstream code needed 74.5 GiB and did not. Loading takes 30–100 s; the slow end is a cold page cache (first load after the files were written or after Ling was restarted).
- **Fast attention** (`--attention-bf16-reduction`) is opt-in: 5.21× vs upstream, at the fidelity cost shown below.
- The very first image on a fresh box is slower once: the reference cabin took 608.4 s the first time and 339.9 s warm (MIOpen tunes the VAE's 3-D convolutions and caches the result in `~/.cache/miopen`).

### Fidelity against the BF16 original

Inference on this box is **deterministic**: the same prompt and seed produced a byte-identical PNG twice, in BF16
and in INT8, so every difference below is caused by the quantization (or by the fast-attention option), not by
run-to-run noise.

| prompt | INT8 SSIM | INT8 PSNR | cond cos (query / VLM tokens) | cond rel L2 (query / VLM) | INT8 fast SSIM | INT8 fast PSNR |
|---|---:|---:|---:|---:|---:|---:|
| four-seasons cabin (upstream's example prompt) | 0.9345 | 22.90 dB | 0.98169 / 0.99672 | 0.1931 / 0.0810 | 0.9150 | 22.13 dB |
| banking-app screen (Ling rewrite) | 0.9666 | 23.76 dB | 0.98638 / 0.99623 | 0.1655 / 0.0868 | 0.9624 | 22.75 dB |
| jazz-night poster (Ling rewrite) | 0.9450 | 23.31 dB | 0.98874 / 0.99587 | 0.1504 / 0.0909 | 0.9456 | 23.55 dB |
| water-cycle infographic (Ling rewrite) | 0.8486 | 17.49 dB | 0.99061 / 0.99500 | 0.1378 / 0.1000 | 0.8439 | 17.42 dB |

`cos` / `rel L2` compare the conditioning tensors the diffusion transformer receives (the MLLM output — the only
part that is quantized). SSIM is windowed 7×7 on luminance; PSNR over RGB.

## Run it

```bash
hf download kingjones777/Ming-Image-0.1-Design-ROCm-INT8 --local-dir Ming-Image-0.1-Design-ROCm-INT8
cd Ming-Image-0.1-Design-ROCm-INT8
# caption -> Ling-3.0-flash-VL rewrites it into the layered JSON prompt -> Ming renders it (one image per process)
bash code/generate_paired.sh --model . --base-url http://127.0.0.1:8090/v1 \
    "A minimalist concert poster for a jazz night called \"Blue Hour\"" --resolution 1024 --output-dir out \
    -- --device-map none --release-mllm-after-conditioning        # add --attention-bf16-reduction for fast mode
```

`PYTHON=/path/to/python` selects the interpreter (it needs a ROCm build of PyTorch and
[`code/requirements-rocm.txt`](code/requirements-rocm.txt)); `--pe-model` names the chat model your endpoint serves.
This is the command shape our end-to-end run used (see [Samples](#samples)); 1024² is the resolution we measured —
`infer.py`'s own default is 2048², which we did not run.

`generate_paired.sh` calls `pe_ling.py` (standard library only) against an OpenAI-compatible endpoint
(default `http://127.0.0.1:8090/v1`, our Ling-3.0-flash-VL llama-server seat), validates the rewrite against the
schema the upstream system prompt demands (one retry with the errors), then runs `infer.py`. You can also pass
your own JSON prompt straight to `infer.py --prompt prompt.json`.

The code is in [`code/`](code/): upstream `inclusionAI/Ming-Image` at `62c6072` plus the changes below
([`code/rocm.patch`](code/rocm.patch)).

## What changed for ROCm, and why

1. **`transformer_engine` removed.** The vision tower imported NVIDIA Transformer Engine (CUDA-only) for a
   single `te.RMSNorm`; it is now a plain RMSNorm with the same parameter name (`weight`), so all 65 vision norm
   tensors in the checkpoint load unchanged. A dead import in `modeling_bailing_moe_v2.py` went too.
2. **`--attn-implementation eager` now reaches the towers.** `BailingMM2Config` declared no `sub_configs`, so
   transformers never copied the chosen attention implementation into the vision and language configs; their
   `flash_attention_2` defaults raised `ImportError` at model construction on any box without flash-attn. Declaring
   `sub_configs` fixes it. (Upstream's `--validate-only` cannot catch this — it never builds the model.)
3. **Contiguous attention inputs.** The diffusion transformer handed PyTorch's attention permuted views of
   `[B, L, H, D]` tensors. On gfx1151 the only working SDPA kernel is the math one, and with those strides its
   fp32 GEMMs fall onto an 8×8×8 macro-tile: 644 ms per attention call at the cabin prompt's length
   (L = 5,759) versus 363 ms for the same tensors made contiguous, bit-identical output. The profiler put
   SDPA-math at 87.6% of all GPU time before the fix. `diffusion/transformer.py` now passes
   contiguous tensors whenever diffusers' default native backend is active.
4. **INT8 MLLM — routed experts only.** Weight-only, per-output-channel symmetric INT8 (fp32 scales) for the
   256 routed experts of the language model's 19 MoE layers — 14,592 Linears, which hold
   almost all of its weights. Everything that runs on every token stays byte-identical BF16: attention
   (`query_key_value`, `dense`), the shared experts, layer 0's dense MLP, the routers (`gate`, `image_gate`,
   `audio_gate`), embeddings, `lm_head`, norms and the vision tower (691 tensors).
   `mllm/` goes from 34.00 GB to 18.76 GB. Relative weight error
   ‖W − Ŵ‖/‖W‖: mean 0.00833, p99 0.01035, max
   0.01412. `quant/quantize_stream.py` writes it shard by shard without building the model
   (6.7 min; peak RSS 17.84 GiB, sampled on an earlier run of the same
   tool); `quant/load_int8.py` builds the model on the meta device and loads the INT8 shards straight onto the GPU, so
   BF16 weights for the quantized layers never exist in memory. The scales stay fp32 through `.to(bfloat16)`.
   *Measured alternative:* quantizing the always-on Linears too saves another 0.30 GB but raised the mean
   conditioning error (rel L2, VLM tokens) from 0.090 to 0.108 and changed the cabin prompt's
   surround from scenery to white (SSIM 0.699); mean SSIM over the four prompts 0.877 vs 0.924.
5. **Connector stored as bf16.** It shipped as float32 (6.17 GB) but `infer.py` always loads it as bf16; storing
   it pre-rounded halves the download and every tensor equals `fp32.to(bfloat16)` exactly (all 338 checked).
6. **`--release-mllm-after-conditioning`** (opt-in, one image per process): the language model, vision tower and
   connector are only needed for the ~1–2 s conditioning pass, so they are freed before the 12 diffusion steps.
   PyTorch peak allocated on the cabin prompt (BF16): 55.6 → 47.2 GiB, byte-identical image.
7. **`--attention-bf16-reduction`** (opt-in): lets the math SDPA kernel stay in bf16 instead of upcasting to fp32 —
   363 → 115 ms per call, attention error vs an fp32 reference 1.658e-03 → 5.032e-03
   (rel L2, random inputs). Image-level cost is in the fidelity table.

## Things that do not work on gfx1151 (measured, so you don't have to)

- **Only the math SDPA kernel runs.** AOTriton's efficient and flash kernels are unavailable by default
  (`UNAVAILABLE: No available kernel. Aborting execution.`).
- **Do not set `TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1`**, even though PyTorch's warning suggests it: the shipped
  `amd-gfx11xx` AOTriton kernels then get selected and fail (`UNAVAILABLE: HIP error: invalid argument`), and a full
  generation crashes with the same `HIP error: invalid argument`.
- **The first image on a fresh box is slower once.** The same cabin prompt took 608.4 s the
  first time and 339.9 s warm; MIOpen tunes the VAE's 3-D convolutions on first use and its cache
  (`~/.cache/miopen`) grew during that first run.
- **Inference is deterministic here, BF16 and INT8:** same prompt + seed → byte-identical PNG (both checked), which
  is what makes the fidelity numbers above exact rather than statistical.

## Pairing with Ling-3.0-flash-VL

Ming-Image's text-to-image quality depends on a structured, Figma-like JSON prompt (canvas settings +
positioned layers with colours). Upstream ships the rewriter system prompt (`assets/t2i_rewriter_system_prompt.txt`) and names Ling-3.0-flash-VL as the model to run it.
`pe_ling.py` sends that prompt verbatim plus your caption to any OpenAI-compatible endpoint, extracts the JSON, validates it, and retries once with the validation errors if needed.

Measured against our Ling-3.0-flash-VL seat (llama-server, `ling-3.0-flash-vl-mtp-halo-STRIX_LEAN`, same box) — all three valid on the first attempt:

| caption | rewrite time | layers |
|---|---:|---:|
| banking-app screen (Ling rewrite) | 45.9 s | 7 |
| jazz-night poster (Ling rewrite) | 34.8 s | 8 |
| water-cycle infographic (Ling rewrite) | 49.8 s | 14 |

The exact captions and the JSON Ling returned are in [`samples/`](samples/).

## Samples

INT8 build, `--release-mllm-after-conditioning`, 1024², 12 steps, cfg 1.0, seed 42 — exactly the runs in
the tables above. Full-resolution RGBA PNGs and the JSON prompts are in `samples/`.

**cabin upstream** — upstream's own structured prompt (`assets/t2i_four_seasons_cabin_prompt.json`), no rewrite · [prompt JSON](samples/cabin_upstream.json)

![cabin_upstream](samples/cabin_upstream.png)

**ui banking** — caption → Ling-3.0-flash-VL: *A mobile banking app home screen: a balance card at the top, a recent transactions list, quick-action buttons for send, pay and top up, and a bottom navigation bar. Clean modern fintech style.* · [prompt JSON](samples/ui_banking.json)

![ui_banking](samples/ui_banking.png)

**poster jazz** — caption → Ling-3.0-flash-VL: *A minimalist concert poster for a jazz night called "Blue Hour" on Friday, October 3, 8 PM at The Lantern Room, with a saxophone silhouette over a deep blue gradient.* · [prompt JSON](samples/poster_jazz.json)

![poster_jazz](samples/poster_jazz.png)

**info water** — caption → Ling-3.0-flash-VL: *An infographic that explains the water cycle in four labeled stages - evaporation, condensation, precipitation, collection - with arrows and simple flat icons.* · [prompt JSON](samples/info_water.json)

![info_water](samples/info_water.png)

### End to end on the box

`ming-paired` (our box's wrapper around `generate_paired.sh`, not part of this repo) with Ling-3.0-flash-VL and two other model servers running: it paused 2 of them for the render and restarted them afterwards (halo-bonsai8b, halo-bonsai). Ling rewrote the caption in 48.3 s (9 layers); the whole command took 323 s.

*A landing page hero section for a coffee subscription service called Daily Grind: the headline Fresh beans every Monday, a short tagline, a Start your subscription button, and a photo of latte art on the right. Warm earthy palette.* · [prompt JSON](samples/e2e_daily_grind.json)

![end to end](samples/e2e_daily_grind.png)

## Files

| path | size | what |
|---|---:|---|
| `mllm/` | 18.78 GB | language model + vision tower; the 14,592 routed-expert Linears are **INT8** (`int8_manifest.json` lists them) |
| `transformer/` | 12.31 GB | diffusion transformer, BF16, unchanged |
| `connector/` | 3.09 GB | Qwen2 connector, **stored as bf16** (upstream ships fp32; runtime identical) |
| `vae/` | 253.82 MB | VAE (4-channel RGBA), unchanged |
| `mlp/` | 124.84 MB | conditioning MLP, unchanged |
| `scheduler/` | 173 B | flow-matching scheduler config, unchanged |
| `code/` | 2.31 MB | patched inference code + `rocm.patch` + tools |
| `samples/` | 4.56 MB | the sample images and prompts shown above |
| **total** | **34.56 GB** | upstream: 52.88 GB |

## Reproduction

```
upstream : inclusionAI/Ming-Image-0.1-Design  revision 1cd7fac3b0dcb54196fe2cd12b80da09edf8fcf4
code     : inclusionAI/Ming-Image @ 62c6072e1ff15af83f7c4963a0a1954c1424e80e + code/rocm.patch  (branch rocm-halo @ f986f7a (upstream inclusionAI/Ming-Image 62c6072 + 3 commits); rocm.patch sha256 03fe16f8ed566bf99ec654caa687c7e659f489df6d077cfb7fc7aee134ebc777)
python   : 3.13.5 · torch 2.10.0 (HIP 7.13.99004) · transformers 4.57.1 · diffusers 0.36.0 · accelerate 1.13.0 · safetensors 0.8.0
box      : amd-halo · AMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 125 GiB RAM visible, 128 GiB GTT · ROCm 7.13.0 · kernel 6.18.35+rex+2-amd64
power    : platform_profile=balanced · governor=powersave · GPU 83–107 W at 100% busy (step probe, 0.5 s samples)
env      : nothing set; in particular TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL is NOT set (it crashes gfx1151)
date     : 2026-09-22 to 2026-09-23 (runs crossed midnight, America/Chicago)
```

Build the INT8 package from the upstream download:

```bash
python code/quant/quantize_stream.py <upstream>/mllm <package>/mllm \
    --exclude '\.attention\.|\.shared_experts\.|^model\.model\.layers\.0\.mlp\.'   # routed experts only
python code/tools/convert_connector.py <upstream>/connector <package>/connector # fp32 -> bf16, proven exact
# transformer/ vae/ mlp/ scheduler/ LICENSE are the upstream files, unchanged
python code/tools/verify_package.py <upstream> <package>                        # the checks behind this card
```

Measure (one process per image, the pairing configuration):

```bash
cd code && PYTHONPATH=. python tools/ming_bench.py --prompts <prompt.json> --out <dir> -- \
    --model <package> --task text-to-image --resolution 1024 --device-map none \
    --attn-implementation eager --release-mllm-after-conditioning        # add --attention-bf16-reduction for fast
python tools/fidelity_compare.py <bf16-dir> <int8-dir> --json fidelity.json
```

BF16 references ran with Ling stopped (the box cannot hold both); every INT8 run had Ling-3.0-flash-VL resident.
GPU memory is the amdgpu `mem_info_gtt_used` peak sampled every second, minus a baseline taken after GTT settled.

## License

MIT, same as the original. Model weights, architecture and inference code © 2026 inclusionAI
([LICENSE](LICENSE)). The INT8 quantization, the ROCm changes and the pairing scripts are ours; they are listed in
this card and in `code/rocm.patch`.