# Image21-MLX informal evaluation

One seed per case, one machine, same pinned MLX runtime. Sequential BF16-then-Q8 desktop run without repeated or interleaved trials; order, thermal state and other applications may affect timing. Pixel similarity is not a perceptual quality score; no cross-runtime parity claim.

| Case | BF16 s | Q8 s | BF16 peak GiB | Q8 peak GiB | RGB PSNR dB |
|---|---:|---:|---:|---:|---:|
| portrait | 440.91 | 568.86 | 16.41 | 16.19 | 42.03 |
| english_text | 464.32 | 566.88 | 16.43 | 16.20 | 19.89 |
| chinese_text | 474.14 | 567.43 | 16.44 | 16.20 | 36.97 |
| composition | 480.91 | 562.52 | 16.44 | 16.20 | 37.17 |
| texture | 487.64 | 445.07 | 16.43 | 16.20 | 38.41 |
| rgba | 485.45 | 557.30 | 16.42 | 16.20 | 29.81 |
| edit | 606.29 | 665.00 | 18.68 | 16.20 | 51.09 |

MLX peak allocated memory excludes the OS, other applications and some process allocations. It is not minimum machine RAM.

Timing includes prompt encoding, phase-by-phase component loading, denoising and VAE decoding; PNG writing is excluded. The after-image RSS field is a snapshot after component release, not a peak.

Existing system swap is recorded separately from per-image swap change. This desktop-session run leaves other applications open. See system-context.json and the per-model environment.json files.

All editing pairs use the same BF16 portrait input. Visual findings and release limitations are in visual-review.json.

![All evaluated pairs](comparison.png)

## Visual review / 视觉检查

逐对检查七类样本，未发现明显严重的量化退化：中英文文字均正确，杯子数量与顺序、毛衣改色及真实透明通道均保留。英文海报的字体和台灯结构有可见变化，龙贴纸的表情与细节也有变化，其余样本更接近。两种精度的人物图都比提示词要求裁得更紧，透明图均有少量接近零的背景 alpha 残留。这是助手进行的非盲法、每类单种子视觉检查，不代表无损或统计等价。

Assistant inspection of all seven paired cases found no obvious severe quality regression in this small sample. English and Chinese text are correct in both precisions; cup count/order and sweater recoloring succeed; real RGBA transparency is retained. The English poster changes visibly in typeface and lamp structure, and the dragon changes expression/details. Other pairs are closer. Both portraits are cropped more tightly than requested, and both transparency samples contain slight near-zero background alpha. This is an unblinded, single-seed visual check, not evidence of lossless quantization or statistical equivalence.

- **portrait**: Very close to BF16 in facial features, framing, silver hair, blue knit sweater and window light. Small local skin/hair/knit texture changes are visible, without an obvious severe degradation. Same chest-up rather than waist-up crop limitation.
- **english_text**: Exact, legible CREATE WITH LIGHT and September 2026; no extra text. Compared with BF16 the serif typeface/underline and lamp stem/base change visibly, but both remain coherent and satisfy the prompt. This demonstrates non-identical generation, not an obvious readability regression.
- **chinese_text**: Both Chinese lines and comma are correct and legible, matching BF16. Layout, cream backdrop and cup placement remain very close; small latte/foam texture variations are visible. No obvious text accuracy regression in this sample.
- **composition**: Exactly three cups in the requested red/blue/yellow order and one green apple in front of the blue cup. Overall layout is very close to BF16. Minor handle/shadow geometry changes are visible; no counting or spatial-order regression.
- **texture**: Very close to BF16: coherent bird anatomy, fine blue/orange feather detail, wet moss and droplets. Minor local texture/brightness changes without obvious smoothing or broken detail.
- **rgba**: Full-body green dragon with orange wings and white sticker outline. Expression and surface details change visibly from BF16 while silhouette and requested content remain coherent. 60.69% of pixels have alpha exactly zero (BF16 60.47%); both have canvas-edge alpha at most 4/255. No loss of actual transparency; neither output is a perfectly binary matte.
- **edit**: Successful red sweater recoloring with facial features, hair, pose and window background very close to the BF16 edit. Minor local texture changes; no obvious edit-consistency regression. Neither output guarantees pixel-identical preservation outside the garment.