Incorporo-user commited on
Commit
515a7da
·
verified ·
1 Parent(s): 31a7041

Card: Pareto frontier, judge detail by area, verified dataset size

Browse files
.gitattributes CHANGED
@@ -43,3 +43,5 @@ assets/masthead.png filter=lfs diff=lfs merge=lfs -text
43
  assets/numbers.png filter=lfs diff=lfs merge=lfs -text
44
  assets/qib.png filter=lfs diff=lfs merge=lfs -text
45
  assets/side_by_side.png filter=lfs diff=lfs merge=lfs -text
 
 
 
43
  assets/numbers.png filter=lfs diff=lfs merge=lfs -text
44
  assets/qib.png filter=lfs diff=lfs merge=lfs -text
45
  assets/side_by_side.png filter=lfs diff=lfs merge=lfs -text
46
+ assets/judge_detail.png filter=lfs diff=lfs merge=lfs -text
47
+ assets/pareto.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -59,7 +59,7 @@ shows all of that in pictures, along with every model we trained on the way to A
59
  generator, 512-token context.
60
  - **Decoder:** [SD-VAE-ft-MSE](https://huggingface.co/stabilityai/sd-vae-ft-mse), or
61
  [TAESD](https://huggingface.co/madebyollin/taesd) for speed.
62
- - **Data:** [FLUX-Reason-6M](https://huggingface.co/datasets/LucasFang/FLUX-Reason-6M), about 6M images with 8
63
  English captions each. **Compute:** Arrhenius (EuroHPC, NAISS, Sweden), NVIDIA GH200.
64
  - **Licence:** MIT, for the code and the weights.
65
 
@@ -182,6 +182,10 @@ and aggregation code, greedy decoding, thinking on.
182
  | **Agate** | **28.4** | 26.3 | 34.2 | 28.9 | 36.8 | 15.5 |
183
  | Stable Diffusion 1.5 | **28.9** | 37.5 | 28.2 | 25.1 | 39.3 | 14.3 |
184
 
 
 
 
 
185
  On all 1,000 prompts Agate scores **28.2** against SD 1.5's **29.1**, close but behind. It leads on aesthetics (33.8 vs 28.0) and alignment (28.2 vs 25.7). It trails on quality (25.9 vs 36.7) and real-world fidelity (36.6 vs 38.9). The judge sees every image at 512 px, so Agate's 256 px output is upscaled first, which likely weighs on its quality score. Across our lineage the total climbs from 17.3 to 28.4 on the 300-prompt subset.
186
 
187
  Judge outputs that failed to parse: 0. Scores are 0–100.
 
59
  generator, 512-token context.
60
  - **Decoder:** [SD-VAE-ft-MSE](https://huggingface.co/stabilityai/sd-vae-ft-mse), or
61
  [TAESD](https://huggingface.co/madebyollin/taesd) for speed.
62
+ - **Data:** [FLUX-Reason-6M](https://huggingface.co/datasets/LucasFang/FLUX-Reason-6M), 5,654,461 of its images, each with 8
63
  English captions each. **Compute:** Arrhenius (EuroHPC, NAISS, Sweden), NVIDIA GH200.
64
  - **Licence:** MIT, for the code and the weights.
65
 
 
182
  | **Agate** | **28.4** | 26.3 | 34.2 | 28.9 | 36.8 | 15.5 |
183
  | Stable Diffusion 1.5 | **28.9** | 37.5 | 28.2 | 25.1 | 39.3 | 14.3 |
184
 
185
+ <p align="center"><img src="assets/pareto.png" alt="Qwen-Image-Bench total against generator size, with the Pareto frontier" width="100%"></p>
186
+
187
+ <p align="center"><img src="assets/judge_detail.png" alt="Every Qwen-Image-Bench area for Agate and the open models" width="100%"></p>
188
+
189
  On all 1,000 prompts Agate scores **28.2** against SD 1.5's **29.1**, close but behind. It leads on aesthetics (33.8 vs 28.0) and alignment (28.2 vs 25.7). It trails on quality (25.9 vs 36.7) and real-world fidelity (36.6 vs 38.9). The judge sees every image at 512 px, so Agate's 256 px output is upscaled first, which likely weighs on its quality score. Across our lineage the total climbs from 17.3 to 28.4 on the 300-prompt subset.
190
 
191
  Judge outputs that failed to parse: 0. Scores are 0–100.
assets/judge_detail.png ADDED

Git LFS Details

  • SHA256: 0ce2273c035adf0736ecf7d67c52f7ac4eb22091065a6e19c37e299914b5ca47
  • Pointer size: 131 Bytes
  • Size of remote file: 315 kB
assets/pareto.png ADDED

Git LFS Details

  • SHA256: 02112edcb526dd7a7d9a1167d039dbad1c5968b5420b54e680f1e45d433b9c6c
  • Pointer size: 131 Bytes
  • Size of remote file: 165 kB
assets/qib.png CHANGED

Git LFS Details

  • SHA256: fc46d329cb8ed9963adf47969d954be6f71d171c2a6570cd487e86d5646444dd
  • Pointer size: 131 Bytes
  • Size of remote file: 166 kB

Git LFS Details

  • SHA256: c3cb1cc92c688b27a1942b6235b6047118045981170fd0b5ebbfa7fea611bb71
  • Pointer size: 131 Bytes
  • Size of remote file: 186 kB