Card: Pareto frontier, judge detail by area, verified dataset size
Browse files- .gitattributes +2 -0
- README.md +5 -1
- assets/judge_detail.png +3 -0
- assets/pareto.png +3 -0
- assets/qib.png +2 -2
.gitattributes
CHANGED
|
@@ -43,3 +43,5 @@ assets/masthead.png filter=lfs diff=lfs merge=lfs -text
|
|
| 43 |
assets/numbers.png filter=lfs diff=lfs merge=lfs -text
|
| 44 |
assets/qib.png filter=lfs diff=lfs merge=lfs -text
|
| 45 |
assets/side_by_side.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 43 |
assets/numbers.png filter=lfs diff=lfs merge=lfs -text
|
| 44 |
assets/qib.png filter=lfs diff=lfs merge=lfs -text
|
| 45 |
assets/side_by_side.png filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
assets/judge_detail.png filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
assets/pareto.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -59,7 +59,7 @@ shows all of that in pictures, along with every model we trained on the way to A
|
|
| 59 |
generator, 512-token context.
|
| 60 |
- **Decoder:** [SD-VAE-ft-MSE](https://huggingface.co/stabilityai/sd-vae-ft-mse), or
|
| 61 |
[TAESD](https://huggingface.co/madebyollin/taesd) for speed.
|
| 62 |
-
- **Data:** [FLUX-Reason-6M](https://huggingface.co/datasets/LucasFang/FLUX-Reason-6M),
|
| 63 |
English captions each. **Compute:** Arrhenius (EuroHPC, NAISS, Sweden), NVIDIA GH200.
|
| 64 |
- **Licence:** MIT, for the code and the weights.
|
| 65 |
|
|
@@ -182,6 +182,10 @@ and aggregation code, greedy decoding, thinking on.
|
|
| 182 |
| **Agate** | **28.4** | 26.3 | 34.2 | 28.9 | 36.8 | 15.5 |
|
| 183 |
| Stable Diffusion 1.5 | **28.9** | 37.5 | 28.2 | 25.1 | 39.3 | 14.3 |
|
| 184 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 185 |
On all 1,000 prompts Agate scores **28.2** against SD 1.5's **29.1**, close but behind. It leads on aesthetics (33.8 vs 28.0) and alignment (28.2 vs 25.7). It trails on quality (25.9 vs 36.7) and real-world fidelity (36.6 vs 38.9). The judge sees every image at 512 px, so Agate's 256 px output is upscaled first, which likely weighs on its quality score. Across our lineage the total climbs from 17.3 to 28.4 on the 300-prompt subset.
|
| 186 |
|
| 187 |
Judge outputs that failed to parse: 0. Scores are 0–100.
|
|
|
|
| 59 |
generator, 512-token context.
|
| 60 |
- **Decoder:** [SD-VAE-ft-MSE](https://huggingface.co/stabilityai/sd-vae-ft-mse), or
|
| 61 |
[TAESD](https://huggingface.co/madebyollin/taesd) for speed.
|
| 62 |
+
- **Data:** [FLUX-Reason-6M](https://huggingface.co/datasets/LucasFang/FLUX-Reason-6M), 5,654,461 of its images, each with 8
|
| 63 |
English captions each. **Compute:** Arrhenius (EuroHPC, NAISS, Sweden), NVIDIA GH200.
|
| 64 |
- **Licence:** MIT, for the code and the weights.
|
| 65 |
|
|
|
|
| 182 |
| **Agate** | **28.4** | 26.3 | 34.2 | 28.9 | 36.8 | 15.5 |
|
| 183 |
| Stable Diffusion 1.5 | **28.9** | 37.5 | 28.2 | 25.1 | 39.3 | 14.3 |
|
| 184 |
|
| 185 |
+
<p align="center"><img src="assets/pareto.png" alt="Qwen-Image-Bench total against generator size, with the Pareto frontier" width="100%"></p>
|
| 186 |
+
|
| 187 |
+
<p align="center"><img src="assets/judge_detail.png" alt="Every Qwen-Image-Bench area for Agate and the open models" width="100%"></p>
|
| 188 |
+
|
| 189 |
On all 1,000 prompts Agate scores **28.2** against SD 1.5's **29.1**, close but behind. It leads on aesthetics (33.8 vs 28.0) and alignment (28.2 vs 25.7). It trails on quality (25.9 vs 36.7) and real-world fidelity (36.6 vs 38.9). The judge sees every image at 512 px, so Agate's 256 px output is upscaled first, which likely weighs on its quality score. Across our lineage the total climbs from 17.3 to 28.4 on the 300-prompt subset.
|
| 190 |
|
| 191 |
Judge outputs that failed to parse: 0. Scores are 0–100.
|
assets/judge_detail.png
ADDED
|
Git LFS Details
|
assets/pareto.png
ADDED
|
Git LFS Details
|
assets/qib.png
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|