eval/benchmark_summary.md

# IrishCore-GlobalPointer-ContextPII-135M-v1-rc10

## ONNX q8

| Suite | F1 | Examples/s |
| --- | ---: | ---: |
| Irish core | 1.0000 | 125.8211 |
| Irish extended | 1.0000 | 43.9857 |
| Demographic holdout v2 | 1.0000 | 81.6769 |
| Gov contact policy v1 | 1.0000 | 81.2823 |
| Gov chatbot red-team v2 | 0.9861 | 76.3120 |
| Gov chatbot gap holdout v2 | 1.0000 | 77.8256 |
| Context red-team v1 | 1.0000 | 77.3513 |
| Context red-team v2 | 1.0000 | 83.4298 |
| Context red-team v3 | 1.0000 | 115.7841 |
| Context red-team v4 | 1.0000 | 115.2060 |
| Location coverage v1 | 1.0000 | 86.6758 |
| Location coverage v2 | 1.0000 | 63.4965 |
| Location coverage v3 | 1.0000 | 103.8788 |
| Numeric qafix v2 | 1.0000 | 138.1308 |
| Multilingual PPSN overall | 0.9333 | 113.8253 |
| Multilingual PPSN label-only | 1.0000 | — |

## Full checkpoint

| Suite | F1 | Examples/s |
| --- | ---: | ---: |
| Irish core | 1.0000 | 31.3243 |
| Irish extended | 1.0000 | 20.9516 |
| Demographic holdout v2 | 1.0000 | 40.7134 |
| Gov contact policy v1 | 1.0000 | 22.0155 |
| Gov chatbot red-team v2 | 0.9861 | 30.5354 |
| Gov chatbot gap holdout v2 | 1.0000 | 51.4926 |
| Context red-team v1 | 1.0000 | 23.6091 |
| Context red-team v2 | 1.0000 | 12.7370 |
| Context red-team v3 | 1.0000 | 17.8810 |
| Context red-team v4 | 1.0000 | 30.4955 |
| Location coverage v1 | 1.0000 | 27.4425 |
| Location coverage v2 | 1.0000 | 24.3613 |
| Location coverage v3 | 1.0000 | 35.4460 |
| Numeric qafix v2 | 1.0000 | 29.4201 |
| Multilingual PPSN overall | 0.9333 | 56.5542 |
| Multilingual PPSN label-only | 1.0000 | — |