tanpo-ops / evaluation /COMPARE_OPS.md
d4rkninja's picture
docs: evaluation/COMPARE_OPS.md
f4ae83a verified
|
Raw History Blame Contribute Delete
1.11 kB
# DarkLab Base vs FT Compare
**Generated:** 2026-09-18T15:32:32.450342+00:00
**Base model:** `unsloth/LFM2.5-1.2B-Instruct`
**Owner / creator:** DarkNinja Solutions / d4rkninja
## Overall
| Domain | Base % | FT % | Delta | BEATS_BASE |
|---|---:|---:|---:|:---:|
| `ops` | 89.5% | 90.7% | +1.2 | YES |
## ops — per-category
| Category | Base % | FT % | Delta | FT wins |
|---|---:|---:|---:|:---:|
| `capacity_planning` | 94.5% | 100.0% | +5.5 | yes |
| `handoffs` | 100.0% | 92.6% | -7.4 | no |
| `identity_guardrails` | 72.9% | 90.0% | +17.1 | yes |
| `incident_postmortem` | 92.6% | 88.9% | -3.7 | no |
| `okr_execution` | 83.3% | 94.5% | +11.1 | yes |
| `operating_cadence` | 88.9% | 85.2% | -3.7 | no |
| `sops` | 88.9% | 85.2% | -3.7 | no |
| `vendor_ops` | 88.9% | 94.5% | +5.6 | yes |
- **FT trails base in:** `handoffs`, `incident_postmortem`, `operating_cadence`, `sops`
- Do **not** publish until a format patch lifts these categories.
## Rule
- Publish LoRA + merged + GGUF **only if** `BEATS_BASE` is true.
- Automated rubric ≠ human preference.