d4rkninja commited on
Commit
f4ae83a
·
verified ·
1 Parent(s): a627ecc

docs: evaluation/COMPARE_OPS.md

Browse files
Files changed (1) hide show
  1. evaluation/COMPARE_OPS.md +32 -0
evaluation/COMPARE_OPS.md ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # DarkLab Base vs FT Compare
2
+
3
+ **Generated:** 2026-09-18T15:32:32.450342+00:00
4
+ **Base model:** `unsloth/LFM2.5-1.2B-Instruct`
5
+ **Owner / creator:** DarkNinja Solutions / d4rkninja
6
+
7
+ ## Overall
8
+
9
+ | Domain | Base % | FT % | Delta | BEATS_BASE |
10
+ |---|---:|---:|---:|:---:|
11
+ | `ops` | 89.5% | 90.7% | +1.2 | YES |
12
+
13
+ ## ops — per-category
14
+
15
+ | Category | Base % | FT % | Delta | FT wins |
16
+ |---|---:|---:|---:|:---:|
17
+ | `capacity_planning` | 94.5% | 100.0% | +5.5 | yes |
18
+ | `handoffs` | 100.0% | 92.6% | -7.4 | no |
19
+ | `identity_guardrails` | 72.9% | 90.0% | +17.1 | yes |
20
+ | `incident_postmortem` | 92.6% | 88.9% | -3.7 | no |
21
+ | `okr_execution` | 83.3% | 94.5% | +11.1 | yes |
22
+ | `operating_cadence` | 88.9% | 85.2% | -3.7 | no |
23
+ | `sops` | 88.9% | 85.2% | -3.7 | no |
24
+ | `vendor_ops` | 88.9% | 94.5% | +5.6 | yes |
25
+
26
+ - **FT trails base in:** `handoffs`, `incident_postmortem`, `operating_cadence`, `sops`
27
+ - Do **not** publish until a format patch lifts these categories.
28
+
29
+ ## Rule
30
+
31
+ - Publish LoRA + merged + GGUF **only if** `BEATS_BASE` is true.
32
+ - Automated rubric ≠ human preference.