d4rkninja commited on
Commit
ce66046
·
verified ·
1 Parent(s): 9d96cd6

Upload evaluation/COMPARE_BASE.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. evaluation/COMPARE_BASE.md +45 -12
evaluation/COMPARE_BASE.md CHANGED
@@ -1,12 +1,45 @@
1
- # DarkLab Base vs Fine-tune Comparison
2
-
3
- Official DarkLab 20-task domain rubric comparison against `unsloth/LFM2.5-1.2B-Instruct`.
4
-
5
- | Domain | Base rubric | Fine-tune rubric | Delta vs base |
6
- |---|---:|---:|---:|
7
- | **Product — format-v1** | 92.6% | **99.4% (99.38%)** | **+6.8 pts** |
8
- | Hiring | 83.8% | **92.9%** | **+9.2 pts** |
9
-
10
- Product format-v1 quality: **7.98** vs base **7.35**. Hiring row retained from the existing comparison: **92.9% vs 83.8%**.
11
-
12
- Automated rubric results are directional; they do not replace human judgment.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Base vs Fine-tune comparison
2
+
3
+ **Generated:** 2026-09-17T22:14:33+05:30
4
+ **Base model:** `unsloth/LFM2.5-1.2B-Instruct`
5
+ **Same tasks + same scorer as domain PEFT suite**
6
+
7
+ ## Overall
8
+
9
+ | Domain | Base LFM2.5 | Fine-tune | Delta |
10
+ |---|---:|---:|---:|
11
+ | tanpo-product | 92.6% | **88.4%** | -4.2 (trails) |
12
+ | tanpo-hiring | 83.8% | **92.9%** | +9.2 (beats) |
13
+
14
+ | Domain | Base quality | FT quality |
15
+ |---|---:|---:|
16
+ | tanpo-product | 7.35 | 6.84 |
17
+ | tanpo-hiring | 6.40 | 7.18 |
18
+
19
+ ## tanpo-product per-category
20
+
21
+ | Category | Base % | FT % | Delta |
22
+ |---|---:|---:|---:|
23
+ | `competitive_positioning` | 93.8 | 87.5 | -6.2 |
24
+ | `crisis_tradeoffs` | 100.0 | 91.7 | -8.3 |
25
+ | `customer_discovery` | 87.5 | 83.3 | -4.2 |
26
+ | `gtm_product_loops` | 93.8 | 81.2 | -12.5 |
27
+ | `metrics_okrs` | 100.0 | 88.9 | -11.1 |
28
+ | `org_operating_cadence` | 93.8 | 88.2 | -5.5 |
29
+ | `pricing_monetization` | 88.9 | 88.9 | +0.0 |
30
+ | `product_improvement` | 83.3 | 87.5 | +4.2 |
31
+ | `roadmap_sequencing` | 93.8 | 100.0 | +6.2 |
32
+
33
+ ## tanpo-hiring per-category
34
+
35
+ | Category | Base % | FT % | Delta |
36
+ |---|---:|---:|---:|
37
+ | `debrief_calibration` | 83.3 | 87.5 | +4.2 |
38
+ | `hiring_scorecards` | 81.2 | 100.0 | +18.8 |
39
+ | `interview_loop_design` | 85.2 | 100.0 | +14.8 |
40
+ | `job_descriptions` | 88.2 | 100.0 | +11.8 |
41
+ | `offer_framing` | 81.2 | 87.5 | +6.2 |
42
+ | `outbound_recruiting` | 93.8 | 100.0 | +6.2 |
43
+ | `rejection_pipeline_ops` | 62.5 | 87.5 | +25.0 |
44
+ | `screening_scripts` | 81.2 | 100.0 | +18.8 |
45
+ | `structured_interview_kits` | 85.2 | 77.8 | -7.4 |