Jainamshahhh commited on
Commit
f459d4d
·
verified ·
1 Parent(s): 9a7dba5

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +19 -0
README.md CHANGED
@@ -58,6 +58,25 @@ Base and tuned generated in one process under identical greedy decoding, scored
58
  | enumerated calendar | 200 | 23.5% | **90.5%** |
59
  | range slice, 2031 to 2035 | 150 | 14.7% | **76.7%** |
60
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
61
  ## The gate that nearly killed this entry, and the rule written before it ran
62
 
63
  The pre-registered Day-0 gate measured a **blended** F1 floor of 34%, which sat between the pass and
 
58
  | enumerated calendar | 200 | 23.5% | **90.5%** |
59
  | range slice, 2031 to 2035 | 150 | 14.7% | **76.7%** |
60
 
61
+ ## Significance, stated rather than assumed
62
+
63
+ Paired design, so significance is an **exact McNemar test** on the rows where exactly one
64
+ model was right, with the null that the adapter is no better than the base.
65
+
66
+ | slice | only tuned right | only base right | exact two-sided p |
67
+ |---|---|---|---|
68
+ | headline held-out | 405 | 5 | 7.2e-113 |
69
+ | hard shard | 241 | 5 | 1.3e-64 |
70
+ | enumerated calendar | 135 | 1 | 3.1e-39 |
71
+ | range slice, 2031 to 2035 | 95 | 2 | 6.0e-26 |
72
+
73
+ Note the middle column, which is **not** zero here. Alone in this portfolio, this base wins
74
+ rows outright: 13 of them across the four slices. That is the same fact the top of this card
75
+ opens with, arriving independently from the test rather than from the prose. A base that
76
+ scores 69 to 86% on plain day-walking should win some rows, and it does.
77
+
78
+ Recompute with the released `significance.py` from the released generation files.
79
+
80
  ## The gate that nearly killed this entry, and the rule written before it ran
81
 
82
  The pre-registered Day-0 gate measured a **blended** F1 floor of 34%, which sat between the pass and