Update embeddinggemma-300m-memory-ft-v2 documentation

README.md

@@ -24,13 +24,13 @@ An English embedding model for finding the passages that answer a question in no
 
 | Stronger private retrieval | Less forgetting | Local deployment |
 |---|---|---|
-| nDCG@10 **0.4363 → 0.5574**; precision@10 **47.40% → 61.18%** | Recovers **more than half** of the first fine-tune's loss on FiQA and SciFact | **300M parameters**, ONNX, CPU / CUDA / Vulkan |
+| nDCG@10 **0.4418 → 0.5621**; precision@10 **49.71% → 64.16%** | Recovers **more than half** of the first fine-tune's loss on FiQA and SciFact | **300M parameters**, ONNX, CPU / CUDA / Vulkan |
 
-The private comparison covers **970 queries and 82,719 passages**, using a
-shared reference of **127,665 graded query–passage pairs** and reviewed results
-through rank 50. Daecore accepted some general-retrieval loss for stronger
-evidence retrieval on its document workflow. The tables below show both the gains
-and the remaining public-benchmark gaps.
+The private comparison uses **925 queries with known useful evidence**, drawn
+from 970 queries over **82,719 passages**, with reviewed results through rank
+50. Daecore accepted some general-retrieval loss for stronger evidence retrieval
+on its document workflow. The tables show both the gains and remaining
+public-benchmark gaps.
 
 Embed queries and passages with the same revision, and re-embed stored passages when switching to v2.
 
@@ -99,44 +99,46 @@ the 50 candidates admitted after Gemma and BM25 are fused and deduplicated.
 
 ### Gemma alone: dense retrieval on Daecore data
 
-Both models search the same **82,719 passages for all 970 queries**, using FP32
-exact dense retrieval, the same frozen query forms and matched 128/1,024-token
-input limits. BM25 and Ettin do not contribute to these results. Each cell
-reads upstream → **Daecore v2**.
+Both models search the same **82,719 passages**, using FP32 exact dense
+retrieval, the same frozen query forms and matched 128/1,024-token input
+limits. BM25 and Ettin do not contribute. Each cell reads upstream →
+**Daecore v2**.
 
-**Reference:** 127,665 graded query–passage pairs · shared top-50 reference
-(September 2026) · abstentions excluded. This pooled reference covers both
-models and is not an exhaustive labeling of the corpus.
+**Reference:** 925 queries with known grade-2/3 evidence · **122,231 graded
+query–passage pairs** for those queries, within the shared 127,665-pair top-50
+reference (September 2026). The same subset applies to both models, including
+queries where either model misses all useful passages.
 
 | Cutoff | Hit | Precision | Known-positive recall | nDCG |
 |---|---:|---:|---:|---:|
-| 1 | 53.40% → **64.74%** | 53.40% → **64.74%** | 1.44% → **1.79%** | 0.4592 → **0.5313** |
-| 3 | 72.99% → **79.38%** | 50.83% → **63.08%** | 3.94% → **5.03%** | 0.4414 → **0.5407** |
-| 5 | 79.48% → **84.74%** | 49.95% → **62.66%** | 6.17% → **8.06%** | 0.4346 → **0.5453** |
-| 10 | 84.85% → **88.14%** | 47.40% → **61.18%** | 11.18% → **15.07%** | 0.4363 → **0.5574** |
-| 20 | 88.76% → **90.52%** | 43.94% → **58.11%** | 19.61% → **27.56%** | 0.4526 → **0.5819** |
-| 50 | 93.20% → **94.43%** | 38.60% → **50.45%** | 41.12% → **57.92%** | 0.5107 → **0.6450** |
+| 1 | 56.00% → **67.89%** | 56.00% → **67.89%** | 1.44% → **1.79%** | 0.4621 → **0.5366** |
+| 3 | 76.54% → **83.24%** | 53.30% → **66.15%** | 3.94% → **5.03%** | 0.4466 → **0.5440** |
+| 5 | 83.35% → **88.86%** | 52.38% → **65.71%** | 6.17% → **8.06%** | 0.4398 → **0.5486** |
+| 10 | 88.97% → **92.43%** | 49.71% → **64.16%** | 11.18% → **15.07%** | 0.4418 → **0.5621** |
+| 20 | 93.08% → **94.92%** | 46.08% → **60.94%** | 19.61% → **27.56%** | 0.4580 → **0.5888** |
+| 50 | 97.73% → **99.03%** | 40.48% → **52.92%** | 41.12% → **57.92%** | 0.5145 → **0.6525** |
 
 ![Gemma v2 dense retrieval compared with upstream](figures/gemma-comparison.svg)
 
-Every top-50 position is graded or explicitly excluded as ungradable, with no
-missing-label ranges. All 970 queries remain eligible at every cutoff. At rank
-50, 164 upstream positions and 167 v2 positions are excluded without backfill.
-Precision pools retained positions; recall averages the 925 queries with judged
-useful evidence. The corpus is not exhaustively labeled.
+Every measured top-50 position is graded or explicitly excluded as ungradable,
+with no missing-label ranges. All 925 queries remain eligible at every dense
+cutoff. At rank 50, 164 upstream positions and 167 v2 positions are excluded
+without backfill. Precision pools retained positions. The corpus is not
+exhaustively labeled; the other 45 source queries have no *known* useful
+passage, which does not prove none exists.
 
-At 50, v2 finds useful evidence for **94.43%** of queries and retrieves **57.92%**
-of known useful passages, versus 93.20% and 41.12% upstream. The actual hybrid
-pool below has lower coverage on this panel, despite stronger early precision
-after reranking. Adding BM25 and capping the fused list at 50 can displace dense
-candidates; the two pools should not be treated as interchangeable.
+At 50, v2 finds useful evidence for **99.03%** of these queries and retrieves
+**57.92%** of their known useful passages, versus 97.73% and 41.12% upstream.
+The hybrid pool below has lower coverage on the same subset, despite stronger
+early precision after reranking. Adding BM25 and capping the fused list at 50
+can displace dense candidates.
 
 Grades 2–3 count as useful. nDCG uses gains 0, 1, 3 and 7 against the same
-reviewed reference for both models, averaging all 970 queries. Rankings from
-query forms are combined with reciprocal-rank fusion, and identical passage
-text is deduplicated. The [evaluation companion](evaluation/README.md)
-includes anonymized grades, model identities, label coverage and metric code.
-This is a reused development panel, not an untouched test of new workspaces.
+reviewed reference for both models. Rankings from query forms are combined
+with reciprocal-rank fusion, and identical passage text is deduplicated. The
+[evaluation companion](evaluation/README.md) includes all-query
+coverage, anonymized grades, identities and metric code. This is a reused
+development panel, not an untouched test of new workspaces.
 
 ### Gemma alone: public dense retrieval
 
@@ -158,16 +160,16 @@ The model was trained at 768, 512, 256 and 128 dimensions. To use a smaller
 width, keep the first dimensions and normalize the shortened vector again.
 Use the same width for queries and passages. Each quality column below is
 **v2 nDCG@10**, computed from exact rankings of cached FP32 vectors. The Daecore
-column covers the same 970-query dense panel with reviewed top-10 labels at
-each width, against the same **127,665-pair reference**. The 768-wide results
+column covers the same 925-query dense panel with reviewed top-10 labels at
+each width, against the same **122,231 graded pairs** for these queries. The 768-wide results
 reproduce the corresponding full-width scores.
 
 | Dimensions | Bytes per FP32 vector | Daecore | SciFact | FiQA | NFCorpus | SciDocs | ArguAna |
 |---|---:|---:|---:|---:|---:|---:|---:|
-| 768 (default) | 3,072 | 0.5574 | 0.7783 | 0.4468 | 0.3890 | 0.1852 | 0.6259 |
-| 512 | 2,048 | 0.5605 | 0.7841 | 0.4374 | 0.3876 | 0.1793 | 0.6137 |
-| 256 | 1,024 | 0.5292 | 0.7757 | 0.4091 | 0.3660 | 0.1708 | 0.5987 |
-| 128 | 512 | 0.5035 | 0.7488 | 0.3668 | 0.3311 | 0.1484 | 0.5533 |
+| 768 (default) | 3,072 | 0.5621 | 0.7783 | 0.4468 | 0.3890 | 0.1852 | 0.6259 |
+| 512 | 2,048 | 0.5648 | 0.7841 | 0.4374 | 0.3876 | 0.1793 | 0.6137 |
+| 256 | 1,024 | 0.5323 | 0.7757 | 0.4091 | 0.3660 | 0.1708 | 0.5987 |
+| 128 | 512 | 0.5069 | 0.7488 | 0.3668 | 0.3311 | 0.1484 | 0.5533 |
 
 At 512 dimensions, each raw FP32 vector uses one-third less storage, with
 slightly higher Daecore nDCG@10 in this measurement and lower scores on four
@@ -179,40 +181,41 @@ smaller widths have not been qualified through the full hybrid pipeline.
 ### Full pipeline: Gemma v2 + BM25 + Ettin
 
 This replay measures semantic and lexical search, reciprocal-rank fusion, then
-Ettin reranking over **970 development queries and 82,719 passages**. It includes
-queries with no known useful evidence. Both model cards share this CUDA FP16
-table; it is not either model's standalone score.
+Ettin reranking over **82,719 passages**. The table uses the same **925 queries
+with known useful evidence** as Gemma's dense comparison, selected from the
+970-query source panel. A query remains included when this pipeline misses its
+answer. Both cards share this CUDA FP16 table; it is not either model's
+standalone score.
 
-**Reference:** 970 queries · 82,719 passages · **127,665 graded query–passage
-pairs** in the shared top-50 reference (September 2026). Abstentions are excluded.
+**Reference:** 122,231 graded query–passage pairs for these 925 queries, within
+the shared 127,665-pair top-50 reference (September 2026). Abstentions are excluded.
 
 | Return depth | Hit | Precision | Known-positive recall | nDCG |
 |---|---:|---:|---:|---:|
-| 1 | 72.33% | 72.33% | 2.12% | 0.6458 |
-| 3 | 85.36% | 71.58% | 5.89% | 0.6374 |
-| 5 | 88.45% | 70.28% | 9.35% | 0.6326 |
-| 10 | 89.90% | 67.18% | 17.04% | 0.6315 |
-| 20 | 91.03% | 61.79% | 29.72% | 0.6417 |
-| 50 | 93.09% | 40.43% | 45.98% | 0.6051 |
-
-**The hybrid candidate ceiling is 93.09% Hit@50 and 45.98% known-positive
-recall@50.** These are coverage measures for the actual 50-candidate pool;
-reranking cannot change them. The row at 50 exposes that pool, while Daecore
-returns a selected prefix of 3–20. Fixed-depth scores do not evaluate the
-selector's choice.
-
-Grades 2–3 count as useful. At depth 1, both providers use the same 965 queries;
-five are omitted because a first result was ungradable. Depths 3–50 retain all
-970. Precision pools judged positions; recall averages the 925 queries with
-known useful evidence (920 at depth 1). Abstentions remove 52 positions at
-rank 20 and 93 at rank 50 per provider, without drawing in deeper passages.
-The corpus is not exhaustively labeled.
-
-Dense retrieval and this replay use the same expanded relevance reference.
-Earlier Hit and precision are unchanged; recall and nDCG were recomputed as
-more relevant passages became known. The [evaluation companion](evaluation/README.md)
-provides the records, exclusions and CUDA/Vulkan comparison; the
-[pipeline overview](https://huggingface.co/Daecore) explains retrieval and selection.
+| 1 | 75.87% | 75.87% | 2.12% | 0.6513 |
+| 3 | 89.51% | 75.07% | 5.89% | 0.6423 |
+| 5 | 92.76% | 73.71% | 9.35% | 0.6376 |
+| 10 | 94.27% | 70.46% | 17.04% | 0.6378 |
+| 20 | 95.46% | 64.81% | 29.72% | 0.6491 |
+| 50 | 97.62% | 42.40% | 45.98% | 0.6130 |
+
+**The hybrid candidate ceiling is 97.62% Hit@50 and 45.98% known-positive
+recall@50** on this shared answerable subset. These coverage measures describe
+the actual 50-candidate pool; reranking cannot change them. Daecore returns a
+selected prefix of 3–20, so fixed-depth scores do not evaluate the selector's
+choice.
+
+Grades 2–3 count as useful. At depth 1, both providers use the same **920
+queries**; five are omitted because a first result was ungradable. Depths
+3–50 use all 925. Precision pools judged positions. Abstentions remove 52
+positions at rank 20 and 93 at rank 50 per provider, without drawing in deeper
+passages. Known-positive recall counts judged passages, not every useful
+passage in the corpus.
+
+The [evaluation companion](evaluation/README.md) retains
+coverage across all 970 source queries and explains the shared reference,
+exclusions and CUDA/Vulkan comparison. The [pipeline overview](https://huggingface.co/Daecore)
+explains retrieval and selection.
 
 ## Training data and objective
 
@@ -273,7 +276,7 @@ Vulkan runs through the native ONNX Runtime WebGPU plugin, [`onnxruntime-ep-webg
 
 Checks used Windows x64 with an NVIDIA RTX 3060 Ti (8 GB); other GPUs and Linux are untested, even where Vulkan is available.
 
-The task-specific data is mostly generated and model-judged, with no human reference panel, so it describes this development distribution rather than other users' documents. The dense and full-pipeline panels share queries and a pooled relevance reference, but evaluate different retrieval paths. Some questions omit the project or task context needed for an unambiguous judgment. The 970-query pipeline panel uses natural-language questions and reports no separate terse-query or paraphrase robustness scores. Passage relevance does not measure distinct-fact coverage or downstream agent success.
+The task-specific data is mostly generated and model-judged, with no human reference panel, so it describes this development distribution rather than other users' documents. The dense and full-pipeline panels share queries and a pooled relevance reference, but evaluate different retrieval paths. Some questions omit the project or task context needed for an unambiguous judgment. The 970-query source panel uses natural-language questions and reports no separate terse-query or paraphrase robustness scores. Passage relevance does not measure distinct-fact coverage or downstream agent success.
 
 ## License
 

evaluation/README.md

@@ -26,7 +26,7 @@ python metrics.py --directory .
 python figures.py --output ../figures
 ```
 
-No packages or network access are needed. Each result matches its corresponding summary. This recomputes metrics from saved results; running private retrieval again would require the private corpus.
+No packages or network access are needed. Each result matches its corresponding summary. The `answerable` section of the dense and serving summaries supplies the card tables; `private.answerable` supplies the private width column. The outer `models` sections retain all-query results. This recomputes metrics from saved results; running private retrieval again would require the private corpus.
 
 ## Dense retrieval
 
@@ -37,6 +37,11 @@ cosine rankings take the top 50 for each frozen query form, then merge them
 with reciprocal-rank fusion (constant 60). Identical text is deduplicated;
 there is no parent-document filter, BM25 or reranker. The record includes
 model identities, source hashes, top-50 grades and reference grade counts.
+The card reports the **925 queries with at least one known grade-2/3 passage**
+in the shared reference. This same subset is used for both models, including
+queries where a model retrieves no useful evidence. It contains **122,231
+graded pairs** within the full 127,665-pair reference. The other 45 have no
+known useful passage; their true corpus-wide answerability is unresolved.
 This is a reused development panel, not a fresh holdout.
 
 The shared reference contains **127,665 graded query–passage pairs**. The top-50
@@ -49,22 +54,25 @@ from either primary judge remained excluded. The primary assistant read all
 stratified sample. Sixteen quotation repairs changed no grades. No person
 reviewed the labels.
 
-Every top-50 position is graded or explicitly excluded. All 970 queries remain
-eligible at 1, 3, 5, 10, 20 and 50. Upstream excludes 164 positions at rank 50;
-v2 excludes 167. Cut the original prefix first, drop abstentions second, and
-never backfill. A wholly abstained prefix would omit the query from both arms
-at that depth. Precision pools retained positions. Recall averages the 925
-queries with known useful passages; these are not exhaustive corpus labels.
-nDCG uses gains 0, 1, 3 and 7, reindexes retained positions and averages all
-970 queries; an ideal gain of zero contributes zero.
-
-**Reference expansion, not model change.** The existing top-20 grades and
-rankings are unchanged, so Hit and precision are unchanged. New relevant
-passages enlarge recall's denominator and can strengthen the ideal nDCG
-ranking. Dense nDCG@10 is now 0.4363 upstream and 0.5574 for v2, versus
-0.4506 and 0.5748 on the earlier 87,434-pair reference. Both models are recomputed together.
-Dense, smaller-width and current pipeline results now share this reference;
-the historical predecessor comparison below keeps its original reference.
+Every top-50 position is graded or explicitly excluded. All 925 answerable
+queries remain eligible at 1, 3, 5, 10, 20 and 50. Upstream excludes 164
+positions at rank 50; v2 excludes 167. Cut the original prefix first, drop
+abstentions second, and never backfill. A wholly abstained prefix would omit
+the query from both arms at that depth. Precision pools retained positions.
+Recall counts known useful passages, not exhaustive corpus labels. nDCG uses
+gains 0, 1, 3 and 7 and reindexes retained positions.
+
+**Reference and population changes.** Expanding the reference preserved the
+old top-20 grades, rankings, Hit and precision. Across all 970 queries, dense
+nDCG@10 changed from 0.4506 to 0.4363 upstream and from 0.5748 to 0.5574 for
+v2 because newly judged relevant passages can raise ideal DCG. The earlier
+reference contained 87,434 graded pairs.
+
+The subsequent decision to headline the same 925 known-answerable queries
+changes the averaging population, not the labels or rankings. On that subset,
+nDCG@10 is **0.4418 upstream and 0.5621 for v2**. The shared reference supports
+dense, smaller-width and current pipeline results. The historical predecessor
+comparison below keeps its original reference and population.
 
 **Public data.**
 The model card compares v2 with upstream EmbeddingGemma on SciFact, FiQA, NFCorpus, SciDocs and ArguAna. `promotion.json` includes per-query nDCG@10 for all five datasets, using full corpora, matched preprocessing and official relevance judgments. V2 remains below upstream on every panel. The previous fine-tune was measured only on FiQA (0.4009) and SciFact (0.7679); its other three results are absent, not estimated.
@@ -81,7 +89,7 @@ order for ties and official linear-gain nDCG. All 1,406 ArguAna queries remain,
 including five whose positives are absent from the corpus. These public
 scores are unchanged. Private scoring uses the same frozen query forms,
 dense-only fusion and expanded relevance reference as the 768-wide comparison.
-All 970 queries remain eligible at top 10; excluded positions are 22, 26, 16
+The same 925 answerable queries remain eligible at top 10; excluded positions are 22, 26, 16
 and 37 respectively. The 768-wide results reproduce both references exactly.
 Smaller widths have not been qualified through the full pipeline and do not
 shorten the encoder's forward pass.
@@ -89,21 +97,38 @@ shorten the encoder's forward pass.
 ## Effect in the retrieval pipeline
 
 The identical table on both cards comes from `serving.json`: Gemma v2 +
-BM25 + fusion + unchanged Ettin, on all 970 queries. CUDA FP16 and Vulkan FP32
-use the same 50 candidates. At 50, Hit is 93.09% and known-positive recall is
-45.98%; those candidate-coverage measures cannot change through reranking.
-Dense v2 reaches 94.43% and 57.92% on this panel. Fusion with a fixed 50-slot
-budget can displace dense candidates even while reranking improves early
-precision.
-
-At depths 3–50, Hit and nDCG average all 970 queries; recall averages the 925
-with known useful evidence. Depth 1 uses 965 shared judged queries, including
-920 with known positives; five are omitted because a first result was
-ungradable. Precision pools retained positions. Each provider excludes 52
+BM25 + fusion + unchanged Ettin. It uses the **same 925 known-answerable
+queries and 122,231 graded pairs** as the dense table. CUDA FP16 and Vulkan
+FP32 use the same 50 candidates for each query. At 50, Hit is **97.62%** and
+known-positive recall is **45.98%**; reranking cannot change candidate
+coverage. Dense v2 reaches 99.03% and 57.92% on this subset. Fusion with a
+fixed 50-slot budget can displace dense candidates even while reranking
+improves early precision.
+
+Depths 3–50 use all 925 queries. Depth 1 uses 920 shared judged queries; five
+are omitted because a first result was ungradable. Each provider excludes 52
 positions at rank 20 and 93 at rank 50 without backfill. Fixed cutoffs are
 separate from the selector's choice of a 3–20-item prefix. Vulkan matches
 CUDA at Hit@5, Hit@10, Hit@20 and Hit@50; Hit@3 differs by one query and
-nDCG@10 by 0.0003.
+nDCG@10 by 0.0004.
+
+### All-query coverage
+
+The records preserve all 970 source queries. This audit view includes the 45
+without known useful evidence and is separate from the card's answerable-only
+quality table. Hybrid top 1 uses 965 queries after shared abstention
+exclusions; dense top 1 retains all 970. The counts below use the full
+127,665-pair reference.
+
+| Retrieval path | Queries | Hit@3 | Hit@50 | nDCG@10 |
+|---|---:|---:|---:|---:|
+| Upstream Gemma, dense | 970 | 72.99% | 93.20% | 0.4363 |
+| Gemma v2, dense | 970 | 79.38% | 94.43% | 0.5574 |
+| Gemma v2 + BM25 + Ettin, CUDA | 970 | 85.36% | 93.09% | 0.6315 |
+
+A query with no known positive contributes zero Hit; grade-1 passages can
+still contribute to nDCG. These numbers should not be mixed with the card's
+925-query results.
 
 ### Historical predecessor comparison
 

evaluation/dimensions-summary.json

@@ -305,6 +305,259 @@
         }
       }
     },
+    "answerable": {
+      "queries": 925,
+      "models": {
+        "768": {
+          "1": {
+            "scored_queries": 919,
+            "excluded_queries": 6,
+            "hit": 0.676822633297062,
+            "precision": 0.676822633297062,
+            "macro_precision": 0.676822633297062,
+            "ndcg": 0.53541634281569,
+            "known_positive_recall": 0.01788016629109334,
+            "recall_queries": 919,
+            "useful": 622,
+            "retained": 919,
+            "excluded_positions": 0,
+            "mean_useful": 0.676822633297062,
+            "mean_retained": 1.0
+          },
+          "3": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.8324324324324325,
+            "precision": 0.6614996395097332,
+            "macro_precision": 0.6616216216216216,
+            "ndcg": 0.5439797460516042,
+            "known_positive_recall": 0.05034823759106709,
+            "recall_queries": 925,
+            "useful": 1835,
+            "retained": 2774,
+            "excluded_positions": 1,
+            "mean_useful": 1.9837837837837837,
+            "mean_retained": 2.998918918918919
+          },
+          "5": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.8886486486486487,
+            "precision": 0.6571366688325753,
+            "macro_precision": 0.6574774774774775,
+            "ndcg": 0.5486144310934794,
+            "known_positive_recall": 0.08055790386567772,
+            "recall_queries": 925,
+            "useful": 3034,
+            "retained": 4617,
+            "excluded_positions": 8,
+            "mean_useful": 3.28,
+            "mean_retained": 4.991351351351351
+          },
+          "10": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.9243243243243243,
+            "precision": 0.6416341569137408,
+            "macro_precision": 0.6421261261261262,
+            "ndcg": 0.5620625351358739,
+            "known_positive_recall": 0.15074270958466607,
+            "recall_queries": 925,
+            "useful": 5921,
+            "retained": 9228,
+            "excluded_positions": 22,
+            "mean_useful": 6.401081081081081,
+            "mean_retained": 9.976216216216216
+          }
+        },
+        "512": {
+          "1": {
+            "scored_queries": 919,
+            "excluded_queries": 6,
+            "hit": 0.6779107725788901,
+            "precision": 0.6779107725788901,
+            "macro_precision": 0.6779107725788901,
+            "ndcg": 0.5517902481993886,
+            "known_positive_recall": 0.01775688496570127,
+            "recall_queries": 919,
+            "useful": 623,
+            "retained": 919,
+            "excluded_positions": 0,
+            "mean_useful": 0.6779107725788901,
+            "mean_retained": 1.0
+          },
+          "3": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.8356756756756757,
+            "precision": 0.6661854926019487,
+            "macro_precision": 0.6664864864864865,
+            "ndcg": 0.5454577108638902,
+            "known_positive_recall": 0.05057307077004017,
+            "recall_queries": 925,
+            "useful": 1846,
+            "retained": 2771,
+            "excluded_positions": 4,
+            "mean_useful": 1.9956756756756757,
+            "mean_retained": 2.9956756756756757
+          },
+          "5": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.8832432432432432,
+            "precision": 0.6616117850953206,
+            "macro_precision": 0.6617657657657657,
+            "ndcg": 0.5530614761090054,
+            "known_positive_recall": 0.08121602068063129,
+            "recall_queries": 925,
+            "useful": 3054,
+            "retained": 4616,
+            "excluded_positions": 9,
+            "mean_useful": 3.3016216216216216,
+            "mean_retained": 4.99027027027027
+          },
+          "10": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.9254054054054054,
+            "precision": 0.643213356461405,
+            "macro_precision": 0.6438635778635778,
+            "ndcg": 0.5647836166130059,
+            "known_positive_recall": 0.15004173661513098,
+            "recall_queries": 925,
+            "useful": 5933,
+            "retained": 9224,
+            "excluded_positions": 26,
+            "mean_useful": 6.414054054054054,
+            "mean_retained": 9.971891891891891
+          }
+        },
+        "256": {
+          "1": {
+            "scored_queries": 919,
+            "excluded_queries": 6,
+            "hit": 0.6332970620239391,
+            "precision": 0.6332970620239391,
+            "macro_precision": 0.6332970620239391,
+            "ndcg": 0.5139126379605161,
+            "known_positive_recall": 0.015976956236435802,
+            "recall_queries": 919,
+            "useful": 582,
+            "retained": 919,
+            "excluded_positions": 0,
+            "mean_useful": 0.6332970620239391,
+            "mean_retained": 1.0
+          },
+          "3": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.7978378378378378,
+            "precision": 0.6245487364620939,
+            "macro_precision": 0.6252252252252252,
+            "ndcg": 0.5122675608706706,
+            "known_positive_recall": 0.046233615834183193,
+            "recall_queries": 925,
+            "useful": 1730,
+            "retained": 2770,
+            "excluded_positions": 5,
+            "mean_useful": 1.8702702702702703,
+            "mean_retained": 2.9945945945945946
+          },
+          "5": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.8616216216216216,
+            "precision": 0.6214285714285714,
+            "macro_precision": 0.6217837837837837,
+            "ndcg": 0.519156867098927,
+            "known_positive_recall": 0.0744111972114428,
+            "recall_queries": 925,
+            "useful": 2871,
+            "retained": 4620,
+            "excluded_positions": 5,
+            "mean_useful": 3.1037837837837836,
+            "mean_retained": 4.994594594594594
+          },
+          "10": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.9156756756756756,
+            "precision": 0.6102447476716483,
+            "macro_precision": 0.6107237237237237,
+            "ndcg": 0.5322729475032464,
+            "known_positive_recall": 0.1419199356602385,
+            "recall_queries": 925,
+            "useful": 5635,
+            "retained": 9234,
+            "excluded_positions": 16,
+            "mean_useful": 6.091891891891892,
+            "mean_retained": 9.982702702702703
+          }
+        },
+        "128": {
+          "1": {
+            "scored_queries": 919,
+            "excluded_queries": 6,
+            "hit": 0.6006528835690969,
+            "precision": 0.6006528835690969,
+            "macro_precision": 0.6006528835690969,
+            "ndcg": 0.4720969998445515,
+            "known_positive_recall": 0.015019269226427342,
+            "recall_queries": 919,
+            "useful": 552,
+            "retained": 919,
+            "excluded_positions": 0,
+            "mean_useful": 0.6006528835690969,
+            "mean_retained": 1.0
+          },
+          "3": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.7762162162162162,
+            "precision": 0.6061482820976491,
+            "macro_precision": 0.6061261261261262,
+            "ndcg": 0.48302039261543295,
+            "known_positive_recall": 0.044531751738185646,
+            "recall_queries": 925,
+            "useful": 1676,
+            "retained": 2765,
+            "excluded_positions": 10,
+            "mean_useful": 1.8118918918918918,
+            "mean_retained": 2.9891891891891893
+          },
+          "5": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.8432432432432433,
+            "precision": 0.6026475694444444,
+            "macro_precision": 0.6025585585585586,
+            "ndcg": 0.4894363954353818,
+            "known_positive_recall": 0.07159737689223461,
+            "recall_queries": 925,
+            "useful": 2777,
+            "retained": 4608,
+            "excluded_positions": 17,
+            "mean_useful": 3.002162162162162,
+            "mean_retained": 4.981621621621621
+          },
+          "10": {
+            "scored_queries": 925,
+            "excluded_queries": 0,
+            "hit": 0.907027027027027,
+            "precision": 0.5936177140996418,
+            "macro_precision": 0.5940930930930931,
+            "ndcg": 0.506862270631805,
+            "known_positive_recall": 0.13623796718702827,
+            "recall_queries": 925,
+            "useful": 5469,
+            "retained": 9213,
+            "excluded_positions": 37,
+            "mean_useful": 5.912432432432432,
+            "mean_retained": 9.96
+          }
+        }
+      }
+    },
     "corpus_passages": 82719
   }
 }

evaluation/figures.py

@@ -63,6 +63,8 @@ def reranker(summary: dict) -> str:
 
 
 def retriever(summary: dict) -> str:
+    corpus_passages = summary['corpus_passages']
+    summary = summary['answerable']
     cutoffs = sorted(map(int, summary['models']['upstream']))
     x = list(range(len(cutoffs)))
     panels = []
@@ -77,7 +79,7 @@ def retriever(summary: dict) -> str:
             xlim=(0, len(cutoffs) - 1), ylim=(0, 1.02), xticks=list(enumerate(map(str, cutoffs)))))
     return svg.render_grid(panels, columns=3,
                            title='Gemma v2: matched dense retrieval on Daecore data',
-                           subtitle=f"{summary['queries']:,} queries · {summary['corpus_passages']:,} passages · reviewed abstentions excluded",
+                           subtitle=f"{summary['queries']:,} known-answerable queries · {corpus_passages:,} passages · abstentions excluded",
                            legend=legend, panel_width=300, panel_height=270)
 
 

evaluation/metrics.py

@@ -117,10 +117,11 @@ def summarize_reranker(data: dict) -> dict:
 
 
 def summarize_retriever(data: dict) -> dict:
-    """Fully reviewed dense rankings, including queries without known positives."""
+    """Dense rankings, with both full-panel and known-answerable summaries."""
     if type(data['corpus_passages']) is not int or data['corpus_passages'] <= 0:
         raise ValueError('Dense comparison requires a positive corpus size')
     return {**_summarize_ranked_grades(data, include_selected=False),
+            'answerable': _summarize_ranked_grades(data, include_selected=False, answerable_only=True),
             'corpus_passages': data['corpus_passages']}
 
 
@@ -152,7 +153,7 @@ def summarize_dimensions(data: dict) -> dict:
     return result
 
 
-def _summarize_ranked_grades(data: dict, *, include_selected: bool) -> dict:
+def _summarize_ranked_grades(data: dict, *, include_selected: bool, answerable_only: bool = False) -> dict:
     """Reviewed exclusions inside each original dense or hybrid prefix.
 
     A null is permitted only for an explicitly excluded judging abstention.
@@ -185,7 +186,13 @@ def _summarize_ranked_grades(data: dict, *, include_selected: bool) -> dict:
             if any((grade is not None if drop else type(grade) is not int or grade not in range(4))
                    for grade, drop in zip(ranked, excluded, strict=True)):
                 raise ValueError('Only declared abstentions may lack grades')
+    # The shared relevance reference defines answerability, never one arm's
+    # retrieved prefix. Validate every source row before applying this filter.
+    if answerable_only:
+        rows = [row for row in rows if row['reference_grade_counts']['2'] + row['reference_grade_counts']['3'] > 0]
     output = {'queries': len(rows), 'models': {}}
+    if not rows:
+        return output  # No conditional score exists; do not substitute zeros.
     requested_cutoffs = tuple(k for k in CUTOFFS if k <= depth_limit)
     if include_selected:
         requested_cutoffs += ('selected',)
@@ -254,10 +261,16 @@ def summarize_promotion(data: dict) -> dict:
     return output
 
 
+def summarize_serving(data: dict) -> dict:
+    """Current hybrid quality uses the same reference-defined eligible queries."""
+    return {**summarize_promotion(data),
+            'answerable': _summarize_ranked_grades(data, include_selected=True, answerable_only=True)}
+
+
 SUMMARIZERS = {
     'classifier': summarize_classifier, 'reranker': summarize_reranker,
     'retriever': summarize_retriever, 'dimensions': summarize_dimensions,
-    'promotion': summarize_promotion, 'serving': summarize_promotion,
+    'promotion': summarize_promotion, 'serving': summarize_serving,
 }
 
 

evaluation/retriever-summary.json

@@ -186,5 +186,194 @@
       }
     }
   },
+  "answerable": {
+    "queries": 925,
+    "models": {
+      "upstream": {
+        "1": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.56,
+          "precision": 0.56,
+          "macro_precision": 0.56,
+          "ndcg": 0.46208494208494205,
+          "known_positive_recall": 0.014444554022554491,
+          "recall_queries": 925,
+          "useful": 518,
+          "retained": 925,
+          "excluded_positions": 0,
+          "mean_useful": 0.56,
+          "mean_retained": 1.0
+        },
+        "3": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.7654054054054054,
+          "precision": 0.5329967544175983,
+          "macro_precision": 0.532972972972973,
+          "ndcg": 0.44657619780045693,
+          "known_positive_recall": 0.03938697535513605,
+          "recall_queries": 925,
+          "useful": 1478,
+          "retained": 2773,
+          "excluded_positions": 2,
+          "mean_useful": 1.597837837837838,
+          "mean_retained": 2.997837837837838
+        },
+        "5": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.8335135135135135,
+          "precision": 0.5238198354265916,
+          "macro_precision": 0.523927927927928,
+          "ndcg": 0.4398171919899157,
+          "known_positive_recall": 0.06172226483192968,
+          "recall_queries": 925,
+          "useful": 2419,
+          "retained": 4618,
+          "excluded_positions": 7,
+          "mean_useful": 2.615135135135135,
+          "mean_retained": 4.992432432432432
+        },
+        "10": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.8897297297297297,
+          "precision": 0.49707348796878387,
+          "macro_precision": 0.49713856713856713,
+          "ndcg": 0.4417676357044915,
+          "known_positive_recall": 0.11180689389376004,
+          "recall_queries": 925,
+          "useful": 4586,
+          "retained": 9226,
+          "excluded_positions": 24,
+          "mean_useful": 4.957837837837838,
+          "mean_retained": 9.974054054054054
+        },
+        "20": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9308108108108109,
+          "precision": 0.46080026024723486,
+          "macro_precision": 0.4614682460502894,
+          "ndcg": 0.4579770106113412,
+          "known_positive_recall": 0.19611550928151497,
+          "recall_queries": 925,
+          "useful": 8499,
+          "retained": 18444,
+          "excluded_positions": 56,
+          "mean_useful": 9.188108108108109,
+          "mean_retained": 19.93945945945946
+        },
+        "50": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9772972972972973,
+          "precision": 0.4048301002473636,
+          "macro_precision": 0.4055311496261139,
+          "ndcg": 0.5145143143872781,
+          "known_positive_recall": 0.41118693689165037,
+          "recall_queries": 925,
+          "useful": 18657,
+          "retained": 46086,
+          "excluded_positions": 164,
+          "mean_useful": 20.16972972972973,
+          "mean_retained": 49.8227027027027
+        }
+      },
+      "finetuned": {
+        "1": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.6789189189189189,
+          "precision": 0.6789189189189189,
+          "macro_precision": 0.6789189189189189,
+          "ndcg": 0.5365765765765765,
+          "known_positive_recall": 0.017869431761799028,
+          "recall_queries": 925,
+          "useful": 628,
+          "retained": 925,
+          "excluded_positions": 0,
+          "mean_useful": 0.6789189189189189,
+          "mean_retained": 1.0
+        },
+        "3": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.8324324324324325,
+          "precision": 0.6614996395097332,
+          "macro_precision": 0.6616216216216216,
+          "ndcg": 0.5439797460516042,
+          "known_positive_recall": 0.05034823759106709,
+          "recall_queries": 925,
+          "useful": 1835,
+          "retained": 2774,
+          "excluded_positions": 1,
+          "mean_useful": 1.9837837837837837,
+          "mean_retained": 2.998918918918919
+        },
+        "5": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.8886486486486487,
+          "precision": 0.6571366688325753,
+          "macro_precision": 0.6574774774774775,
+          "ndcg": 0.5486144310934794,
+          "known_positive_recall": 0.08055790386567772,
+          "recall_queries": 925,
+          "useful": 3034,
+          "retained": 4617,
+          "excluded_positions": 8,
+          "mean_useful": 3.28,
+          "mean_retained": 4.991351351351351
+        },
+        "10": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9243243243243243,
+          "precision": 0.6416341569137408,
+          "macro_precision": 0.6421261261261262,
+          "ndcg": 0.5620625351358739,
+          "known_positive_recall": 0.15074270958466607,
+          "recall_queries": 925,
+          "useful": 5921,
+          "retained": 9228,
+          "excluded_positions": 22,
+          "mean_useful": 6.401081081081081,
+          "mean_retained": 9.976216216216216
+        },
+        "20": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9491891891891892,
+          "precision": 0.6094215861657722,
+          "macro_precision": 0.6098764600343548,
+          "ndcg": 0.5888460119487677,
+          "known_positive_recall": 0.27556154002694083,
+          "recall_queries": 925,
+          "useful": 11242,
+          "retained": 18447,
+          "excluded_positions": 53,
+          "mean_useful": 12.153513513513513,
+          "mean_retained": 19.942702702702704
+        },
+        "50": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9902702702702703,
+          "precision": 0.5291539179306902,
+          "macro_precision": 0.5297403182882328,
+          "ndcg": 0.6524763537042031,
+          "known_positive_recall": 0.5791516627126307,
+          "recall_queries": 925,
+          "useful": 24385,
+          "retained": 46083,
+          "excluded_positions": 167,
+          "mean_useful": 26.36216216216216,
+          "mean_retained": 49.81945945945946
+        }
+      }
+    }
+  },
   "corpus_passages": 82719
 }

evaluation/serving-summary.json

@@ -216,5 +216,224 @@
       }
     }
   },
-  "public": {}
+  "public": {},
+  "answerable": {
+    "queries": 925,
+    "models": {
+      "cuda": {
+        "1": {
+          "scored_queries": 920,
+          "excluded_queries": 5,
+          "hit": 0.758695652173913,
+          "precision": 0.758695652173913,
+          "macro_precision": 0.758695652173913,
+          "ndcg": 0.6513457556935818,
+          "known_positive_recall": 0.021209009540392988,
+          "recall_queries": 920,
+          "useful": 698,
+          "retained": 920,
+          "excluded_positions": 0,
+          "mean_useful": 0.758695652173913,
+          "mean_retained": 1.0
+        },
+        "3": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.8951351351351351,
+          "precision": 0.7507235890014472,
+          "macro_precision": 0.7515315315315315,
+          "ndcg": 0.6422807686434969,
+          "known_positive_recall": 0.0589083117861917,
+          "recall_queries": 925,
+          "useful": 2075,
+          "retained": 2764,
+          "excluded_positions": 11,
+          "mean_useful": 2.2432432432432434,
+          "mean_retained": 2.9881081081081082
+        },
+        "5": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9275675675675675,
+          "precision": 0.7371044646727352,
+          "macro_precision": 0.7373873873873874,
+          "ndcg": 0.6375956845322198,
+          "known_positive_recall": 0.09351524011264825,
+          "recall_queries": 925,
+          "useful": 3401,
+          "retained": 4614,
+          "excluded_positions": 11,
+          "mean_useful": 3.676756756756757,
+          "mean_retained": 4.988108108108108
+        },
+        "10": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9427027027027027,
+          "precision": 0.7045750216825672,
+          "macro_precision": 0.704967824967825,
+          "ndcg": 0.6377512015628823,
+          "known_positive_recall": 0.17039667688880922,
+          "recall_queries": 925,
+          "useful": 6499,
+          "retained": 9224,
+          "excluded_positions": 26,
+          "mean_useful": 7.025945945945946,
+          "mean_retained": 9.971891891891891
+        },
+        "20": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9545945945945946,
+          "precision": 0.6480919340849957,
+          "macro_precision": 0.648584612953034,
+          "ndcg": 0.6491335337062211,
+          "known_positive_recall": 0.29717756138114526,
+          "recall_queries": 925,
+          "useful": 11956,
+          "retained": 18448,
+          "excluded_positions": 52,
+          "mean_useful": 12.925405405405405,
+          "mean_retained": 19.943783783783783
+        },
+        "50": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9762162162162162,
+          "precision": 0.424031024546656,
+          "macro_precision": 0.42417201849319736,
+          "ndcg": 0.6129768665312807,
+          "known_positive_recall": 0.4598077844640693,
+          "recall_queries": 925,
+          "useful": 19572,
+          "retained": 46157,
+          "excluded_positions": 93,
+          "mean_useful": 21.158918918918918,
+          "mean_retained": 49.89945945945946
+        },
+        "selected": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9037837837837838,
+          "precision": 0.8188024463092021,
+          "macro_precision": 0.7382827833849506,
+          "ndcg": 0.6407625959742856,
+          "known_positive_recall": 0.128026594631235,
+          "recall_queries": 925,
+          "useful": 5757,
+          "retained": 7031,
+          "excluded_positions": 17,
+          "mean_useful": 6.223783783783784,
+          "mean_retained": 7.601081081081081
+        }
+      },
+      "vulkan": {
+        "1": {
+          "scored_queries": 920,
+          "excluded_queries": 5,
+          "hit": 0.7597826086956522,
+          "precision": 0.7597826086956522,
+          "macro_precision": 0.7597826086956522,
+          "ndcg": 0.6516563146997929,
+          "known_positive_recall": 0.021228078953055077,
+          "recall_queries": 920,
+          "useful": 699,
+          "retained": 920,
+          "excluded_positions": 0,
+          "mean_useful": 0.7597826086956522,
+          "mean_retained": 1.0
+        },
+        "3": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.894054054054054,
+          "precision": 0.7507235890014472,
+          "macro_precision": 0.7515315315315315,
+          "ndcg": 0.6421528020036147,
+          "known_positive_recall": 0.05893952330659241,
+          "recall_queries": 925,
+          "useful": 2075,
+          "retained": 2764,
+          "excluded_positions": 11,
+          "mean_useful": 2.2432432432432434,
+          "mean_retained": 2.9881081081081082
+        },
+        "5": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9275675675675675,
+          "precision": 0.7366139171905485,
+          "macro_precision": 0.736954954954955,
+          "ndcg": 0.6371821653908674,
+          "known_positive_recall": 0.09344587015679974,
+          "recall_queries": 925,
+          "useful": 3398,
+          "retained": 4613,
+          "excluded_positions": 12,
+          "mean_useful": 3.6735135135135133,
+          "mean_retained": 4.987027027027027
+        },
+        "10": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9427027027027027,
+          "precision": 0.7051490514905149,
+          "macro_precision": 0.7054877734877735,
+          "ndcg": 0.6381124055299272,
+          "known_positive_recall": 0.17053838380216288,
+          "recall_queries": 925,
+          "useful": 6505,
+          "retained": 9225,
+          "excluded_positions": 25,
+          "mean_useful": 7.032432432432432,
+          "mean_retained": 9.972972972972974
+        },
+        "20": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9545945945945946,
+          "precision": 0.6479835212489159,
+          "macro_precision": 0.6484765048449259,
+          "ndcg": 0.6490652861835569,
+          "known_positive_recall": 0.2971249330393808,
+          "recall_queries": 925,
+          "useful": 11954,
+          "retained": 18448,
+          "excluded_positions": 52,
+          "mean_useful": 12.923243243243244,
+          "mean_retained": 19.943783783783783
+        },
+        "50": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9762162162162162,
+          "precision": 0.424031024546656,
+          "macro_precision": 0.42417201849319736,
+          "ndcg": 0.6129402668287812,
+          "known_positive_recall": 0.4598077844640693,
+          "recall_queries": 925,
+          "useful": 19572,
+          "retained": 46157,
+          "excluded_positions": 93,
+          "mean_useful": 21.158918918918918,
+          "mean_retained": 49.89945945945946
+        },
+        "selected": {
+          "scored_queries": 925,
+          "excluded_queries": 0,
+          "hit": 0.9027027027027027,
+          "precision": 0.8181689141234918,
+          "macro_precision": 0.7380948996103794,
+          "ndcg": 0.640658479488419,
+          "known_positive_recall": 0.12825382711378852,
+          "recall_queries": 925,
+          "useful": 5764,
+          "retained": 7045,
+          "excluded_positions": 17,
+          "mean_useful": 6.231351351351352,
+          "mean_retained": 7.616216216216216
+        }
+      }
+    }
+  }
 }

publication-manifest.json

@@ -12,7 +12,7 @@
   "export_receipt_sha256": "e24e1692f7e03c9af8040bb3f6e310a061d305f485861007f1f37d36808b20d8",
   "serving_sha256": "8c088b1635875d5406e148782b303cfc5a961138600783fb4842b413b98a98c3",
   "source_model_tree_sha256": "30a7b9f1c42b1fcee29a6729742685d5b744e8fa87d13cefab2fd2d49ea9150e",
-  "staged_at": "2026-09-30T01:53:48+00:00",
+  "staged_at": "2026-09-30T02:12:24+00:00",
   "files": {
     "added_tokens.json": {
       "sha256": "50b2f405ba56a26d4913fd772089992252d7f942123cc0a034d96424221ba946",
@@ -80,8 +80,8 @@
       "binding": "packaging record"
     },
     "evaluation/README.md": {
-      "sha256": "390b5e5d525ae6c0b2c3370050af1cc14b4574127bb258db911fb0afed670494",
-      "size": 9703,
+      "sha256": "bf96501863235b426b679d84e0bba96bc5b5d7fdba76571966af92993ce4150e",
+      "size": 11100,
       "binding": "packaging record"
     },
     "evaluation/serving-qualification.json": {
@@ -90,8 +90,8 @@
       "binding": "packaging record"
     },
     "evaluation/metrics.py": {
-      "sha256": "bbccf29f34db24aefc5630adcaf9630e5f214010902dbe1df0032618aa329a45",
-      "size": 18329,
+      "sha256": "22536c4437be0b8c678df0dee4c45e31c5413e91f1fecf104750486f3e372d46",
+      "size": 19120,
       "binding": "packaging record"
     },
     "evaluation/chunking.md": {
@@ -100,13 +100,13 @@
       "binding": "packaging record"
     },
     "evaluation/retriever-summary.json": {
-      "sha256": "b320430acd9938d08e31eb00693dd9f775e9890b17427f86577c9bbd91dd1bba",
-      "size": 6071,
+      "sha256": "1cf1586a83ee90246588cd31b93b7aec0115c620c39ed646d4b3936fed1f7b4f",
+      "size": 12432,
       "binding": "packaging record"
     },
     "evaluation/dimensions-summary.json": {
-      "sha256": "3b160deb874b98e835b92b15070afe36e548b1ce692a7ef817af77026e5064f8",
-      "size": 9753,
+      "sha256": "4fd5d6bd5659612618e9b4ca3c5245e9b497dfcb0a8d0f3b3b70d65e55b358b1",
+      "size": 18756,
       "binding": "packaging record"
     },
     "evaluation/promotion-summary.json": {
@@ -115,8 +115,8 @@
       "binding": "packaging record"
     },
     "evaluation/serving-summary.json": {
-      "sha256": "cd87757b53482e2ec9450a3c64f31a32afad2710900fbcbd7885828929aa9535",
-      "size": 7035,
+      "sha256": "029b323fee810cae91c0a37c4d234e560a727b8219d582f1e7bd5fcb74b317de",
+      "size": 14521,
       "binding": "packaging record"
     },
     "evaluation/retriever.json": {
@@ -140,8 +140,8 @@
       "binding": "packaging record"
     },
     "evaluation/figures.py": {
-      "sha256": "6bc89020363df7a0cfcdebaa4e8be5b37385ab97b2b7bbf3f60cb9b46b2ac63e",
-      "size": 5408,
+      "sha256": "f13945b350e007deb18a2eafbe1c5708a71b9df62ee12b6833d07f3e2a06094e",
+      "size": 5490,
       "binding": "packaging record"
     },
     "evaluation/svg_figures.py": {
@@ -150,15 +150,15 @@
       "binding": "packaging record"
     },
     "figures/gemma-comparison.svg": {
-      "sha256": "4393db8afc6de32877a66b863708e396a1503d753932680dc7e4e590c4ccd513",
-      "size": 14507,
+      "sha256": "700dd8262d27ce78ba7433efb054da3d5ea950c5bfaf4ae83f909eae283d4022",
+      "size": 14511,
       "binding": "packaging record"
     },
     "README.md": {
-      "sha256": "b5fe1ab8b6d3821cf5ee42e9ffbb2d88d8509233d27834382ce63a385f5d60a4",
-      "size": 16388,
+      "sha256": "7c9add0392d4d28e8e048a2308ac4cf88b5ac0d9ee2e5e49160625d3731975cb",
+      "size": 16352,
       "binding": "model card with upload-relative links",
-      "source_sha256": "58c1418eb860b7e8d2db5b84085780bb665ffbd92d06224a59cff700cebe0f02"
+      "source_sha256": "38f074c37bde704f74971f782fa291850fdd325a4030671a344ef0ba4152a93d"
     }
   }
 }