Upload evidence_faithful_reasoning_release_brief.md with huggingface_hub

evidence_faithful_reasoning_release_brief.md

@@ -0,0 +1,118 @@
+# Evidence-Faithful Reasoning
+
+## Release Brief
+
+Released: 2026-04-07  
+Author: darcar0
+
+Hugging Face model release:
+[`darcar0/evidence-faithful-reasoning-pilot-3`](https://huggingface.co/darcar0/evidence-faithful-reasoning-pilot-3)
+
+Public GitHub release repo: `PUBLIC_REPO_URL`
+
+## Executive summary
+
+I built this project because I wanted a model release where reasoning had to
+cash out into recoverable evidence instead of hiding behind fluent language.
+The result is a strict evidence-faithful reasoning benchmark, a
+benchmark-winning hybrid system, and pilot 3: the strongest standalone model
+from the same project, released as a LoRA adapter on Hugging Face.
+
+The contract is strict. On every bounded evidence packet, the system has to:
+
+1. answer correctly,
+2. identify the right evidence,
+3. quote the exact supporting text, and
+4. abstain with `Insufficient evidence.` when the packet does not justify a
+   claim.
+
+This release ships two finished outputs. The benchmark-facing winner is a
+hybrid stack — bridge `checkpoint-2` plus `deterministic_v3` packet-local
+quote normalization — that clears every gate on the frozen held-out
+`probe_v0` benchmark. The main downloadable artifact is pilot 3, the
+standalone model release, which beats the earlier bridge model on a fresh
+mixed public holdout and roughly doubles raw quote-faithful behavior at the
+model level.
+
+## Benchmark-facing winner
+
+The benchmark-facing winner is the strongest full system from the project.
+Training moved the model past the older frozen baseline. Deterministic
+packet-local normalization closed the remaining quote-faithfulness and strict
+grounded-success gap without leaving the bounded-packet setting.
+
+| Metric | Frozen probe_v0 |
+|---|---:|
+| Task success | **1.0000** |
+| Strict grounded success | **1.0000** |
+| Mean evidence F1 | **1.0000** |
+| Mean quote F1 | **1.0000** |
+| Verify label accuracy | **1.0000** |
+| Grounded QA accuracy | **1.0000** |
+| Contrastive consistency | **1.0000** |
+| Invalid / missing rate | **0.0000** |
+
+Canonical memo:
+[`reports/sft_v1_final_artifact_status.md`](../reports/sft_v1_final_artifact_status.md)
+
+## Standalone model release: pilot 3
+
+Pilot 3 is the strongest standalone model from the project and the release I
+want people to download first. It is the first standalone checkpoint in the
+project to hold up across multiple non-`probe_v0` evaluation surfaces, and it
+moves a meaningful amount of the winning behavior into the model itself.
+
+Fresh 36-task mixed public holdout:
+
+| Stack | Task | Strict | Evidence F1 | Quote F1 |
+|---|---:|---:|---:|---:|
+| Bridge raw | 0.8611 | 0.2222 | 0.8815 | 0.3343 |
+| Pilot 3 raw | 0.8889 | 0.4444 | 0.9093 | 0.6815 |
+| Bridge + `deterministic_v3` | 0.8611 | 0.5833 | 0.8815 | 0.8815 |
+| **Pilot 3 + `deterministic_v3`** | **0.8889** | **0.5833** | **0.9093** | **0.9093** |
+
+Pilot 3 beats bridge on task accuracy, evidence F1, and quote F1 in both raw
+and normalized form, ties normalized strict, and roughly doubles raw quote F1
+(`0.3343` → `0.6815`) at the model level.
+
+Canonical memos:
+- [`reports/standalone_model_v2_freeze_memo.md`](../reports/standalone_model_v2_freeze_memo.md)
+- [`reports/standalone_model_v2_holdout_v1_bridge_vs_pilot3_status.md`](../reports/standalone_model_v2_holdout_v1_bridge_vs_pilot3_status.md)
+
+## Project arc and stopping point
+
+The project followed a full research-engineering loop: baseline mapping,
+benchmark design, prompt and structure interventions, a training-backed bridge
+model, deterministic quote normalization, and then a teacher-student
+distillation cycle to move the winning behavior into the model itself.
+
+A targeted follow-up, pilot 4, fixed one specific FEVER
+month/date temporal-insufficiency case but weakened broader behavior on larger
+evaluation surfaces. I treated that as a stop signal rather than churning for
+one more pilot. The release froze at the point where the standalone model was
+strongest and the benchmark-facing system was already complete.
+
+## Intended use and boundaries
+
+This is a specialized grounded reasoning release, not a general-purpose
+chatbot replacement. It is built for bounded document QA, claim verification,
+policy/compliance workflows, and other settings where every answer has to be
+justified from a closed body of text.
+
+Important boundaries:
+
+- Perfect `probe_v0` belongs to the hybrid stack, not to pilot 3 alone.
+- The Hugging Face download is the LoRA adapter only; the benchmark-winning
+  configuration is adapter + `deterministic_v3`.
+- Frozen `probe_v0` item-level contents are intentionally not published with
+  the release.
+
+## Release surfaces
+
+- Pilot 3 on Hugging Face:
+  [`darcar0/evidence-faithful-reasoning-pilot-3`](https://huggingface.co/darcar0/evidence-faithful-reasoning-pilot-3)
+- Technical note:
+  [`docs/technical_note_evidence_faithful_reasoning.md`](./technical_note_evidence_faithful_reasoning.md)
+- Release page:
+  [`release/index.html`](../release/index.html)
+- Public GitHub release repo: `PUBLIC_REPO_URL`