Upload README.md with huggingface_hub

README.md

@@ -21,56 +21,53 @@ tags:
   - research
 ---
 
-# Evidence-Faithful Reasoning — Pilot 3
+# Evidence-Faithful Reasoning
 
-**Pilot 3 is a LoRA adapter that makes its reasoning-distilled 27B base model
-earn every answer from a bounded evidence packet.**
+*Pilot 3 is the standalone model release.*
 
-I built this project because I wanted a release where reasoning had to cash
-out into recoverable evidence, not just fluent confidence. In this project,
-the model has to point to the right packet units, quote exact support, and
-abstain cleanly when the text runs out.
-
-Given a closed packet of source text and a task, the model has to do four
-things at once:
-
-1. answer correctly,
-2. identify the right evidence units,
-3. quote the exact supporting text, and
-4. abstain with `Insufficient evidence.` when the packet does not justify
-   a claim.
-
-Pilot 3 is the strongest standalone model from that project. The same project
-also produced a benchmark-winning hybrid system (bridge `checkpoint-2` plus a
-deterministic packet-local normalizer); **this page is the front door for the
-standalone model.**
+This adapter turns its reasoning-distilled 27B base model into an
+evidence-first reader for closed packets of text. I built it because I wanted
+a release where reasoning had to prove itself: every answer has to land on the
+right evidence, quote that evidence verbatim, and stop with
+`Insufficient evidence.` when the packet does not justify a claim. The result
+is the strongest standalone model from the project, packaged here as a LoRA
+adapter you can load directly tonight.
 
 ## Resources & Guides
 
 - [Technical brief (PDF)](./evidence_faithful_reasoning_release_brief.pdf)
 - [Technical note](./technical_note_evidence_faithful_reasoning.md)
 - [Fresh public holdout chart](./standalone_holdout_comparison.svg)
-- Public GitHub release repo: `PUBLIC_REPO_URL`
+- [Frozen benchmark progression chart](./benchmark_progression.svg)
+- [Release architecture chart](./project_release_arc.svg)
+
+![Fresh public holdout: standalone release vs bridge](./standalone_holdout_comparison.svg)
 
-![Pilot 3 release proof](./standalone_holdout_comparison.svg)
+*Fresh 36-task mixed public holdout: the standalone release beats the earlier
+bridge model on task accuracy, evidence F1, and quote F1, while the
+packet-local normalizer lifts the full stack to `0.9093` quote F1.*
 
-*Fresh 36-task mixed public holdout: pilot 3 beats bridge on task accuracy,
-evidence F1, and quote F1, while tying normalized strict after
-`deterministic_v3`.*
+## Why this release exists
 
-## Release highlights
+I built this project to force reasoning models to show their work in the only
+place that counts: the evidence itself. Fluent answers were not enough. I
+wanted a model that had to retrieve the right units, quote them exactly, and
+fail closed when the packet ran out. This page leads with the standalone
+release because it is the artifact you can load immediately, inspect directly,
+and use without reconstructing the whole benchmark stack.
+
+## At a glance
 
-- **Strongest standalone model from the project.** First standalone
-  checkpoint to hold up across multiple non-`probe_v0` evaluation
-  surfaces.
-- **Roughly doubles raw quote-faithfulness** over the earlier bridge
-  model on a fresh public holdout (raw quote F1 `0.3343` → `0.6815`).
-- **Beats bridge on a fresh 36-task mixed public holdout** in task
-  accuracy, evidence F1, and quote F1, while tying normalized strict
-  grounded success.
-- **Zero invalid outputs** on every reported evaluation surface.
 - LoRA adapter on top of
   [`Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2`](https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2).
+- Strongest standalone model from the project and the release I want people to
+  download first.
+- On a fresh 36-task public holdout, raw task improves from `0.8611` to
+  `0.8889`, raw strict from `0.2222` to `0.4444`, and raw quote F1 from
+  `0.3343` to `0.6815` over the earlier bridge model.
+- Zero invalid outputs on every reported evaluation surface.
+- The project also produced a benchmark-winning hybrid stack, but that is a
+  separate result described under *Release architecture*.
 
 ## Quick start
 
@@ -88,14 +85,12 @@ base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
 model = PeftModel.from_pretrained(base, adapter_id)
 ```
 
-The base model is 27B parameters — load it in your usual quantization.
+The base model is 27B parameters, so load it in your usual quantization.
 
-### How to prompt it
+## Prompt format
 
-Pilot 3 expects an evidence-first strict-contract prompt. The system
-message tells the model to find the smallest sufficient evidence set,
-copy verbatim quotes from those units, and emit a structured JSON
-response. A short version of the recommended prompt:
+This release works best with an evidence-first prompt that makes the answer
+subordinate to the cited text. A minimal version:
 
 ```
 You are answering from a bounded evidence packet only.
@@ -129,19 +124,13 @@ The model then writes a JSON object with this shape:
 }
 ```
 
-The prompt skeleton above is enough to run the adapter directly. In the
-benchmark-winning configuration, the JSON output is then passed through a
-deterministic packet-local quote normalizer; *Project context* explains what
-that adds and what it does not.
-
-## Evaluation results
+## Evaluation
 
 ### Fresh 36-task mixed public holdout
 
-A held-out slice of 18 FEVER verify-claim tasks plus 18 HotpotQA grounded
-QA tasks, drawn from public sources and de-duplicated against every
-training, dev, and `probe_v0` row. Source: the standalone holdout
-comparison report.
+A held-out slice of 18 FEVER verify-claim tasks plus 18 HotpotQA grounded-QA
+tasks, drawn from public sources and de-duplicated against every training,
+dev, and `probe_v0` row.
 
 | Stack | Task | Strict | Evidence F1 | Quote F1 |
 |---|---:|---:|---:|---:|
@@ -150,10 +139,9 @@ comparison report.
 | Bridge + `deterministic_v3` | 0.8611 | 0.5833 | 0.8815 | 0.8815 |
 | **Pilot 3 + `deterministic_v3`** | **0.8889** | **0.5833** | **0.9093** | **0.9093** |
 
-Pilot 3 beats bridge on task accuracy, evidence F1, and quote F1 in both
-raw and normalized form, ties normalized strict, and roughly **doubles**
-raw quote F1 at the model level. Grounded-QA accuracy on this slice is
-`1.0000`.
+The standalone release beats the earlier bridge model on task accuracy,
+evidence F1, and quote F1 in both raw and normalized form, ties normalized
+strict, and roughly doubles raw quote F1 at the model level.
 
 ### Fixed dev triage slice (21 tasks)
 
@@ -163,110 +151,72 @@ raw quote F1 at the model level. Grounded-QA accuracy on this slice is
 
 ### Untouched 104-task Hotpot shadow slice
 
-Pilot 3 raw improved quote-faithful behavior over the raw bridge model on
-this slice, and pilot 3 + `deterministic_v3` matched bridge +
-`deterministic_v3` at the system level. Reported as a parity outcome in
-the standalone freeze memo.
+Pilot 3 raw improved quote-faithful behavior over the raw bridge model on this
+slice, and pilot 3 + `deterministic_v3` matched bridge +
+`deterministic_v3` at the system level. That surface remains a narrative
+parity result because the report does not publish per-metric cells for it.
 
-## Project context
+## Release architecture
 
-The release ships two distinct outputs from one project:
+This project ends in two finished artifacts, not one:
 
-1. **Benchmark-winning hybrid system** — bridge `checkpoint-2` plus
-   `deterministic_v3` packet-local normalization. Perfect on the frozen
-   held-out `probe_v0` benchmark under the full
-   answer–evidence–quote–abstain contract: task `1.0000`, strict
-   grounded success `1.0000`, evidence F1 `1.0000`, quote F1 `1.0000`,
-   verify accuracy `1.0000`, invalid rate `0.0000`. This is the
-   strongest full system result from the project.
-2. **Pilot 3 standalone model** — this Hugging Face release. The
-   strongest version of the project's evidence-faithful behavior that
-   moved into the model itself, evaluated across multiple non-`probe_v0`
-   surfaces.
+1. **Standalone model release** — this page. Pilot 3 is the strongest
+   version of the project's evidence-faithful behavior that moved into the
+   model itself, evaluated across multiple non-`probe_v0` surfaces.
+2. **Benchmark-facing hybrid stack** — bridge `checkpoint-2` plus the
+   `deterministic_v3` packet-local normalizer. That stack is the benchmark
+   winner and the only configuration that clears every gate on frozen held-out
+   `probe_v0`.
 
-The two releases are intentionally separate. The hybrid stack is the
-benchmark-facing winner. Pilot 3 is the main downloadable model release.
-Perfect `probe_v0` belongs to the hybrid stack, not to pilot 3 alone.
+The separation is deliberate. This page is for the standalone release you can
+download now. The benchmark winner is documented here because it explains the
+project's full result, not because those perfect `probe_v0` numbers belong to
+the adapter alone.
 
 ## Intended use
 
-This is a **specialized grounded reasoning model**, not a general-purpose
-chatbot replacement. It is built for:
+Use this release for work that has to stay inside a fixed body of text:
 
 - bounded document QA with explicit evidence requirements,
-- claim verification and grounded QA from fixed evidence packets,
-- policy, compliance, contract, and internal-document workflows where
-  every answer has to be justified from a closed body of text,
+- claim verification and grounded QA from closed evidence packets,
+- policy, compliance, contract, and internal-document workflows where each
+  answer must be justified from the provided text,
 - research on evidence-faithful reasoning and abstention behavior.
 
-The contract is strict by design: correctness alone does not count as
-success. Every answer must come with recoverable support, and the model
-must abstain when the packet does not justify a claim.
-
-## Training data
-
-Training and evaluation surfaces are public-data-backed and derived from:
-
-- **FEVER** — verify-claim data with support / contradict / insufficient
-  labels.
-- **HotpotQA** — multi-hop grounded QA over short evidence packets.
-- Project-local bounded packet scaffolding built on top of those
-  upstream sources.
-
-The held-out benchmark `probe_v0` was kept frozen and was **not** used as
-a tuning surface for the standalone selection cycle that produced this
-adapter.
-
 ## Limitations
 
-- **The downloadable artifact is the LoRA adapter only.** The
-  `deterministic_v3` packet-local normalizer is a separate post-processing
-  step that lives in the project repository. The benchmark-winning
-  configuration is *adapter + normalizer*; downloading the adapter alone
-  reproduces the standalone-model numbers, not the perfect hybrid
-  numbers.
-- **Perfect `probe_v0` belongs to the hybrid stack, not to pilot 3
-  alone.** Treat `1.0000` numbers on `probe_v0` as the hybrid stack's
-  result.
-- **Specialized for bounded evidence packets.** This is not a
-  general-purpose chatbot or open-domain QA model. Performance outside
-  the closed-packet setting is not characterized.
-- **Frozen benchmark contents are not published with the release.** Raw
-  `probe_v0` item-level contents are intentionally withheld to preserve
-  the held-out gate.
-- **Perfect `probe_v0` is not proof of general faithful reasoning.** It
-  is proof that the system meets the strict contract on a single frozen
-  bounded benchmark.
-
-## Why pilot 3 is the release checkpoint
-
-The project deliberately stopped at pilot 3.
-
-A targeted follow-up, pilot 4, was built to fix one specific FEVER
-month/date temporal-insufficiency error. It fixed that single row, but
-it weakened broader behavior on the larger evaluation surfaces. That
-made pilot 4 a stop signal — useful negative evidence that further
-local-fix iteration was trading visible gains for wider regressions —
-not a better release. The project froze at the last point where the
-standalone model was strongest across multiple surfaces.
-
-## References
-
-- Base model: [Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2](https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2)
-- FEVER: [fever/fever](https://huggingface.co/datasets/fever/fever)
-- HotpotQA: [hotpotqa/hotpot_qa](https://huggingface.co/datasets/hotpotqa/hotpot_qa)
-- Technical brief (PDF): [evidence_faithful_reasoning_release_brief.pdf](./evidence_faithful_reasoning_release_brief.pdf)
-- Technical note: [technical_note_evidence_faithful_reasoning.md](./technical_note_evidence_faithful_reasoning.md)
+- The downloadable artifact is the LoRA adapter only. The base model is
+  required.
+- The `deterministic_v3` packet-local normalizer is not included in this
+  download. The benchmark-winning configuration is adapter + normalizer, while
+  the adapter alone reproduces the standalone-model results shown above.
+- Perfect `probe_v0` belongs to the benchmark-facing hybrid stack, not to this
+  adapter alone.
+- Specialized for closed-packet reasoning, not open-ended chat or open-domain
+  QA.
+- Frozen `probe_v0` item-level contents are intentionally not published with
+  the release.
 
 ## Citation
 
+References:
+
+- Base model:
+  [Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2](https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2)
+- Datasets:
+  [fever/fever](https://huggingface.co/datasets/fever/fever),
+  [hotpotqa/hotpot_qa](https://huggingface.co/datasets/hotpotqa/hotpot_qa)
+- Technical brief (PDF):
+  [evidence_faithful_reasoning_release_brief.pdf](./evidence_faithful_reasoning_release_brief.pdf)
+- Technical note:
+  [technical_note_evidence_faithful_reasoning.md](./technical_note_evidence_faithful_reasoning.md)
+
 ```bibtex
-@misc{evidence_faithful_reasoning_pilot_3_2026,
-  title        = {Evidence-Faithful Reasoning --- Pilot 3},
-  author       = {{darcar0}},
+@misc{darcar0_evidence_faithful_reasoning_pilot_3_2026,
+  title        = {Evidence-Faithful Reasoning: Pilot 3},
+  author       = {darcar0},
   year         = {2026},
   howpublished = {Hugging Face model release},
-  url          = {https://huggingface.co/darcar0/evidence-faithful-reasoning-pilot-3},
-  note         = {Standalone LoRA adapter from the evidence-faithful reasoning project.}
+  url          = {https://huggingface.co/darcar0/evidence-faithful-reasoning-pilot-3}
 }
 ```