| title: Reproduction - Differentiable Conformal Training for LLM Reasoning Factuality | |
| emoji: 🧭 | |
| colorFrom: indigo | |
| colorTo: blue | |
| sdk: static | |
| pinned: false | |
| short_description: Native released-data DCF audit; 6/6 claims supported | |
| tags: | |
| - trackio | |
| - open-reproductions | |
| - icml2026-repro | |
| - paper-XfndtVLIub | |
| # Native released-data reproduction | |
| The exact official DCF implementation at commit `0b4d5487a9868a18c4f9aa5b3d96cdccc705ca97` runs on all 50 released MATH reasoning graphs (503 claim nodes). The audit also recomputes ten 14,600-decision agreement rows, the MATH/FELM headline metrics, five released calibration suites, two prediction suites, an end-to-end gradient, and an ancestor-removal destructive control. | |
| Claim 1 is directly falsified as a composite reliability claim: its retention gain is real, but released coverage misses the 97% target. Claims 2–6 are verified within their exact source scope. No paper-scale result was invented, substituted, or repaired with a nearby result. | |
| ```bash | |
| python3 -m pip install -r requirements.txt | |
| PYTHONDONTWRITEBYTECODE=1 PYTHONHASHSEED=0 PYTHONWARNINGS=error python3 native_release_audit.py --output-dir /tmp/dcf-native | |
| PYTHONDONTWRITEBYTECODE=1 PYTHONHASHSEED=0 PYTHONWARNINGS=error python3 validate_evidence.py | |
| PYTHONDONTWRITEBYTECODE=1 python3 verify_manifest.py | |
| ``` | |
| The semantic-quality-gate v4 forecast is 12 verified points locally. Judge points count only after a persisted exact-current-SHA verdict. | |