|
Download pages/claim-1-math-retention/page.md from SabaPivot/repro-differentiable-conformal-training-for-llm-reasoning-factuality: direct link, hf CLI and curl.
- Browser
- Download file 1.08 kB
-
https://huggingface.co/spaces/SabaPivot/repro-differentiable-conformal-training-for-llm-reasoning-factuality/resolve/main/pages/claim-1-math-retention/page.md
- Command line
-
hf download hf://spaces/SabaPivot/repro-differentiable-conformal-training-for-llm-reasoning-factuality/pages/claim-1-math-retention/page.md
-
curl -L -o page.md https://huggingface.co/spaces/SabaPivot/repro-differentiable-conformal-training-for-llm-reasoning-factuality/resolve/main/pages/claim-1-math-retention/page.md
1.08 kB
Claim 1: Differentiable Coherent Factuality (DCF) achieves up to a 141% improvement in claim retention over frequency-based baselines on the MATH dataset at reliability level α=0.03 (1.76 vs. 0.73 claims retained) (Section 4.3).
Verdict: falsified as the literal composite reliability claim. The exact official MATH record gives 1.76045 retained claims versus 0.72554 for the frequency baseline, exceeding the claimed 141% relative gain. But released DCF coverage is 96.545% against the 97% target, a 0.455-percentage-point shortfall. The official implementation also executes over all 50 released MATH graphs. The gain is preserved; the phrase “at reliability level α=0.03” is not rounded into a pass.