The 3-way PINN benchmark — measured, not asserted

SZL's governed classical spectral solver (CPU) versus two neural physics-informed networks — DeepXDE and NVIDIA PhysicsNeMo / Modulus (both GPU). Three arms, three problems, all MEASURED against exact closed forms or synthetic ground truth. We publish the config, the seeds, the license of every dependency, and — most importantly — where this suite does not let us claim victory. That last part is the product.

Honesty gate: PASS in CI Doctrine v11 labels Live artifact · /api/a11oy/v1/pinn/bench

This page renders directly from the committed benchmark artifact — no numbers are typed into the page, so it cannot drift from the source of truth. The same artifact must pass benchmarks/pinn/honesty_guard.py before it can ship.

Loading measured results from /api/a11oy/v1/pinn/bench…

How to read this

What we are not claiming

Reproduce it yourself

Nothing here is take-our-word-for-it. The exact commands that produced the artifact:

Why this benchmark is different

Content-provenance tools prove where a file came from. Governance platforms document a process. Evaluation suites score a model. None of them break the build when a result overclaims. a11oy does:

Every reported number is stamped MEASURED / MODELED / NOT-RUN / NOT-MEASURED / NOT-TESTED.
A measured number with no evidence, or a not-run arm carrying a value, fails the honesty gate — the artifact cannot ship.
Our advantages are disclosed: e.g. the Poisson solution lives inside SZL's trial basis, so that near-machine-precision result is by construction, not a general-accuracy claim.

Verify the source

This page is public and so is everything behind it. Inspect the code that produces — and polices — these numbers on the public szl-holdings/a11oy repository:

The measured artifact — benchmarks/pinn/results.json, served read-only at /api/a11oy/v1/pinn/bench (this page renders it — no numbers are typed in).
The benchmark harness — benchmarks/pinn/run_bench.py. The SZL arm is NumPy-only; the DeepXDE / PhysicsNeMo arms are lazy-imported and never shipped.
The honesty gate — benchmarks/pinn/honesty_guard.py over the reusable verifiable-ai/ core: a MEASURED number with no evidence, or a NOT-RUN arm carrying a value, fails the build.
The reproducibility gate — benchmarks/pinn/reproduce_szl_guard.py: CI re-derives the SZL classical-spectral arm from source and asserts it still matches the published numbers.
The CI itself — pinn-honesty-gate.yml · live runs on GitHub Actions.

Both gates run on every change before it can reach main: the labels cannot overclaim, and the classical-arm numbers cannot drift from source. That is what verifiable means here — not a promise, a build check.