Add README.md

README.md

@@ -23,7 +23,7 @@ language:
 
 # ZTFlynn/LFM2-1.2B-Extract-Cascadia-ternary3
 
-[`LiquidAI/LFM2-1.2B-Extract`](https://huggingface.co/LiquidAI/LFM2-1.2B-Extract) compressed to **747 MB**
+[`LiquidAI/LFM2-1.2B-Extract`](https://huggingface.co/LiquidAI/LFM2-1.2B-Extract) compressed to **774 MB**
 with Cascadia — a spline manifold plus per-band lookup tables at **0.60 bytes
 per weight** — and executable on CPU by a C runtime whose entire dependency
 list is libc, libm and libgomp.
@@ -32,8 +32,8 @@ list is libc, libm and libgomp.
 |---|---|
 | Base model | [`LiquidAI/LFM2-1.2B-Extract`](https://huggingface.co/LiquidAI/LFM2-1.2B-Extract) |
 | Parameters | 16 layers, hidden 2048 |
-| Checkpoint → package | 2.23 GB → **747 MB** (3.14x) |
-| Bits per weight | 5.09 |
+| Checkpoint → package | 2.23 GB → **774 MB** (3.03x) |
+| Bits per weight | 5.28 |
 | Tensors compressed | 16 |
 | Architecture | 16 blocks, GQA 32q/8kv, gated short convolutions |
 
@@ -41,30 +41,65 @@ list is libc, libm and libgomp.
 
 | | perplexity |
 |---|---:|
-| `LiquidAI/LFM2-1.2B-Extract` (bf16) | 165.62 |
-| This package (ternary-3) | 175.84 |
-| **Result** | **+6.17% perplexity** (95% CI [1.0468x, 1.0768x], t = +8.31) |
+| `LiquidAI/LFM2-1.2B-Extract` (bf16) | 265.77 |
+| This package (ternary-3) | 267.71 |
+| **Result** | **no detectable cost** (95% CI [0.9861x, 1.0329x], t = +0.78) |
 
-8,176 paired tokens, FineWeb-Edu, 512-token windows. Scoring both
-models on identical tokens and comparing per token cuts the standard error
-6.0x versus two independent means, which is what makes this
-resolution achievable.
+16,352 paired tokens of FineWeb-Edu in **31 independent
+512-token windows**. Both models score identical tokens and are
+compared per token, which cuts the standard error 17.0x versus
+two independent means.
 
-A measured cost. At this size the ternary-3 rate is a real trade: 3.14x compression for 6.2% higher perplexity. Larger models in this family pay far less — see the table below.
+The window is the unit of inference, not the token: tokens inside one window
+share a context, and counting them as independent samples inflates the
+t-statistic several-fold. Resolving a difference of a few percent takes
+hundreds of windows.
 
-### How compression cost scales with model size
+The difference does not resolve. Stated precisely: this measurement bounds the perplexity change to within **3.3%** and cannot distinguish it from zero — which is a limit on the evidence, not a proof that the cost is zero. Reconstruction fidelity below is measured directly and carries no such uncertainty.
 
-Measured across the family with the same corpus and method:
+### Reconstruction fidelity
 
-| model | parameters | perplexity cost |
-|---|---:|---|
-| LFM2.5-230M | 0.23B | +7.7% |
-| LFM2-350M | 0.35B | +3.5% |
-| LFM2-24B-A2B | 24B | no detectable cost (< 0.3%) |
+Perplexity measures how good a model is on a corpus, not how faithful a copy
+is, and the two disagree here: models compressed to identical reconstruction
+error differ by 16 percentage points of measured perplexity. Fidelity has no
+sampling uncertainty and no dependence on corpus domain, so it is measured
+directly and reported alongside.
 
-Redundancy grows faster than the format's error, so larger models compress
-more nearly losslessly. Below roughly 350M parameters the ternary-3 rate
-becomes a visible trade rather than a free one.
+| | |
+|---|---:|
+| Relative L2 error vs the bf16 checkpoint | **0.0553** |
+| Systematic gain (1.0000 is faithful) | 0.9993 |
+| Measured over | 93 of 93 tensors, 100% of parameters |
+
+By tensor class:
+
+| class | rel L2 | share of model |
+|---|---:|---:|
+| linear | 0.0576 | 1,036M params |
+| embedding | 0.0265 | 134M params |
+
+The tied embedding, the tensor whose error reaches the logits undamped, reconstructs at **0.0265**.
+
+### Where the compression cost comes from
+
+The cost is concentrated in one tensor. The tied embedding, which also
+serves as `lm_head`, is compressed by a single global codebook — no bands,
+no spline manifold, no exact outliers — while every linear tensor gets 32
+bands, a spline, and 0.5% of its weights kept exact. Its error is the only
+error in the model that reaches the logits with nothing downstream to
+absorb it.
+
+Measured on LFM2-350M, relative L2 reconstruction error:
+
+| tensor | codebook | rel L2 |
+|---|---:|---:|
+| tied embedding, 27 entries | 27 | 0.078 |
+| tied embedding, 81 entries | 81 | **0.027** |
+| a typical linear (32 bands x 27) | 864 | 0.057 |
+
+At 27 entries the embedding is the worst-reconstructed tensor in the model.
+This package uses **81** entries for it, which costs about 6% in size and
+makes it the best-reconstructed tensor instead.
 
 ## Usage
 
@@ -109,7 +144,7 @@ Greedy, generated to natural completion.
 
 | file | size |
 |---|---|
-| `weights.bin` | 745 MB |
+| `weights.bin` | 772 MB |
 | `manifest.json` | per-tensor geometry and offsets |
 | `aux.bin` | RMSNorm scales, conv kernels, architecture constants |
 | `tokenizer.bin` | vocabulary, merges, Unicode tables |