ZTFlynn/LFM2-350M-Cascadia-ternary3

LiquidAI/LFM2-350M compressed to 241 MB with Cascadia — a spline manifold plus per-band lookup tables at 0.675 bytes per weight — and executable on CPU by a C runtime whose entire dependency list is libc, libm and libgomp.

Base model	LiquidAI/LFM2-350M
Parameters	16 layers, hidden 1024
Checkpoint → package	676 MB → 241 MB (2.96x)
Bits per weight	5.40
Tensors compressed	16
Resident memory	298 MB on CPU, 355 MB on GPU — 1.24x the package
Architecture	16 blocks, GQA 16q/8kv, gated short convolutions

Quality

	perplexity
LiquidAI/LFM2-350M (bf16)	246.04
This package (ternary-3)	248.74
Result	+1.10% perplexity — marginal (95% CI [1.0003x, 1.0219x], t = +2.01)

Against the 4-bit formats the hardware runs natively

NVIDIA Blackwell executes NVFP4 and MXFP4 in the tensor cores with no software decode. Cascadia derives a per-element band instead of storing a per-block scale, which costs real decode time -- so the question is whether it buys anything. Every format below was scored through one evaluation path on the same 119,574 tokens, with the same window-clustered pairing.

format	bytes/weight	perplexity	vs this package
bf16 (uncompressed)	2.0000	242.65	—
Cascadia ternary-3	0.6746	244.80	—
NVFP4 (e2m1 + e4m3 / 16)	0.5625	266.74	1.090x worse (t = +7.80)
int4 + fp16 scale / 32	0.5625	270.84	1.106x worse (t = +9.55)
MXFP4 (e2m1 + ue8m0 / 32)	0.5312	314.84	1.286x worse (t = +22.44)

Cascadia is 20% larger than NVFP4 and resolvably better than all three, while its own gap to bf16 is not statistically resolvable on this corpus (t = +1.52). The result is size-dependent and does not generalise: on the smaller LFM2.5-230M, int4 with a per-32 fp16 scale beats Cascadia while being smaller.

Reproduce with tools/format_shootout.py. Its cascadia arm reconstructs this package and is checked against the C runtime's own perplexity before any number is reported; a mismatch aborts rather than prints.

Running it

Measured on a Jetson AGX Thor: CPU is 12 threads of a 14-core Arm part, GPU is the integrated Blackwell. Decode is the median of five runs; prefill is from a 162-token prompt. Both backends produce byte-identical output.

	CPU (12 threads)	GPU
decode	8.06 tok/s	63.57
prefill	55.7 tok/s	228.3

Load takes 0.48 s on CPU and 0.28 s on GPU, the latter including the host-to-device upload.

117,530 paired tokens of FineWeb-Edu in 229 independent 512-token windows. Both models score identical tokens and are compared per token, which cuts the standard error 10.3x versus two independent means.

The window is the unit of inference, not the token: tokens inside one window share a context, and counting them as independent samples inflates the t-statistic several-fold. Resolving a difference of a few percent takes hundreds of windows.

A cost near the edge of what this sample resolves: t = +2.01 clears the conventional 1.96 on its own, but 13 models were measured together and holding family-wise error at 5% across them needs |t| > 2.89. Read 1.1% as the point estimate, with the interval above carrying the real uncertainty.

The corpus is general web text (FineWeb-Edu). How much a given model is affected depends on its own sensitivity and on how close that text is to its domain, so this figure is specific to both. Two models in this family compressed to the same 0.055 reconstruction error measured +13.4% and −2.4% here. The fidelity figure below describes the compression itself and does not vary that way.

Reconstruction fidelity

Perplexity measures how good a model is on a corpus, not how faithful a copy is, and the two disagree here: models compressed to identical reconstruction error differ by 16 percentage points of measured perplexity. Fidelity has no sampling uncertainty and no dependence on corpus domain, so it is measured directly and reported alongside.

Relative L2 error vs the bf16 checkpoint	0.0533
Systematic gain (1.0000 is faithful)	0.9993
Measured over	93 of 93 tensors, 100% of parameters

By tensor class:

class	rel L2	share of model
linear	0.0584	287M params
embedding	0.0272	67M params

The tied embedding, the tensor whose error reaches the logits undamped, reconstructs at 0.0272.

Where the compression cost comes from

The cost is concentrated in one tensor. The tied embedding, which also serves as lm_head, is compressed by a single global codebook — no bands, no spline manifold, no exact outliers — while every linear tensor gets 32 bands, a spline, and 0.5% of its weights kept exact. Its error is the only error in the model that reaches the logits with nothing downstream to absorb it.

Measured on LFM2-350M, relative L2 reconstruction error:

tensor	codebook	rel L2
tied embedding, 27 entries	27	0.078
tied embedding, 81 entries	81	0.027
a typical linear (32 bands x 27)	864	0.057

At 27 entries the embedding is the worst-reconstructed tensor in the model. This package uses 81 entries for it, which costs about 6% in size and makes it the best-reconstructed tensor instead.

Usage

Executed by the Cascadia C runtime. This is a compressed package, not a transformers checkpoint.

git clone https://github.com/EntroMorphic/cassie && cd cassie
cmake -S src/c -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j

huggingface-cli download ZTFlynn/LFM2-350M-Cascadia-ternary3 --local-dir ./pkg
./build/cascadia_generate ./pkg 512 --chat "Explain gradient descent."

Sampling is --temp / --top-k / --top-p / --seed; the default is greedy and seed-reproducible. Generation stops at <|im_end|>, so max_new is a ceiling.

On an NVIDIA GPU

The same package runs entirely on the GPU, with byte-identical output and roughly 14-16x the throughput. Opt-in, so the default build is unchanged:

cmake -S src/c -B build -DCASCADIA_CUDA=ON -DCMAKE_BUILD_TYPE=Release \
    && cmake --build build -j
./build/cascadia_generate_cuda ./pkg 512 --chat "Explain gradient descent."

Same arguments, same tokens. Needs CUDA and sm_75 or newer.

Python

from transformers import AutoModelForCausalLM
from cascadia import load_compressed

model = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM2-350M", dtype="bfloat16")
model, stats = load_compressed(model, "./pkg", model_id="LiquidAI/LFM2-350M")

Sample output

Prompt: "How many eggs are in a baker's dozen?"

A baker's dozen is a common measurement used in baking, particularly in recipes that involve a large number of eggs. A standard baker's dozen is defined as 12 eggs. Therefore, a baker's dozen contains 12 eggs.

Greedy, generated to natural completion.

Package contents

file	size
weights.bin	239 MB
manifest.json	per-tensor geometry and offsets
aux.bin	RMSNorm scales, conv kernels, architecture constants
tokenizer.bin	vocabulary, merges, Unicode tables

Format specified in docs/package_format.md and machine-verified against every package.

How it works

A B-spline surface is fitted to each weight matrix to capture large-scale structure. Each weight is assigned to one of 32 bands by its spline value, and a k-means codebook is learned per band over the residuals. The top 0.5% of errors are kept exactly as f32. Codebook indices pack in base 3, five trits per byte, since 3⁵ = 243 fits a byte.

Reconstruction is W = spline(j,c) + codebook[band][index], evaluated inside the matvec so no dense weight matrix is ever built. Because the spline carries dynamic range, the residual tables need no per-block scale factors.

Limitations

• Runs under the Cascadia C runtime rather than transformers directly.
• The runtime executes ternary-3 packages; other presets convert but are not yet supported by the kernel.
• Batch-1 CPU inference, suited to edge and batch workloads.
• Greedy and sampled decoding; no beam search.

Acknowledgements

Deeply inspired by Magneato/deepseek-r1-qwen-7b-lutc, which demonstrated LUT-cascade compression of a 7B model at 5.45 bits per weight. The Guanaco LUT cascade — no-scale residuals, variable bit rate, and f32 outlier preservation — is the foundation this builds on. Cascadia adds a spline manifold for band selection and a Harmonic Collapse step that removes per-block scale factors entirely. Our thanks to Magneato for publishing both the approach and the weights that made it concrete.

Base model by Liquid AI, used under the LFM Open License.

Citation

@software{cascadia,
  title  = {Cascadia: Spline Manifold LUT Compression for Language Models},
  author = {Josserand-Austin, Tripp},
  year   = {2026},
  url    = {https://github.com/EntroMorphic/cassie}
}