Joakimpalm-Zen/Muse-Glimmer-30B-Surgical-Q4_K-GGUF

Muse-Glimmer-30B, surgical: 4.75% of the decoder removed, healed, Q4_K

Serve with xyntetik-runner — the engine that trains, merges, and quantizes the GGUF it serves

This GGUF is built to be served with xyntetik-runner, a single-binary, dependency-free C11 inference engine (CPU / CUDA / Metal, OpenAI-compatible server). For this artifact the relevant differentiator is the training side:

Runner trains, merges, and quantizes GGUFs deterministically: the same data and seed reproduce the same adapter file byte for byte, and every written artifact carries a provenance record.

One binary covers the loop: --train (AdamW LoRA on the served GGUF, deterministic by default, gated in CI by adapter sha256), --lora to serve the adapter beside the frozen base, --merge-lora to fold it in with per-tensor requantization and a merge.json provenance record, and --score for teacher-forced held-out evals. Adapters interoperate both ways: a Runner-trained adapter scores identically (1.000 on its held-out eval) served by stock llama.cpp, and community F16 adapters load straight back — measured, not assumed.

Measured — the finding every fine-tuner should know before merging (held-out eval where the fine-tune scores 1.00 and the base model 0.69; verified in two independent runtimes):

merge output type	eval after merge	fine-tune survives?
F16	1.00	yes
Q8_0	1.00	yes
Q4_0	0.69 — the base model's score	no: erased

A 4-bit merge silently deletes the adaptation while perplexity still looks fine. Merge to Q8_0 or F16, or keep serving base + adapter.

This artifact is that toolchain eating its own cooking. The surgery was applied by byte-level substitution into the parent GGUF, and the Q4_K file was written by Runner's own quantizer. The healed weights survive Q8_0 at a cost of +0.00006 KLD, and the Q4_K cost you see in the fidelity table was PREDICTED to 0.6% before the file was built, from the measured independence of quantization noise and surgery error. If you fine-tune on top of this model, the merge-erasure table above applies to you.

Quickstart

# the same binary that wrote this artifact:
runner -m model.gguf --train data.jsonl --train-out adapter.gguf
# same data + seed -> byte-identical adapter, build provenance in adapter.gguf.train.json
runner -m model.gguf --lora adapter.gguf --merge-lora merged --quant q8_0
# provenance lands in merged.gguf.merge.json; do not merge to 4-bit (table above)

The exact boundary of the determinism claim, including what is deliberately not promised: docs/determinism-scope.md.

Run it

This file is 14.61 GB, so it wants a machine with roughly 17 GB of usable memory or more.

# Apple Silicon Mac
curl -L https://github.com/Joakimpalm-Zen/xyntetik-runner/releases/latest/download/runner-macos-arm64.tar.gz | tar xz
curl -L -o Muse-Glimmer-30B-surgical-4p75-Q4_K.gguf \
  https://huggingface.co/Joakimpalm-Zen/Muse-Glimmer-30B-Surgical-Q4_K-GGUF/resolve/main/Muse-Glimmer-30B-surgical-4p75-Q4_K.gguf
./runner-macos-arm64 -m Muse-Glimmer-30B-surgical-4p75-Q4_K.gguf -i

Linux and Windows builds are on the same release page. Any GGUF runtime with muse-glimmer support should load it: every tensor is an ordinary GGUF type in the standard shape; 9 FFN tensors happen to be exact zeros, which is the surgery.

A 14.61 GB GGUF of Meta's Muse-Glimmer-30B with the FFN sublayers of layers 4, 9 and 48 structurally removed (weights exactly zero, standard tensor shape), the damage repaired by short local distillation against the frozen parent, and the result quantized to Q4_K.

It passes a strict fidelity bar against the unmodified BF16 parent on a held-out split: margin-qualified top-1 agreement 97.74% (bar: >=97%) and mean KLD 0.04810 (bar: <=0.05).

Measured fidelity vs the BF16 parent

artifact	size	mean KLD	margin-qualified top-1	bar
this file (surgery + heal, Q4_K)	14.61 GB	0.04810	97.74%	PASS
same weights, Q8_0	27.58 GB	0.03076	98.65%	PASS
same weights, BF16	51.90 GB	0.03056	98.64%	PASS
parent at Q4_K (quant-alone control)	14.61 GB	0.01930	99.12%	PASS

Per-domain behavior: the healed model sits within a few percent of the parent in every measured domain (11-domain perplexity spread 0.046 vs 0.373 for un-healed surgery). Healing distils toward the parent: this is a smaller model that behaves like the original, not a different model that happens to score well.

Paired benchmark check (2026-08-29): the fidelity bar does not transfer

The fidelity table above is a real, strict measurement. This section reports what it does not buy. One preregistered measurement night translated the house bar into paired public-benchmark deltas versus the frozen parent: lm-evaluation-harness 0.4.9, stock task configs, both models scored through the identical loglikelihood path on identical items, per-item pairing, exact binomial confidence intervals on the discordant pairs, margins fixed before sampling.

config	benchmark	preregistered margin	net paired delta	95% CI	verdict
these weights at BF16 (pre-quant)	HellaSwag acc_norm	±0.5 pts	−0.916	[−1.230, −0.583]	FAIL

The flip counts say the deficit is directional, not noise: on the 2.89% of items where the two models disagree, the surgical model loses roughly 2:1 (191 parent-right-variant-wrong against 99 the reverse).

Stated plainly:

• This model is measurably worse than its parent on HellaSwag. The deficit is about 0.9 points and the confidence interval lies entirely beyond the preregistered ±0.5 equivalence margin. Do not read the fidelity table above as benchmark equivalence: the two criteria disagree, and the paired test is the stricter one.
• The deficit was measured at the BF16 surgery stage. The shipped file adds Q4_K quantization on top (its own fidelity row is above). A direct benchmark row for the quantized file is currently blocked for tooling reasons (no available continuation-loglikelihood serving path for this architecture), not by choice; it will be added when the path exists, and there is no reason to expect quantization to improve it.
• Across the two surgery levels the KLD-to-benchmark mapping is monotone and super-linear (30.0 to 39.5 points lost per unit of KLD, from 4.75% to 6.34% removed): deeper surgery costs more benchmark accuracy per unit of measured divergence, so fidelity numbers under-predict benchmark harm as surgery grows.

The parent-fidelity table remains this card's primary claim: these weights track the parent's token distribution closely, and that is what they are for. The benchmark row exists so nobody has to take distribution fidelity as a proxy for task equivalence. We measured that it is not one.

How it was made, in five lines

1. Depth-sensitivity mapping found the FFN sublayers whose removal is cheapest, chosen depth-separated (measured to matter).
2. The three FFNs were removed; un-healed, that fails the bar (KLD 0.10021).
3. One FFN per cut (the first surviving layer below it) was retrained briefly against the frozen parent: 7.4 M tokens, ~28 minutes on one 24 GB GPU slice. Healed BF16: 0.03056, better than the parent with only two un-healed cuts.
4. Quantization noise and surgery error were measured to be additive and independent (0.965 to 0.994 times the naive sum across four configurations), so the Q4_K cost was PREDICTED before the file was built: predicted 0.0484, measured 0.04810.
5. The GGUF was produced by byte-level substitution of the 18 modified tensors into the parent GGUF, then quantized by xyntetik-runner's own quantizer. The BF16 path was validated exactly (KLD 0.00000 against the source checkpoint).

Notes and limits

• The 9 removed-FFN tensors are stored as exact zeros in standard shape, so any runtime that runs the parent GGUF can run this file with no loader changes. The zeros still occupy their quantized bytes; true tensor removal would shave a further ~5%.
• Tested in xyntetik-runner (native muse-glimmer support). Other GGUF runtimes should work if they run the parent model at Q4_K; that is expected, not verified here.
• The fidelity bar is parent-agreement on a held-out split (45,056 positions), not a capability benchmark. The full method write-up, including the depth-absorbability law and the quantization-independence result: Muse-Glimmer-30B-surgery-report.
• Healing used 135k unique training tokens (a deliberately small budget); the numbers above are lower bounds on what the method yields with more data.

Base model: Muse-Glimmer-30B by Meta, Apache-2.0. This derivative keeps the license and changes 18 tensors (9 zeroed, 9 replaced with healed weights) as described.

Decontamination re-score (2026-08-29)

A window-level audit of the healing corpus found that 283 of the eval split's 43,855 windows (0.65%, one domain) had leaked into the training pool through near-duplicate source documents. This affected every healed artifact's published number, so all of them were re-scored on the decontaminated subset.

This artifact passes on clean data: KLD 0.04903 (bar 0.05), margin-qualified top-1 97.70% (bar 97%). The published full-split numbers reproduce from the file on disk to five decimals.

Stated plainly, because thin margins should be said and not discovered: this artifact's clean-data headroom under the KLD bar is 0.00097, which is 1.37 times the measurement procedure's noise floor (plus or minus 0.00071), and its original build gate was cleared by 0.00014. It is a genuine, exactly reproducible pass with thin margins on every axis we have examined. If your use case needs comfortable headroom rather than a demonstrated boundary case, use the Q8_0 sibling, whose headroom is roughly 15 times the noise floor.

A control decomposition isolated the leak's effect: healed models score better on positions whose content appeared in healing data, by +0.0012 to +0.0014 KLD across three independent artifacts. Small, real, and measured; the naive before-and-after delta would have hidden it behind an opposing domain-mix shift. Full detail will accompany the method write-up's next revision.

Integrity

sha256(Muse-Glimmer-30B-surgical-4p75-Q4_K.gguf) =
103aa117dfce88974bd78e03ee6ca6601c720f01a1ec24ee1cfd0e5fe2f77ce6

SURGERY.json in this repo records the byte-substitution provenance: the parent revision, which tensors were zeroed and which were replaced with healed weights.