infosave/Ternary-Bonsai-2-27B-cmf

Ternary Bonsai 2 27B — CMF Q2TP

CMF conversion of Prism ML's Ternary Bonsai 2 27B MLX pack, supported by Cortiq 0.6.9.

Artifact	Value
File	Ternary-Bonsai-2-27B-Q2TP-Affine.cmf
Size	8,247,698,536 bytes (7.681 GiB)
SHA-256	fb0f1a9cdb0434bc9ce50eb6473bc2ca4a71d6994ad61314bbd7e2f11c4e29e8
Runtime	Cortiq v0.6.9
Validated backends	Vulkan on NVIDIA RTX 4090; native Metal on Apple M4
Scope	Text generation; image/text fusion is not supported by this profile

What is stored

The 402 large packed matrices use existing CMF dtype16 Q2TP, with explicit required affine and signed-Hadamard operators. The affine operator preserves ternary signs and zeros; CMF's fitted scale ladder introduces a small scale approximation. This is not a Q4TP conversion, a new Q2TP-T dtype, or a claim of byte-identical upstream weights.

Auxiliary and normalization tensors retain their declared precisions. Older runtimes must reject unsupported required features; stock 0.6.8 does not support this model profile. The 0.6.9 runtime includes resident Vulkan inference and native Metal Q2 kernels; its Metal ternary optimization operates directly on the packed two-bit weights.

Install

Use the matching platform asset from v0.6.9 releases, or install from crates.io:

cargo install cortiq-cli --version 0.6.9 --locked --features gpu
cortiq --version

Linux needs a working Vulkan driver; confirm the GPU appears in vulkaninfo --summary. On macOS, use an Apple Silicon build for native Metal. For source builds, see SOURCE_BUILD.md. No custom patch is needed for 0.6.9.

Download and verify

curl -L --fail -o Ternary-Bonsai-2-27B-Q2TP-Affine.cmf \
  https://huggingface.co/infosave/Ternary-Bonsai-2-27B-cmf/resolve/main/Ternary-Bonsai-2-27B-Q2TP-Affine.cmf
# Linux:
sha256sum Ternary-Bonsai-2-27B-Q2TP-Affine.cmf
# macOS: shasum -a 256 Ternary-Bonsai-2-27B-Q2TP-Affine.cmf

cortiq verify Ternary-Bonsai-2-27B-Q2TP-Affine.cmf

Compare the complete hash with the table above.

Run: Linux / Vulkan

export XDG_RUNTIME_DIR="${XDG_RUNTIME_DIR:-/tmp}"
export WGPU_BACKEND=vulkan CMF_GPU=wgpu
export CMF_Q2TP_GPU=1 CMF_GPU_PROBE=0 CMF_GPU_WGPU_GRAPH=1 CMF_MTP=0

cortiq run Ternary-Bonsai-2-27B-Q2TP-Affine.cmf \
  --prompt 'Return only a JSON object with city equal to Казань and count equal to 3.' \
  --greedy --seed 42 --no-think --max-tokens 128

Run: Apple Silicon / native Metal

In a separate shell, without the Vulkan variables:

export CMF_GPU=1 CMF_Q2TP_GPU=1 CMF_GPU_PROBE=0 CMF_MTP=0
export CMF_METAL_PREFILL_CHUNK=32

cortiq run Ternary-Bonsai-2-27B-Q2TP-Affine.cmf \
  --prompt 'Explain in two sentences why the sky looks blue.' \
  --greedy --seed 42 --no-think --max-tokens 128

Chunked Metal prefill processes prompt rows in batches; generation uses the native ternary kernel. The one-command-buffer experiment was not faster and is not required. The failed experimental Q8/Q8×2 and alternative packing kernels are not recommended runtime settings.

Measured results and limitations

These are engineering measurements of the accepted development implementations, not universal hardware guarantees or a claim that every release build was benchmarked.

The matched throughput test used 1,122 identical prompt IDs and 128 greedy output tokens, one resident model, one cold run and three warm runs. Decode timing covers 127 callback intervals, separately from prefill and model loading. The upstream reference uses its own internal timing boundary; small boundary differences remain.

Backend / hardware	Warm decode	Interpretation
CMF resident Vulkan, RTX 4090 24 GB	about 42 tok/s	Accepted CMF implementation
Official Prism CUDA reference, same RTX 4090	about 87 tok/s	Reference; CMF has not reached CUDA parity
CMF native Metal, Apple M4 10-core GPU / 24 GiB	7.50–8.74 tok/s, mean 7.94	Latest three warm samples
Earlier CMF Metal implementation, same host	about 5.23 tok/s	Historical, unpaired baseline; not a controlled speedup ratio

20 tok/s on this M4 was not achieved. Prompt processing and time-to-first-token remain separate costs; longer conversations, temperature/clock conditions and application-side sampling affect throughput. The 8.25 GB file size is not peak VRAM/RAM usage: activations, prefill scratch and KV/recurrent state require additional memory.

Quality correspondence

Frozen mixed English/Russian/code corpus: 1,122 token IDs, two independent reset slices [0,513) and [512,1025), scoring 512 next-token targets per slice (1,024 total), without a synthetic BOS.

Path	Mean NLL	Delta from official reference
Official Prism PQ2 CUDA	1.81274952826	—
CMF resident Vulkan	1.812987430	+0.000237902
CMF native Metal, serial decode	1.813012543852	+0.000263016
CMF native Metal, batched prefill	1.812964935385	+0.000215407

This small correspondence corpus is not GPQA, Terminal-Bench, AA-LCR, MMMU-Pro or a broad model-quality evaluation. Six deterministic text checks also covered arithmetic, structured JSON, factual context, English/Russian explanations and a Python function. The fixed throughput test retained identical output IDs after the accepted Metal change; that does not imply identical outputs for every prompt.

The separate ordinary, lossy CMF comparison was a mixed Q2TP/Q4TP candidate and failed the +0.05-NLL acceptance threshold. It is not the model distributed here. Plain Q8 activation experiments also failed the component-fidelity gate. A two-component CUDA activation experiment preserved fidelity but was not integrated as a production CUDA backend; no slice timing is presented as model throughput.

Scope

• Text-only runtime support. Preserved vision records do not imply working multimodal inference.
• The measurements above use ordinary attention, not an O(1)-attention quality certification for Bonsai.
• No promise of unlimited context, fixed total runtime memory, CUDA parity, broad benchmark scores or lossless source equivalence.
• Use 0.6.9, not the historical 0.6.8-labelled patched development binary.

Provenance and licensing

Upstream revision: 3f926b415992eaa2ae9dd7b573706494d6bbf787. Original safetensors SHA-256: 130de5925082c168b7866b2e91b52e44abbafc99017e3ca352b77b5b55a269ed.

See runtime provenance, the frozen inputs under evaluation/, and the retained license bundle. Preserve the upstream model LICENSE/NOTICE, MLX runtime notice and CMF notices when redistributing. Historical source patches remain for reproducibility of old measurements; they are not the installation method for 0.6.9.