---
library_name: cortiq
license: apache-2.0
base_model: prism-ml/Bonsai-8B
tags:
- cortiq
- cmf
- 2-bit
- 1-bit
- quantized
- text-generation
pipeline_tag: text-generation
---

# Bonsai-8B — CMF q1t (2.32 GB, runs anywhere)

[Bonsai-8B](https://huggingface.co/prism-ml/Bonsai-8B) — packed into a single
[CMF](https://github.com/infosave2007/cmf) file with **q1t** encoding (ternary 1-bit/2-bit representation).

Because the model was *trained* for extreme sparsity/low-bit representation, q1t is its native
representation: generation is **token-for-token identical** to the same model stored at higher precision. The whole
model is one 2.32 GB file that memory-maps straight off disk and
decodes at **~20 tok/s on an Apple M-series MacBook** through a
whole-token Metal graph.

## What is CMF?

CMF (Cortiq Model Format) is a single-file LLM container with a small
pure-Rust runtime — no Python, no torch, no C++ toolchain, no CUDA
install:

- **One file** carries the weights, tokenizer and chat template, and
  checks its own integrity.
- **mmap-first**: pages load lazily on first touch; start-up is fast
  and RAM stays near the file size.
- **Per-tensor quantization** (q8 / q4 / vbit / **q1t** for
  1-bit/2-bit trained models).
- **O(1) attention option** (`--o1`): convert attention to a
  constant-memory streaming operator — no retraining, weights
  byte-identical.
- **Skills**: one file can carry a swarm of specialists sharing a base
  model, with self-routing. See the
  [skills guide](https://github.com/infosave2007/cmf/blob/master/docs/SKILLS.md).

Format spec and engine: https://github.com/infosave2007/cmf
Mobile app (Flutter, runs CMF on-device): https://github.com/infosave2007/cmfmobile

## Install

Prebuilt binaries (macOS arm64/x86_64, Linux, Windows):
https://github.com/infosave2007/cmf/releases/latest

Or with a Rust toolchain:

```sh
cargo install cortiq-cli   # needs >= 0.5.5
```

## Download & run

```sh
# download the model file (~2.32 GB)
huggingface-cli download infosave/Bonsai-8B_2bit_cmf bonsai-8b-q1t.cmf --local-dir .

# chat (the file carries its own chat template)
cortiq run bonsai-8b-q1t.cmf --prompt "Привет"

# Apple Silicon: run the whole-token Metal graph (~20 tok/s)
CMF_GPU=1 cortiq run bonsai-8b-q1t.cmf --prompt "Привет"

# raw completion mode (no chat template)
cortiq run bonsai-8b-q1t.cmf --prompt "The capital of France is" --raw --greedy
```

## Run as a server

`cortiq serve` speaks the OpenAI API, so existing clients and SDKs work
unchanged:

```sh
CMF_GPU=1 cortiq serve bonsai-8b-q1t.cmf --port 8080   # + web dashboard on /
```

```sh
curl localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "cmf",
  "messages": [{"role": "user", "content": "Explain mmap in one sentence."}]
}'
```

## Performance (Apple M-series MacBook)

| Mode | Speed |
|---|---:|
| decode, `CMF_GPU=1` (whole-token Metal graph) | **~20.2 tok/s** |
| decode, CPU only | ~4.2 tok/s |
| resident memory | ≈ file size (mmap) |

## Provenance

Converted from prism-ml/Bonsai-8B (Apache-2.0) with:

```sh
cortiq convert --model prism-ml/Bonsai-8B --quant q1t --output bonsai-8b-q1t.cmf
```