README.md

---
license: apache-2.0
base_model: prism-ml/Bonsai-1.7B-unpacked
library_name: mlc-llm
pipeline_tag: text-generation
tags:
- mlc-llm
- web-llm
- webgpu
- qwen3
- bonsai
- prismml
- 1-bit
- quantized
- experimental
---

# Bonsai-1.7B `bonsai_q1_f32` for MLC/WebLLM

This repository contains an experimental MLC/WebLLM conversion of
[`prism-ml/Bonsai-1.7B-unpacked`](https://huggingface.co/prism-ml/Bonsai-1.7B-unpacked).
It is a browser-runtime artifact, not a new model, fine-tune, GGUF, MLX, or ONNX
mirror.

The weights use the local `bonsai_q1_f32` format: binary signs packed into
`uint32` words with one FP32 scale per 128-wide group. Linear layers,
embeddings, and the final lm head are stored in this format.

## Artifact Summary

| Field | Value |
| --- | --- |
| Source checkpoint | `prism-ml/Bonsai-1.7B-unpacked` |
| Architecture | Qwen3-shaped decoder |
| MLC model type | `qwen3` |
| Quantization | `bonsai_q1_f32` |
| Conversation template | `qwen3_nothink` |
| Context window in config | `32768` |
| Prefill chunk in config | `2048` |
| Artifact size | about 254 MB |
| WebGPU library | `libs/bonsai-q1-1.7b-bonsai_q1_f32-webgpu.wasm` |
| WebGPU wasm size | 7,864,665 bytes |

## Runtime Requirement

This artifact requires an MLC/WebLLM runtime with Bonsai q1 support. It is not
expected to load in an unmodified upstream WebLLM build until the Bonsai q1
runtime path is upstreamed.

Use this repository when you control the WebLLM runtime and want to test
browser-local Bonsai inference through WebGPU.

## WebLLM Configuration

```js
const appConfig = {
  model_list: [
    {
      model: "https://huggingface.co/welcoma/Bonsai-1.7B-bonsai_q1_f32-MLC/resolve/main/",
      model_id: "Bonsai-1.7B-q1-MLC",
      model_lib:
        "https://huggingface.co/welcoma/Bonsai-1.7B-bonsai_q1_f32-MLC/resolve/main/libs/bonsai-q1-1.7b-bonsai_q1_f32-webgpu.wasm",
      overrides: {
        context_window_size: 4096,
        prefill_chunk_size: 512,
      },
    },
  ],
};
```

The smaller override values above are intended for local browser smoke tests.
Increase them only after measuring browser memory and cache behavior on the
target device.

## Validation

The 1.7B pilot passed the following checks in the patched MLC/WebLLM workspace:

- ONNX payload parity probes for sampled q1 matmul blocks.
- TVM-free q1 reference checks for the packed layout.
- Conversion-time MLC weight-packing verification.
- Qwen3-shaped q1 graph export and WebGPU compilation.
- WebLLM/WebGPU smoke generation on a controlled browser runtime.

Smoke prompt:

```text
Say only: ready
```

Observed response:

```text
I'm ready.
```

## Limitations

- This is an experimental runtime artifact, not a general `transformers` model
  checkpoint.
- Quality evaluation is limited to conversion/runtime smoke checks; no benchmark
  score is claimed by this repository.
- Browser success depends on WebGPU support, available GPU memory, cache quota,
  and a compatible patched WebLLM runtime.
- The ternary Bonsai family is not represented by this q1 format. Ternary
  models need a separate 2-bit/ternary MLC path.

## Provenance

Original model by Prism ML:

- [prism-ml/Bonsai-1.7B-unpacked](https://huggingface.co/prism-ml/Bonsai-1.7B-unpacked)
- [prismml.com](https://prismml.com/)

MLC/WebLLM conversion by `welcoma`.