Bonsai-1.7B `bonsai_q1_f32` for MLC/WebLLM

This repository contains an experimental MLC/WebLLM conversion of prism-ml/Bonsai-1.7B-unpacked. It is a browser-runtime artifact, not a new model, fine-tune, GGUF, MLX, or ONNX mirror.

The weights use the local bonsai_q1_f32 format: binary signs packed into uint32 words with one FP32 scale per 128-wide group. Linear layers, embeddings, and the final lm head are stored in this format.

Artifact Summary

Field	Value
Source checkpoint	`prism-ml/Bonsai-1.7B-unpacked`
Architecture	Qwen3-shaped decoder
MLC model type	`qwen3`
Quantization	`bonsai_q1_f32`
Conversation template	`qwen3_nothink`
Context window in config	`32768`
Prefill chunk in config	`2048`
Artifact size	about 254 MB
WebGPU library	`libs/bonsai-q1-1.7b-bonsai_q1_f32-webgpu.wasm`
WebGPU wasm size	7,864,665 bytes

Runtime Requirement

This artifact requires an MLC/WebLLM runtime with Bonsai q1 support. It is not expected to load in an unmodified upstream WebLLM build until the Bonsai q1 runtime path is upstreamed.

Use this repository when you control the WebLLM runtime and want to test browser-local Bonsai inference through WebGPU.

WebLLM Configuration

const appConfig = {
  model_list: [
    {
      model: "https://huggingface.co/welcoma/Bonsai-1.7B-bonsai_q1_f32-MLC/resolve/main/",
      model_id: "Bonsai-1.7B-q1-MLC",
      model_lib:
        "https://huggingface.co/welcoma/Bonsai-1.7B-bonsai_q1_f32-MLC/resolve/main/libs/bonsai-q1-1.7b-bonsai_q1_f32-webgpu.wasm",
      overrides: {
        context_window_size: 4096,
        prefill_chunk_size: 512,
      },
    },
  ],
};

The smaller override values above are intended for local browser smoke tests. Increase them only after measuring browser memory and cache behavior on the target device.

Validation

The 1.7B pilot passed the following checks in the patched MLC/WebLLM workspace:

ONNX payload parity probes for sampled q1 matmul blocks.
TVM-free q1 reference checks for the packed layout.
Conversion-time MLC weight-packing verification.
Qwen3-shaped q1 graph export and WebGPU compilation.
WebLLM/WebGPU smoke generation on a controlled browser runtime.

Smoke prompt:

Say only: ready

Observed response:

I'm ready.

Limitations

This is an experimental runtime artifact, not a general transformers model checkpoint.
Quality evaluation is limited to conversion/runtime smoke checks; no benchmark score is claimed by this repository.
Browser success depends on WebGPU support, available GPU memory, cache quota, and a compatible patched WebLLM runtime.
The ternary Bonsai family is not represented by this q1 format. Ternary models need a separate 2-bit/ternary MLC path.

Provenance

Original model by Prism ML:

MLC/WebLLM conversion by welcoma.

Downloads last month: 4

Model tree for welcoma/Bonsai-1.7B-bonsai_q1_f32-MLC

Base model

prism-ml/Bonsai-1.7B-unpacked

Quantized

(11)

this model

Bonsai-1.7B bonsai_q1_f32 for MLC/WebLLM