--- license: apache-2.0 library_name: mlx pipeline_tag: image-text-to-text base_model: dots-studio/dots3-note-prev tags: - jang - mlx - apple-silicon - dots3 - multimodal - moe - quantized extra_gated_prompt: "Access is manually reviewed. This is a preview-weights build with an incomplete runtime story — please read the Runtime status section before requesting." extra_gated_fields: Name: text Intended use: text Hardware: text --- # dots3-note-preview — JANG (94.57 GiB, Apple Silicon / MLX) > ## ⚠️ RUNTIME NOT READY YET — ships with vMLX 0.24 > **This bundle does not run on any released runtime today.** dots3-note is a > new architecture (dual-geometry MLA + DSA indexer + 256-expert MoE + MoE-ViT > + audio tower) and support lands in **vMLX 0.24**. Download and stage it now > if you like, but **wait for 0.24 before expecting it to load.** > See [Runtime status](#runtime-status) for exactly what works and what does not. A measured mixed-precision JANG build of [`dots-studio/dots3-note-prev`](https://huggingface.co/dots-studio/dots3-note-prev) — 280 B total / 16 B active omni-modal MoE — compressed to **94.57 GiB** so it fits a 128 GB Apple Silicon machine with room for context and KV cache. Text, image, video and audio in; text out. Reasoning on by default, native tool calling, 512 K context ceiling (see runtime caveats). --- ## Quality — measured, not claimed Every number below was produced by this exact artifact against the FP8 source checkpoint on a held-out evaluation slice that was excluded from all calibration. | metric | this bundle | reference point | |---|---|---| | **mean KL vs source** | **0.346 nats** | 3.096 = structurally broken | | p90 KL | 0.803 | — | | **top-1 agreement vs source** | **79.2 %** | 0.015 % = structurally broken | | live acceptance rows | **4 / 4 PASS** | — | | modality probes | **3 / 3 PASS** | — | The "structurally broken" column is a deliberately degraded control (a 4-layer truncation of the same model) measured through the same harness, so the headline numbers have a floor to be judged against rather than sounding good in isolation. **Live rows** — exact-string echo with clean EOS; step-by-step arithmetic inside ``; a coding row that restates both parameter names verbatim; and a well-formed native tool call. Zero degenerate repetition (every-period scan, periods 1–16). **Modality probes** — the model identifies shapes and colours in an image, reports the correct direction of motion in an 8-frame clip (*"located on the left side… has shifted slightly to the right"*), and describes a synthesised tone correctly. These run the full deployed path: pixels/audio → encoder → placeholder substitution → quantised LM → text. ## Size and composition | | | |---|---| | bundle | **94.57 GiB** (101.54 GB, 50 shards) | | source (BF16) | 576.9 GB → **6.1× smaller** | | source (FP8, the build input) | 278 GiB → 2.9× smaller | | total / active params | 280 B / 16 B | Bits are allocated **by measurement**, not by name-matching rules: every routed expert unit was individually evaluated and the allocation solved under a byte budget. Attention, the shared experts, embeddings and the untied head are kept at 8-bit; routed experts run at a 2-bit class with measured per-unit lifts; the vision and audio towers carry their own tiers. Weights are stock MLX quantised-matmul storage — no custom kernels, no runtime sidecar, instant `mmap` load. Calibration used activation-aware fitting throughout (AWQ-style scaling, per-expert importance weighting, and error-compensated code optimisation). Method details are intentionally not published. ## Runtime status **Requires vMLX 0.24.** This architecture is new and needs a purpose-built runtime. Being explicit about what exists today, because a model card that hides gaps wastes your time: | capability | status | |---|---| | text generation, reasoning, tool calls | ✅ verified | | image / video / audio understanding | ✅ verified end-to-end | | MTP weights preserved (1 layer, 41 tensors) | ✅ shipped, `runtime_available: false` | | **long context (> 2 K tokens)** | ❌ **needs MLA latent caching + DSA indexer** | | **speculative decode via MTP** | ❌ no decode loop yet | | **published tokens/sec** | ❌ **not quoted — see below** | | Swift / Osaurus runtime | ❌ not yet | | `mlx_lm` tool-call parsing | ❌ no parser exists for this dialect | **On speed:** we will not publish a tok/s figure measured on a verification harness. The bundle reads ~11 GB of weights per token, which is bandwidth-plausible for roughly 35 tok/s on an M5 Max, but the harness used for correctness lacks MLA absorption and fused decode. A real number will be published once vMLX 0.24 lands. Treat any tok/s claim before then as unfounded. **On long context:** the architecture is unusually cache-friendly — only 13 of 46 layers are full-attention and the rest are window-limited, so the entire 512 K KV cache is ~7.9 GB in latent form. That is a runtime feature, not a weights feature, and it is not implemented yet. ## Sampling Vendor-documented defaults, plus a coding preset. `generation_config.json` and `jang_config.chat.sampling_defaults` are stamped to agree. | mode | temperature | top_p | top_k | source | |---|---|---|---|---| | **thinking_general** *(default)* | 1.0 | 0.95 | 0 | vendor README | | **agentic_coding** | **0.6** | 0.95 | 0 | JANG preset | | instruct_nothinking | 0.7 | 0.95 | 0 | vLLM recipe example | `top_k = 0` (disabled) is deliberate — the vendor specifies only temperature and top_p. The coding preset is ours, not the vendor's: 0.6 measurably helps exact-identifier fidelity, which is what matters for agentic work. Greedy decoding with thinking enabled is not recommended for this family. ## Reasoning, tools, modalities **Reasoning is ON by default** — the chat template enables thinking when the kwarg is absent. Disabling it is a *prefill*, not an omission: it appends a marker to the user turn **and** prefills a closed think block. Doing only one gives inconsistent behaviour. There are no reasoning-effort tiers. **Tools** use the native dots XML dialect (`` / `` / ``), served upstream by the `dots` parser — not JSON, not Hermes. Note that `mlx_lm` has no parser for it, so tool calls surface as plain text until one exists. **Modalities** are weight-gated in `jang_config.capabilities` (derived from the tensors actually present, not from config claims): vision, video and audio all true. One implementation note that will save you a debugging session: **video is expanded onto the image token path** by the processor, so derive token ids and media tensors from a single processor call and trust the encoding rather than the config's video token id. **EOS is two ids** — `[151643, 151668]`. Stopping on only the first runs past the end of every assistant turn. ## Provenance Built from `dots-studio/dots3-note-prev-fp8` (the vendor's own serving checkpoint). FP8 → BF16 dequantisation was verified bit-identical to the reference implementation, and the MLX forward pass was verified per-layer against the upstream reference at both short and long sequence lengths before any quantisation was performed. Honest limitation: because the build input is the FP8 checkpoint, the 8-bit tier inherits FP8 fidelity (~0.027 relative error vs BF16) rather than true 8-bit fidelity (~0.006). This is immaterial for the 2-bit-class expert tiers that dominate the bundle, but it does mean attention quality is source-limited. The vision and audio towers are BF16 in the source and are unaffected. ## License and credits Apache 2.0, inherited from the base model. Base model © Xiaohongshu / dots studio. Quantisation, calibration and packaging by **Jinho Jang** — .