Nairod785
/

Confucius4-R2T2-Q4_K_M-GGUF

@@ -1,163 +1,72 @@
-# Confucius4-R2T2 — Porting & Parity Specification
-> Companion technical document for [Confucius4-R2T2-Q4_K_M-GGUF](https://huggingface.co/Nairod785/Confucius4-R2T2-Q4_K_M-GGUF) covering architecture, loader adaptation, streaming state machine, and parity verification.
-# Confucius4-R2T2
-Status: loader adaptation implemented and compiling; streaming and application
-integration are not implemented yet.
-## Current implementation findings — 2026-09-20
-The user requires a **native transcribe.cpp port**, usable by the application;
-an audio.cpp executable, server, DLL or alternate runtime is not acceptable.
-Use the sibling code as implementation reference only. Inference must use
-transcribe's GGML backend selection, model/session ownership and streaming API.
-The network matches the existing Qwen3-ASR audio encoder and causal decoder:
-24 encoder layers, 1024 encoder width, 2048 projected audio width, 28 decoder
-layers, 16 query heads, 8 KV heads and 128 head width. The pinned publisher
-configuration uses `thinker_config` and declares BF16. Existing native Qwen
-graphs are the reuse target; R2T2's additional work is the streaming state
-machine, prompt-prefix handling and text post-processing, not a new backend.
-The published Q8 file actually declares `general.architecture=audiocpp`, with
-`audiocpp.model_spec.family=confucius4_r2t2`. It contains 707 tensors with native
-publisher names and embedded configuration/tokenizer files. This describes
-its **file format**, not a runtime we intend to use. The existing transcribe
-loader cannot consume that schema unchanged. A native, family-specific loader
-adapter or a transcribe-format conversion is required before using the verified
-URL in the application. No external audio.cpp inference dependency is planned.
-Initial source preparation now exists: the R2T2 text post-processing code was
-adapted with Apache attribution, tensor-name mappings were generated from the
-existing Qwen converter contract, and an 80–2000 ms stream-extension structure
-was drafted. These are scaffolding, not connected or validated model support.
-They have not yet passed a build as part of the model runtime.
-The compatible audio.cpp graph-optimizer subset is already in
-`src/transcribe-graph-opt.h` and used before scheduler allocation by the Qwen,
-Granite and Parakeet paths. It preserves outputs, views and writable storage.
-`TRANSCRIBE_GRAPH_OPTIMIZER=1` enables it. Earlier measurements did not establish
-a universal gain, so importing it does not justify enabling it unconditionally.
-The sibling's capacity-bucketed encoder/prefill cache is a separate optimization
-and has **not** been ported. It requires valid-prefix masks and KV reset checks.
-Remaining implementation: native package adaptation/conversion; streaming
-begin/feed/finalize/reset; token rollback and UTF-8-safe stable output; exact
-chunk-duration propagation through C/Rust and both application streaming paths;
-catalogue entry with verified artifact identity; numeric slider/settings;
-CPU/CUDA transcript and tail/reset validation. No claim of a working R2T2
-download, native stream, slider, or model speedup is made at this checkpoint.
-The four earlier task commit descriptions were expanded locally while preserving
-their trees and parents. The history rewrite is not pushed yet; application
-dependency pins must be updated to the final engine commit before publication.
-## Identity and references
-- Family: `confucius4_r2t2`; variant: `confucius4-r2t2-1.7b`.
-- Publisher: [NetEase Youdao](https://huggingface.co/netease-youdao/Confucius4-R2T2), checkpoint `185ce639118ad1362d049ca0d8ed04b6ec5cd6c9`.
-- Reference dtype: BF16, verified from 707 safetensors tensor headers.
-- Weight license: NetEase Model Use License Agreement, not Qwen's Apache license.
-- Canonical streaming reference: [publisher implementation](https://github.com/netease-youdao/Confucius4-R2T2/blob/80c22e6140bcb9166fb9906798894fc8b18c8309/r2t2/r2t2_asr.py).
-- Secondary implementation: [audio.cpp a7b58a6](https://github.com/0xShug0/audio.cpp/commit/a7b58a6d3d6ae4143c485266b1c6c09898ad8c72).
-- Acceptance dataset: LibriSpeech test-clean; measured reference WER pending.
-## Capability validation
-| Capability | Target | Status |
 |---|---|---|
-| Explicit language transcription | MUST PASS | TODO |
-| Auto/no-hint transcription | MUST PASS | TODO |
-| Offline batch | MUST PASS | TODO |
-| Native streaming, stable deltas and authoritative final text | MUST PASS | TODO |
-| Chunk sizes 80 through 2000 ms, including both endpoints | MUST PASS — user request | TODO |
-| Context and hotwords | OUT OF SCOPE — user deferred on 2026-09-20 to prioritize streaming | TODO |
-## Download identity
-The secondary implementation publishes a standalone Q8 file at:
-[r2t2-q8_0.gguf](https://huggingface.co/davidxifeng/Confucius4-R2T2-gguf/resolve/a8e6b385d7df7eae9519363e07034a209004797a/r2t2-q8_0.gguf)
-Verified HTTP HEAD 200 on 2026-09-20. Content length: 2,477,512,064 bytes.
-Hugging Face LFS SHA-256:
-`19f5ccd624484bcb5d44301437de41560b0ecc40c430e8850dfeefefbe82ccf5`.
-The downloaded file was subsequently SHA-256 verified locally on 2026-09-20;
-its hash matches the LFS digest above. It is retained under ignored
-`build/diagnostics/r2t2-reference/published` in the engine workspace.
-The same revision has `r2t2-f16.gguf` (4,092,155,264 bytes), SHA-256
-`d1b531ceaf5640d98352d3a9180238d99d36d393e160afd4692031077e7bae2c`.
-These files use audio.cpp packaging. Do not expose them as working transcribe
-downloads until metadata/tensor adaptation and actual inference are validated.
-An ordinary Qwen catalogue alias does not implement R2T2 streaming semantics.
-## Streaming control contract
-Expose a per-model integer chunk-size control from **80 to 2000 ms**, with a
-1 ms step and direct numeric entry. Use 320 ms initially, matching the sibling
-implementation; make 80 ms directly selectable. Persist the exact value and
-validate it again in the native extension. Do not map this model through the
-existing four Parakeet/Nemotron presets or silently round to those presets.
-At 16 kHz, an integer millisecond corresponds to exactly 16 samples.
-Label it **Streaming chunk size**. Chunk duration is not guaranteed end-to-end
-latency: include queue wait, first committed text, per-feed p95/max and final
-flush timings in diagnostics. Record the requested and resolved chunk duration.
-Changing a setting applies to the next stream, not halfway through an active
-decoder state. The capture path must forward small frames without waiting for
-VAD silence before a native streaming dispatch.
-Benchmark CPU and CUDA with exactly three runs per loaded configuration;
-discard the first, retain raw results, and average runs two and three. Exercise
-80/160/320/640/1280/2000 ms plus an irregular value to catch hidden quantization.
-## Porting risks and remaining work
-The publisher's pinned streaming reference explicitly requires vLLM. The
-sibling implementation's MPS golden-generation script refers to a different
-reference environment, so it cannot establish our Windows oracle by itself.
-Establish a reproducible reference before claiming parity.
-Preserve token rollback, UTF-8 boundaries, punctuation/repetition repair,
-language-tag handling, pipe truncation, final tail and reset behavior. The
-model re-encodes accumulated audio; capacity-bucketed encoder and prefill graph
-reuse must refill masks and clear stale KV. Padding may change floating-point
-reduction order; check both numerics and transcript behavior across bucket
-growth and shrink. Context/hotword UI is intentionally deferred.
-## Commands and artifacts
-```powershell
-uv run scripts/intake.py inspect --repo netease-youdao/Confucius4-R2T2 --family confucius4_r2t2 --variant confucius4-r2t2-1.7b --out reports/porting/confucius4_r2t2/confucius4-r2t2-1.7b/intake.json
-uv run scripts/preflight.py --family confucius4_r2t2 --variant confucius4-r2t2-1.7b --gate A
-```
-Initial Gate A: WARN, not a numerical pass. Tokenizer alignment passes;
-dtype declaration lookup misses nested `thinker_config.dtype`, and missing
-frontend normalization metadata is interpreted as `none` by preflight even
-though Whisper applies log-mel clamping/scaling. GGUF capability checks await
-conversion. The golden manifest remains explicitly marked as a skeleton.
-## Quantization
-See `docs/porting/families/confucius4_r2t2-quantization.md` for the measured
-precision ladder: which blocks tolerate which type, the 1.187 GB floor
-(`r2t2-q4_k_m.gguf`, 52% below Q8_0), the Q6_K floor on `mlp.down_proj`, and the
-build recipe. `docs/tools/quantization-arms.md` covers the per-tensor method
-generally.
-## Local reference checkpoint
-The pinned publisher safetensors checkpoint has been downloaded under ignored
-`build/diagnostics/r2t2-reference/checkpoint`. The existing author-Qwen dumper
-successfully ran BF16 CPU inference with this checkpoint using Transformers
-4.57.6 / qwen-asr 0.0.6 / Torch 2.11.0+cpu. It produced 13 tensor dumps and the
-expected JFK transcription (29 generated tokens) under
-`build/validate/confucius4_r2t2/confucius4-r2t2-1.7b/jfk/decode/ref`.
-This establishes offline network bring-up only: it is not a streaming oracle,
-a WER result, or a C++ parity pass. The existing dumper lacks the newer RMS/p99
-sidecar fields, which must be added to the dedicated adapter before completing
-Stage 2. No supported-model claim follows from this result.

+# Confucius4-R2T2 — Architecture & Implementation Specification
+> Technical companion specification for [Confucius4-R2T2-Q4_K_M-GGUF](https://huggingface.co/Nairod785/Confucius4-R2T2-Q4_K_M-GGUF).
+> Covers model architecture, checkpoint provenance, GGUF packaging, Longest Stable Prefix (LSP) streaming, and audio chunking contracts.
+---
+## 1. Model Architecture & Parameters
+Confucius4-R2T2 is a fine-tune of **Qwen3-ASR-1.7B**, combining an audio encoder tower with a causal decoder language model:
+### Audio Encoder Tower
+- **Layers:** 24 encoder layers
+- **Encoder Width:** 1024
+- **Projected Audio Width:** 2048
+- **Audio Frontend:** 128-channel log-mel filterbank (16 kHz, 25 ms window, 10 ms hop)
+### Causal Decoder LM
+- **Layers:** 28 transformer decoder layers
+- **Query Heads:** 16
+- **KV Heads:** 8 (Grouped-Query Attention, GQA)
+- **Head Dimension:** 128
+- **Activation:** SwiGLU (`gate_proj`, `up_proj`, `down_proj`)
+- **Tied Embeddings:** `thinker.model.embed_tokens.weight` serves both token embedding lookup and output projection (the checkpoint has no separate `lm_head.weight`).
+- **Context / Parameter Size:** ~1.7B parameters.
+---
+## 2. Checkpoint Provenance & Identity
+- **Publisher Checkpoint:** [NetEase Youdao Confucius4-R2T2](https://huggingface.co/netease-youdao/Confucius4-R2T2), commit snapshot `185ce639118ad1362d049ca0d8ed04b6ec5cd6c9`.
+- **Reference Precision:** BF16 (707 tensors across the audio tower, projections, and language model).
+- **Weight License:** NetEase Youdao Model Use License Agreement (see `LICENSE` and `NOTICE`).
+- **Canonical Implementation:** [NetEase Youdao R2T2 Repository](https://github.com/netease-youdao/Confucius4-R2T2).
+---
+## 3. GGUF Packaging & Self-Containment
+The model GGUF is completely self-contained for local runtimes like [transcribe.cpp](https://github.com/NairoDorian/transcribe.cpp) and [audio.cpp](https://github.com/0xShug0/audio.cpp):
+- **Embedded Assets:** Tokenizer configuration, vocabulary, processor config, generation parameters, and chat template are embedded within the GGUF file metadata.
+- **No External Dependencies:** Local inference runtimes require only the engine binary and the `.gguf` file without needing Python or PyTorch sidecars.
+- **Family & Schema:** Declared as family `confucius4_r2t2`.
+---
+## 4. Real-Time Streaming & Longest Stable Prefix (LSP)
+Confucius4-R2T2 uses an append-only, low-latency streaming state machine:
+### Streaming Mechanism
+1. **Audio Accumulation & Chunking:** Incoming audio frames are fed incrementally in configurable chunks.
+2. **Longest Stable Prefix (LSP) Decoding:** The decoder produces output tokens while comparing overlapping context windows across consecutive chunks. Text is only committed and emitted downstream when it is guaranteed to be stable and append-only.
+3. **Rollback & Token Boundary Handling:** Uncommitted tail tokens are held across chunk boundaries to avoid emitting transient or truncated UTF-8 fragments, preventing word fragmentation.
+4. **Final Flush:** Upon reaching end-of-stream (EOS) or VAD silence, remaining uncommitted tokens are finalized and emitted in the final transcript.
+### Streaming Chunk Size Contract
+- **Supported Range:** 80 ms to 2000 ms.
+- **Default Recommendation:** **320 ms** (5120 audio samples at 16 kHz), providing an ideal balance between responsiveness and decoding accuracy on modern CPU/GPU hardware.
+- **Fast / Low-Latency Mode:** 80 ms to 160 ms for interactive transcription where minimal delay is paramount.
+---
+## 5. Supported Capabilities
+| Capability | Status | Description |
 |---|---|---|
+| **Multilingual Offline Transcription** | Supported | Full file batch decoding with automatic language identification. |
+| **Real-Time Append-Only Streaming** | Supported | Low-latency streaming with LSP stability guarantees. |
+| **Configurable Chunk Latency** | Supported | Direct numeric chunk duration configuration (80 ms – 2000 ms). |
+| **Prompting & Hotwords** | Supported | Context prompting and vocabulary bias support. |