---
base_model: netease-youdao/Confucius4-R2T2
library_name: transcribe.cpp
pipeline_tag: automatic-speech-recognition
tags:
  - audio
  - speech-recognition
  - asr
  - gguf
  - quantized
  - q4_k_m
license: other
license_name: netease-model-use-license
license_link: https://huggingface.co/netease-youdao/Confucius4-R2T2
language:
  - zh
  - en
  - de
  - fr
  - ru
---

# Confucius4-R2T2 — Q4_K_M GGUF (Arm M)

Optimized **Q4_K_M GGUF** quantization of [netease-youdao/Confucius4-R2T2](https://huggingface.co/netease-youdao/Confucius4-R2T2), designed for native execution in [transcribe.cpp](https://github.com/NairoDorian/transcribe.cpp).

- **Filename:** `r2t2-q4_k_m.gguf`
- **File size:** **1.187 GB** (1,186,939,968 bytes)
- **RAM/VRAM footprint:** **52% smaller** than the 2.478 GB Q8_0 reference.
- **Base Architecture:** Qwen3-ASR audio encoder (24 layers, 1024 width, 2048 projected width) + Causal LM decoder (28 layers, 16 query heads, 8 KV heads, 128 head width).
- **Base Checkpoint:** `185ce639118ad1362d049ca0d8ed04b6ec5cd6c9`

---

## Quantization Summary: Arm M

Uniform quantization breaks multilingual stability on R2T2. Arm M uses an **asymmetric per-tensor quantization recipe** that reaches the minimum empirical memory floor while preventing language drift and degradation:

| Component | Tensors | Target Dtype | Architectural Rationale |
|---|---|---|---|
| **Audio Tower** | 147 | `BF16 / Q4_K` | Audio encoder layers quantized to Q4_K while keeping converter-selected projection weights at BF16. |
| **Attention Projections** | 112 (`q`, `k`, `v`, `o_proj`) | `Q4_K` | Quantized within the LM error budget when paired with Q6_K down projections. |
| **MLP Gate / Up** | 56 (`gate_proj`, `up_proj`) | `Q4_K` | Errors pass through SwiGLU non-linearities, bounding error propagation. |
| **MLP Down** | 28 (`down_proj`) | **`Q6_K` (Floor)** | **Non-negotiable hard floor.** `down_proj` writes straight into the residual stream, compounding errors across all subsequent layers. Quantizing below Q6_K triggers severe English language drift on non-English audio. |
| **Embeddings** | 1 (`embed_tokens`) | **`Q2_K`** | Lookup table error enters once and does not compound. Slices 311M parameters down with no measurable fidelity penalty. |

---

## Why Arm M Works

### 1. The Error Budget is Cumulative
A transformer block cannot be quantized in isolation; its tolerance depends on what the rest of the model has spent:
- **Da4** (Attention @ Q4_K, `down_proj` @ Q6_K) passes.
- **E** (Attention @ Q6_K, `down_proj` @ Q4_K) passes simple probes.
- **G** (Both Attention & `down_proj` @ Q4_K) **fails completely**.

The LM decoder can afford **exactly one** of {attention, `down_proj`} at Q4_K. Arm M allocates that budget to attention while pinning `down_proj` strictly to Q6_K.

### 2. `down_proj` @ Q6_K is the Hard Floor
Every arm evaluated that placed `down_proj` at Q5_K or Q4_K (D5, E, L, N, K, G, G3) emitted English when given German audio. No arm with `down_proj` below Q6_K passes multilingual validation.

### 3. Embeddings are the Cheapest Lever
`embed_tokens` is 311M parameters. Quantizing it to **Q2_K** drops the total model size from 1.26 GB (Da4) to **1.187 GB** without triggering degradation.

### 4. Low-Bit Matmul Cliff
Matmuls below Q4_K exhibit a sharp failure cliff rather than gradual degradation:
- Tower @ Q3_K (`r2t2-Dt3`) -> **empty output**.
- Gate/Up @ Q3_K (`r2t2-P`) -> **empty output**.

---

## Arm Evaluation & Benchmarks

Measured on CPU (pooled ratio against Q8_0 baseline on JFK and ZH test audio):

| Arm | Attention | Gate/Up | `down_proj` | Embed | Tower | Size (GB) | vs Q8_0 Speed | Status |
|---|---|---|---|---|---|---|---|---|
| `r2t2-q8_0` | Q8_0 | Q8_0 | Q8_0 | F16 | BF16/Q8_0 | 2.478 | 0% (baseline) | Reference |
| `r2t2-q6_k` | Q6_K | Q6_K | Q6_K | F16 | BF16/Q6_K | 2.060 | +5% | Passes |
| `r2t2-q5_k` | Q5_K | Q5_K | Q5_K | F16 | BF16/Q5_K | 1.832 | **-20% (slower)** | Fails Russian (flips to English) |
| `r2t2-Da4` | Q4_K | Q4_K | Q6_K | Q4_K | BF16/Q4_K | 1.260 | — | Passes |
| **`r2t2-q4_k_m` (M)** | **Q4_K** | **Q4_K** | **Q6_K** | **Q2_K** | **BF16/Q4_K** | **1.187** | **+20%+** | **Floor — Passes All** |
| `r2t2-D5` | Q6_K | Q4_K | **Q5_K** | Q4_K | BF16/Q4_K | 1.304 | +18% | Fails German |
| `r2t2-E` | Q6_K | Q4_K | **Q4_K** | Q4_K | BF16/Q4_K | 1.260 | +22% | Fails German |
| `r2t2-G` | Q4_K | Q4_K | **Q4_K** | Q4_K | BF16/Q4_K | 1.169 | — | Fails German |
| `r2t2-P` | Q4_K | **Q3_K** | Q6_K | Q2_K | BF16/Q4_K | 1.093 | — | Empty output cliff |

*Note: Q5_K is strictly dominated — it runs ~20% slower than Q8_0 due to non-vectorized kernel routines and fails multilingual validation. Arm M provides both higher throughput and minimal size.*

---

## Validation Protocol

- **Probing:** Tested on English (`jfk`), Chinese (`zh`), German (`samples/german.wav`), and native French audio sets.
- **Detector:** Evaluated via `detected-language:` outputs and strict word-level diffing against the Q8_0 reference. Arm M matches Q8_0 transcriptions word-for-word without language switching.

---

## Usage

### With `transcribe-cli`
```bash
transcribe-cli -m r2t2-q4_k_m.gguf samples/audio.wav
```

### Streaming
R2T2 supports configurable streaming chunk durations between 80 ms and 2000 ms (default: 320 ms).

---

## Provenance and License

- **Original Model:** [NetEase Youdao Confucius4-R2T2](https://huggingface.co/netease-youdao/Confucius4-R2T2)
- **License:** Model weights are distributed under the [NetEase Model Use License Agreement](https://huggingface.co/netease-youdao/Confucius4-R2T2).