---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: audio-text-to-text
# base_model: Qwen/Qwen3-Omni-30B-A3B-Thinking
tags:
- audio
- speech
- multimodal
- audio-text-to-text
- reasoning
- safetensors
model_name: BR-Voice-Reasoner
---

# BR-Voice-Reasoner

BR-Voice-Reasoner is a multimodal reasoning model for spoken interaction. It
understands spoken requests and responds with knowledge, reasoning, and
instruction-aware text. It operates directly on audio, allowing applications
to reason over what was said without inserting a separate transcription model
into the pipeline.

The model is designed for spoken question answering, knowledge-intensive voice
queries, multi-step reasoning, instruction following, open-ended interaction,
and spoken interaction involving safety and refusal behavior.

## Model overview

| Item | Value |
|---|---|
| Input | Audio, text, image, and video |
| Output | Text |
| Architecture | Multimodal Mixture-of-Experts |
| Thinker LM | 30B total / 3B activated MoE |
| Primary evaluation language | English |
| Weights | BF16 Safetensors |

Post-training primarily targets speech-conditioned interaction; image and video
capabilities are inherited from the base model and were not comprehensively
re-evaluated.

## Training

BR-Voice-Reasoner is trained from Qwen3-Omni-30B-A3B-Thinking using
cross-modal on-policy distillation. During training, the student processes
spoken requests, while a frozen teacher uses their aligned text forms to provide
token-level learning signals along student-generated trajectories.

Optimization is restricted to two audio-language projection modules. The
language model, vision encoder, and remaining audio encoder parameters stay
frozen, focusing post-training on the audio-language interface used to access
the model's existing knowledge, reasoning, and instruction-following
capabilities.

### Training data

Training data cover spoken knowledge, reasoning, instruction following,
open-ended interaction, and safety-oriented tasks using both real-world and
synthesized speech.

## Evaluation

BR-Voice-Reasoner is evaluated on nine VoiceBench subsets covering knowledge,
reasoning, instruction following, safety, and open-ended spoken interaction.
All scores are reported on a 0–100 scale; higher is better.

<table align="center">
  <thead>
    <tr>
      <th align="left">VoiceBench subset</th>
      <th><div align="center">Qwen3-Omni<br>30B-A3B-Thinking</div></th>
      <th><div align="center">Nemotron 3 Nano Omni</div></th>
      <th><div align="center">BR-Voice-Reasoner</div></th>
    </tr>
  </thead>
  <tbody>
    <tr><td align="left">IFEval</td><td><div align="center">80.6</div></td><td><div align="center"><strong>88.7</strong></div></td><td><div align="center">83.2</div></td></tr>
    <tr><td align="left">BBH</td><td><div align="center">88.9</div></td><td><div align="center"><strong>91.1</strong></div></td><td><div align="center">90.4</div></td></tr>
    <tr><td align="left">AdvBench</td><td><div align="center">97.2</div></td><td><div align="center"><strong>100.0</strong></div></td><td><div align="center">99.8</div></td></tr>
    <tr><td align="left">AlpacaEval</td><td><div align="center">96.4</div></td><td><div align="center">95.0</div></td><td><div align="center"><strong>97.3</strong></div></td></tr>
    <tr><td align="left">CommonEval</td><td><div align="center">90.5</div></td><td><div align="center">91.3</div></td><td><div align="center"><strong>94.2</strong></div></td></tr>
    <tr><td align="left">WildVoice</td><td><div align="center">90.5</div></td><td><div align="center">91.7</div></td><td><div align="center"><strong>93.2</strong></div></td></tr>
    <tr><td align="left">OpenBookQA</td><td><div align="center">94.3</div></td><td><div align="center">93.0</div></td><td><div align="center"><strong>96.0</strong></div></td></tr>
    <tr><td align="left">MMSU</td><td><div align="center">83.0</div></td><td><div align="center">82.3</div></td><td><div align="center"><strong>85.7</strong></div></td></tr>
    <tr><td align="left">SD-QA</td><td><div align="center"><strong>78.1</strong></div></td><td><div align="center">71.4</div></td><td><div align="center">74.3</div></td></tr>
    <tr><td align="left"><strong>VoiceBench Avg</strong></td><td><div align="center">88.8</div></td><td><div align="center">89.4</div></td><td><div align="center"><strong>90.5</strong></div></td></tr>
  </tbody>
</table>

Qwen3-Omni results are taken from the
[official Qwen3-Omni model card](https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Thinking),
while Nemotron 3 Nano Omni results are taken from its
[official model card](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16). BR-Voice-Reasoner results
were obtained using the evaluation protocol described below. BR-Voice-Reasoner
values are rounded to the same one-decimal format. The external columns and
BR-Voice-Reasoner were not produced by a single shared evaluation run;
cross-column differences are therefore reported as references rather than as a
strictly controlled comparison.

### BR-Voice-Reasoner evaluation protocol

| Item | Setting |
|---|---|
| Reasoning | Enabled |
| Generation | temperature 0.6, top-p 0.95, top-k 20 |
| Model judge | GPT-4o-mini, three judgments |
| Overall | Mean of nine normalized subset scores |

AlpacaEval, CommonEval, and WildVoice ratings are normalized to a 0–100 scale
before aggregation. The VoiceBench average is the arithmetic mean of the nine
normalized subset scores.

Full generation, seeding, evaluator, and retry settings are provided in
[`evaluation_protocol.json`](evaluation_protocol.json).

## Installation

```bash
pip install "transformers==5.12.1" "accelerate" "qwen-omni-utils==0.0.9"
```

`ffmpeg` must also be available on the system for media loading. FlashAttention
2 is optional; install it separately with
`pip install flash-attn --no-build-isolation` on compatible hardware.

## Usage

```python
from transformers import (
    Qwen3OmniMoeForConditionalGeneration,
    Qwen3OmniMoeProcessor,
)
from qwen_omni_utils import process_mm_info

MODEL_ID = "brgroup/BR-Voice-Reasoner"

model = Qwen3OmniMoeForConditionalGeneration.from_pretrained(
    MODEL_ID,
    dtype="auto",
    device_map="auto",
    # attn_implementation="flash_attention_2",  # optional
)
processor = Qwen3OmniMoeProcessor.from_pretrained(MODEL_ID)

conversation = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": "example.wav"},
            {"type": "text", "text": "Answer the question in the audio."},
        ],
    }
]

prompt = processor.apply_chat_template(
    conversation,
    add_generation_prompt=True,
    tokenize=False,
)
audios, images, videos = process_mm_info(conversation)
inputs = processor(
    text=prompt,
    audio=audios,
    images=images,
    videos=videos,
    return_tensors="pt",
    padding=True,
)
inputs = inputs.to(model.device).to(model.dtype)

text_ids, _ = model.generate(
    **inputs,
    return_audio=False,
    thinker_return_dict_in_generate=True,
    max_new_tokens=2048,
)
response = processor.batch_decode(
    text_ids.sequences[:, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)
print(response[0])
```

## Limitations

- Primary training and evaluation are English-focused; multilingual, visual,
  and video capabilities inherited from the base model were not comprehensively
  re-evaluated.
- Performance may vary across accents, recording conditions, noise levels,
  domains, and prompting styles.
- The model may mishear speech, hallucinate, reason incorrectly, or produce
  unsafe or biased content.
- Model-based evaluation results may vary with evaluator versions and inference
  settings.

## License and attribution

BR-Voice-Reasoner is released under the Apache License 2.0 and is derived from
[Qwen3-Omni-30B-A3B-Thinking](https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Thinking),
whose original model copyright is `Copyright 2025 Alibaba Cloud`.
Modifications are `Copyright 2026 Bairong Inc.` See [`LICENSE`](LICENSE) and
[`NOTICE`](NOTICE) for license terms and attribution.