---
license: apache-2.0
base_model: Agnes-AI/Agnes-3.0-Flash
library_name: mlx
pipeline_tag: text-generation
tags:
- mlx
- qwen3_5
- agnes
- 8bit
- hybrid-attention
- gated-delta-net
- mtp
language:
- en
- zh
---

# Agnes-3.0-Flash — MLX 8-bit + MTP

8-bit MLX quantization of [Agnes-AI/Agnes-3.0-Flash](https://huggingface.co/Agnes-AI/Agnes-3.0-Flash) (Apache-2.0), grafted with Qwen3.8's vision tower and Agnes's native MTP head for enhanced inference.

- Affine 8-bit, group size 64 — 8.50 bits/weight, ~37.8GB (8 shards)
- 262,144-token context, thinking on/off via the original chat template
- MTP head grafted: Agnes native MTP head with FFN padded 17408→19456 (zero-pad, mathematically identical), quantized to 4-bit
- Vision tower grafted from Qwen3.8 (333 bf16 tensors, byte-identical to Agnes original)

## Architecture

Agnes-3.0-Flash is a 33B hybrid attention model:

- 54 recurrent (Gated DeltaNet) layers + 18 global attention layers
- GQA 6:1 (24q/4kv heads, dim 256), 3-axis RoPE, causal conv kernel 4
- Hidden size 5120, intermediate size 17408 (Agnes native)
- MTP head intermediate size 19456 (Qwen3.8 compatible, padded from 17408)

## What changed

Converted from the original Agnes format to standard Qwen3.5 architecture:

- Folded parallel FFN into main MLP via concatenation (intermediate_size: 19456)
- Renamed `delta_attn` → `linear_attn`, `global_attn` → `self_attn`
- Converted one-centered RMSNorm to standard format
- Cast bf16 → fp16 for serialization compatibility

MTP graft:

- Padded Agnes native MTP head FFN layers (gate_proj, up_proj: +2048 rows; down_proj: +2048 cols) to match Qwen3.8's expected 19456
- Quantized MTP weights to 4-bit affine (group_size=64)
- Grafted 333 vision tensors from Qwen3.8 (byte-identical to Agnes original)

## Benchmark

Measured on M3 Max 64GB, oMLX, temperature 0:

| Metric | Baseline (no MTP) | With MTP | Speedup |
|--------|-------------------|----------|---------|
| Throughput | 15.0 tok/s | 21.7 tok/s | 1.44× |
| Acceptance rate | — | 85.7% | — |
| Tokens per cycle | — | 2.46 | — |

## Usage

```bash
pip install mlx-lm
mlx_lm.generate --model hermitdave/Agnes-3.0-Flash-MLX-8bit-MTP --prompt "Hello" --max-tokens 200
```

Or with oMLX: place under `~/.omlx/models/hermitdave/` and enable `mtp_enabled` + `vlm_mtp_enabled` in model settings.

## Attribution

This conversion was produced by [Hermes Agent](https://hermes-agent.nousresearch.com) (Nous Research). MTP graft methodology: pad Agnes native MTP head FFN to Qwen3.8 size (19456), quantize to 4-bit, graft Qwen3.8 vision tower.