README.md

---
language:
- en
- zh
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
tags:
- agnes-ai
- reasoning
- multimodal
- long-context
- hybrid-attention
---
<p align="center">
  <img width="132" src="assets/agnes_logo.svg" alt="Agnes AI logo">
</p>

<p align="center">
  <a href="https://agnes-ai.com/"><img src="https://img.shields.io/badge/Agnes_AI-Website-3248AF" alt="Agnes AI website"></a>
  <a href="#quickstart"><img src="https://img.shields.io/badge/Agnes--3.0--Flash_Preview-Open_Weights-3248AF" alt="Open weights"></a>
  <img src="https://img.shields.io/badge/License-Apache_2.0-111827" alt="Apache 2.0">
</p>



# Agnes-3.0-Flash Preview


## Model version clarification
This repository contains an earlier open-weight **Preview checkpoint** of Agnes 3.0 Flash. It is distinct from the newer **production/API checkpoint** listed on [Artificial Analysis](https://artificialanalysis.ai/models/agnes-3-0-flash).
The Preview release has **33B parameters** and a context window of **262,144 tokens**. The production/API model uses a different checkpoint and configuration, with a **1M-token context window**. Its benchmark results should not be attributed to the Preview weights released here.
This repository was initially published as `Agnes-3.0-Flash` without the `Preview` suffix. The model card now explicitly identifies this release as **Agnes-3.0-Flash Preview** to clarify the distinction between the open-weight release and the production/API model.
The specifications and Agnes benchmark results below refer to the **Preview checkpoint**.
Hello! 👋 Today we are introducing **Agnes-3.0-Flash Preview**, an **open-weights multimodal preview model** built for people who want flagship-class reasoning without flagship-class hardware.
Highlights:
- **Competitive across core capabilities.** Agnes-3.0-Flash Preview posts competitive results across reasoning, coding, and instruction-following evaluations.
- **Built for demanding work.** A **262 144-token context window**, adjustable reasoning effort, tool calling, and **text, image and video** understanding.


### Benchmarks
> **Benchmark scope:** The Agnes results in the chart and table below belong to the **Agnes-3.0-Flash Preview open-weight checkpoint** released in this repository. They are not results for the production/API Agnes 3.0 Flash model listed on Artificial Analysis.
<p align="center">
  <img style="width:100%;max-width:1100px" src="assets/benchmark-preview.png" alt="Agnes-3.0-Flash Preview benchmark reference results">
</p>


The Agnes-3.0-Flash Preview scores in the chart correspond to the open-weight checkpoint released in this repository.
Reference results across contemporary models are shown below. The figures were compiled from different sources, harnesses, and model snapshots and do not constitute a controlled head-to-head comparison.
<div style="font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;width:100%;margin:0 auto;padding:16px 0;overflow-x:auto">
<table style="width:100%;table-layout:fixed;border-collapse:collapse;font-size:11px;min-width:1180px">
<thead><tr>
<th style="width:15%;padding:9px 6px;text-align:left;border-bottom:2px solid #3248AF;color:#3248AF;font-size:11px">Benchmark</th>
<th style="width:10%;padding:9px 5px;text-align:center;font-weight:700;border-bottom:2px solid #3248AF;color:#3248AF;background:rgba(50,72,175,.09);font-size:10px;line-height:1.3">Agnes-3.0-Flash Preview</th>
<th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Qwen3.6-35B-A3B<br><span style="opacity:.65">35B / 3B active</span></th>
<th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Kimi K2.5<br><span style="opacity:.65">1T / 32B active</span></th>
<th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Muse Glimmer<br><span style="opacity:.65">30B</span></th>
<th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Qwen3.5<br><span style="opacity:.65">27B</span></th>
<th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">DeepSeek V4 Flash 0731<br><span style="opacity:.65">284B / 13B active</span></th>
<th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Qwen3.8<br><span style="opacity:.65">27B</span></th>
<th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Gemini 3.5 Flash<br><span style="opacity:.65">undisclosed</span></th>
<th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Qwen3.8 Flash Next<br><span style="opacity:.65">125B / 6B active</span></th>
<th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">MiniMax M3<br><span style="opacity:.65">428B / 23B active</span></th>
</tr></thead><tbody>
<tr><td style="padding:7px 6px">IFBench</td><td style="padding:7px 5px;text-align:center;font-weight:700;color:#3248AF;background:rgba(50,72,175,.05)">74.20</td><td style="padding:7px 5px;text-align:center">64.4</td><td style="padding:7px 5px;text-align:center">43.7</td><td style="padding:7px 5px;text-align:center">77.0</td><td style="padding:7px 5px;text-align:center">75.6</td><td style="padding:7px 5px;text-align:center">75.8</td><td style="padding:7px 5px;text-align:center">79.5</td><td style="padding:7px 5px;text-align:center">76.3</td><td style="padding:7px 5px;text-align:center">81.3</td><td style="padding:7px 5px;text-align:center">82.9</td></tr>
<tr><td style="padding:7px 6px">SciCode</td><td style="padding:7px 5px;text-align:center;font-weight:700;color:#3248AF;background:rgba(50,72,175,.05)">38.08</td><td style="padding:7px 5px;text-align:center">35.8</td><td style="padding:7px 5px;text-align:center">39.6</td><td style="padding:7px 5px;text-align:center">43.6</td><td style="padding:7px 5px;text-align:center">39.5</td><td style="padding:7px 5px;text-align:center">50.3</td><td style="padding:7px 5px;text-align:center">46.6</td><td style="padding:7px 5px;text-align:center">53.1</td><td style="padding:7px 5px;text-align:center">50.6</td><td style="padding:7px 5px;text-align:center">45.4</td></tr>
<tr><td style="padding:7px 6px">GPQA Diamond</td><td style="padding:7px 5px;text-align:center;font-weight:700;color:#3248AF;background:rgba(50,72,175,.05)">85.05</td><td style="padding:7px 5px;text-align:center">84.1</td><td style="padding:7px 5px;text-align:center">78.9</td><td style="padding:7px 5px;text-align:center">83.5</td><td style="padding:7px 5px;text-align:center">85.8</td><td style="padding:7px 5px;text-align:center">90.8</td><td style="padding:7px 5px;text-align:center">90.5</td><td style="padding:7px 5px;text-align:center">92.2</td><td style="padding:7px 5px;text-align:center">92.3</td><td style="padding:7px 5px;text-align:center">92.9</td></tr>
<tr><td style="padding:7px 6px">AA-LCR</td><td style="padding:7px 5px;text-align:center;font-weight:700;color:#3248AF;background:rgba(50,72,175,.05)">68.33</td><td style="padding:7px 5px;text-align:center">66.7</td><td style="padding:7px 5px;text-align:center">59.0</td><td style="padding:7px 5px;text-align:center">80.0</td><td style="padding:7px 5px;text-align:center">72.3</td><td style="padding:7px 5px;text-align:center">79.7</td><td style="padding:7px 5px;text-align:center">82.0</td><td style="padding:7px 5px;text-align:center">81.0</td><td style="padding:7px 5px;text-align:center">79.7</td><td style="padding:7px 5px;text-align:center">74.0</td></tr>
<tr><td style="padding:7px 6px">AA-Omniscience Accuracy</td><td style="padding:7px 5px;text-align:center;font-weight:700;color:#3248AF;background:rgba(50,72,175,.05)">23.00</td><td style="padding:7px 5px;text-align:center">18.8</td><td style="padding:7px 5px;text-align:center">22.9</td><td style="padding:7px 5px;text-align:center">27.0</td><td style="padding:7px 5px;text-align:center">20.7</td><td style="padding:7px 5px;text-align:center">40.4</td><td style="padding:7px 5px;text-align:center">18.4</td><td style="padding:7px 5px;text-align:center">51.4</td><td style="padding:7px 5px;text-align:center">24.5</td><td style="padding:7px 5px;text-align:center">16.7</td></tr>
</tbody></table></div>

<p style="font-size:11px;opacity:.72">
Higher is better for every row. Header parameter figures mix total and active counts, and harnesses and snapshot dates differ across sources, so treat cross-column comparisons as reference values rather than a controlled head-to-head evaluation.
</p>



## Architecture
Agnes-3.0-Flash Preview is a hybrid-attention decoder: three of every four layers run a gated delta rule (recurrent, with per-layer state independent of sequence length), and the fourth runs standard global attention. Only 18 of the 72 layers therefore hold a KV cache that grows with context.
| | |
|---|---|
| Context length | **262 144** tokens |
| Decoder layers | 72 = 54 delta-rule recurrent + 18 global attention, alternating 3 : 1 |
| Hidden size | 5120 |
| Global attention | 24 query heads / 4 KV heads (6 : 1 GQA), head dim 256; RMS-norm on q and k, sigmoid-gated output |
| Delta-rule layers | 16 key heads / 48 value heads, head dim 128; causal conv (kernel 4) in front, gated RMS-norm; recurrent state in fp32 |
| Feed-forward | SwiGLU, intermediate size 17408; plus a parallel SwiGLU 2048 branch in every layer |
| Positions | 3-axis rotary (text / height / width), interleaved mrope sections 11 : 11 : 10, base 1e7, applied to the first 25 % of each head dim (64 dims) |
| Vocabulary | 248 320 |
| Vision tower | 27 layers, hidden 1152, patch 16, 2 × 2 spatial merge, projected to 5120 |

## Quickstart
<div style="border-left:4px solid #3248AF;background:rgba(50,72,175,.08);border-radius:6px;padding:12px 16px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;font-size:14px;line-height:1.6">
<div style="font-weight:700;color:#3248AF;margin-bottom:6px">REMOTE CODE REQUIRED</div>

<p style="margin:0"><b>Agnes-3.0-Flash Preview</b> ships its own model implementation. Always load it with <code>trust_remote_code=True</code>.</p>

</div>



### Requirements
```bash
pip install "transformers>=5.12" torch torchvision accelerate
```
Tested on transformers 5.12.1. Image and video inputs go through the bundled processor, which needs `torchvision`.

### Transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
path = "Agnes-AI/Agnes-3.0-Flash"
tok = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(
    path, dtype="bfloat16", device_map="auto", trust_remote_code=True
)
msgs = [{"role": "user", "content": "请用三句话解释什么是人工智能。"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
```

### Images and video
Image and video inputs go through the bundled processor (also remote code):
```python
from transformers import AutoProcessor
proc = AutoProcessor.from_pretrained(path, trust_remote_code=True)
msgs = [{"role": "user", "content": [{"type": "image", "image": "photo.jpg"},
                                     {"type": "text", "text": "描述这张图。"}]}]
inputs = proc.apply_chat_template(msgs, add_generation_prompt=True, tokenize=True,
                                  return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256)
print(proc.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])
```

### Reasoning effort
The chat template exposes three reasoning levels — `high` (default), `medium`, `low` — plus a thinking-off switch:
```python
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt",
                              reasoning_effort="medium")   # or enable_thinking=False
```

### Tool calling
The chat template renders tool definitions for you. The model emits calls as `<tool_call><function=…><parameter=…>`, and you feed results back as a `tool` role message:
```python
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Look up current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string", "description": "City name"}},
            "required": ["city"],
        },
    },
}]
msgs = [{"role": "user", "content": "What's the weather in Beijing right now?"}]
ids = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True,
                              return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256)
reply = tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)

# <tool_call>

# <function=get_weather>

# <parameter=city>

# Beijing

# </parameter>

# </function>

# </tool_call>

# run the tool, append the result, generate the final answer
msgs += [{"role": "assistant", "content": reply},
         {"role": "tool", "content": "Clear, 26°C, light northeasterly wind"}]
```
Over the OpenAI API pass `tools=` the same way. The server returns the text above verbatim by default; to get structured `tool_calls`, configure sglang with a tool-call parser matching this format (likewise a reasoning parser, if you want the thinking span in `reasoning_content`).

### SGLang
`serve.sh` starts a server from a stock public image, overlaying three files onto the image's `sglang` package and nothing else. See `sglang_patch/README.md`.
```bash
docker run --gpus all --shm-size 64g -p 30001:8080 \
    -v /path/to/agnes-3.0-flash:/model \
    lmsysorg/sglang:nightly-dev-20260908-20ca564b \
    bash /agnes-3.0-flash/serve.sh --served-model-name Agnes-3.0-Flash
```
`serve.sh` forwards extra command-line arguments to sglang, which is how `--served-model-name` takes effect; `--tp 2` works the same way. The server listens on port 8080 inside the container:
```python
from openai import OpenAI
client = OpenAI(api_key="EMPTY", base_url="http://localhost:30001/v1")
response = client.chat.completions.create(
    model="Agnes-3.0-Flash",
    messages=[{"role": "user", "content": "Design a fault-tolerant event processing architecture."}],
    temperature=1.0,
    max_tokens=2000,
)
print(response.choices[0].message.content)
```
Pass `stream=True` for streaming; `tools=` and `reasoning_effort=` are accepted the same way.

## Hardware Requirements
| Resource | Recommendation |
|---|---|
| GPUs | 1 × NVIDIA H200 141 GB or NVIDIA H100 80 GB (or equivalent) at bf16 |
| Tensor parallel | `--tp 1`; `--tp 2` for maximum context and concurrency |
| Weights on disk | Approximately 66 GB for the bf16 checkpoint |
| Host memory | 128 GB or more recommended |

Actual context length and concurrency depend on KV-cache allocation, runtime overhead, and tensor-parallel configuration; validate the target workload on the intended hardware.

## Recommended Inference Settings
| Setting | Recommended |
|---|---|
| `temperature` | 1.0 |
| `top_p` | 0.95 |
| `top_k` | 20 |
| `reasoning_effort` | `high` for hard reasoning, `low` for latency-sensitive traffic |
| `max_tokens` | 2000 or higher |

These are the checkpoint's own `generation_config.json` defaults.

## Model Capabilities
| Capability | Support |
|---|---|
| Advanced reasoning | Yes, with `high` / `medium` / `low` effort levels |
| Coding and debugging | Yes |
| Long-context analysis | 262 144 tokens |
| Image understanding | Yes |
| Video understanding | Yes |
| Tool calling | Yes (`<tool_call>` / `<tool_response>`) |
| Streaming | Yes |
| OpenAI-compatible APIs | Chat Completions via sglang |

## License
Released under the [Apache License 2.0](LICENSE).

## Citation
```bibtex
@misc{agnes30flash2026,
  title        = {Agnes-3.0-Flash Preview},
  author       = {{Agnes AI}},
  year         = {2026},
  month        = sep,
  howpublished = {Open-weights preview checkpoint},
  url          = {https://agnes-ai.com/}
}
```