| --- |
| language: |
| - en |
| - zh |
| license: apache-2.0 |
| library_name: transformers |
| pipeline_tag: image-text-to-text |
| tags: |
| - agnes-ai |
| - reasoning |
| - multimodal |
| - long-context |
| - hybrid-attention |
| --- |
| <p align="center"> |
| <img width="132" src="assets/agnes_logo.svg" alt="Agnes AI logo"> |
| </p> |
|
|
| <p align="center"> |
| <a href="https://agnes-ai.com/"><img src="https://img.shields.io/badge/Agnes_AI-Website-3248AF" alt="Agnes AI website"></a> |
| <a href="#quickstart"><img src="https://img.shields.io/badge/Agnes--3.0--Flash_Preview-Open_Weights-3248AF" alt="Open weights"></a> |
| <img src="https://img.shields.io/badge/License-Apache_2.0-111827" alt="Apache 2.0"> |
| </p> |
|
|
|
|
|
|
| # Agnes-3.0-Flash Preview |
|
|
|
|
| ## Model version clarification |
| This repository contains an earlier open-weight **Preview checkpoint** of Agnes 3.0 Flash. It is distinct from the newer **production/API checkpoint** listed on [Artificial Analysis](https://artificialanalysis.ai/models/agnes-3-0-flash). |
| The Preview release has **33B parameters** and a context window of **262,144 tokens**. The production/API model uses a different checkpoint and configuration, with a **1M-token context window**. Its benchmark results should not be attributed to the Preview weights released here. |
| This repository was initially published as `Agnes-3.0-Flash` without the `Preview` suffix. The model card now explicitly identifies this release as **Agnes-3.0-Flash Preview** to clarify the distinction between the open-weight release and the production/API model. |
| The specifications and Agnes benchmark results below refer to the **Preview checkpoint**. |
| Hello! 👋 Today we are introducing **Agnes-3.0-Flash Preview**, an **open-weights multimodal preview model** built for people who want flagship-class reasoning without flagship-class hardware. |
| Highlights: |
| - **Competitive across core capabilities.** Agnes-3.0-Flash Preview posts competitive results across reasoning, coding, and instruction-following evaluations. |
| - **Built for demanding work.** A **262 144-token context window**, adjustable reasoning effort, tool calling, and **text, image and video** understanding. |
|
|
|
|
| ### Benchmarks |
| > **Benchmark scope:** The Agnes results in the chart and table below belong to the **Agnes-3.0-Flash Preview open-weight checkpoint** released in this repository. They are not results for the production/API Agnes 3.0 Flash model listed on Artificial Analysis. |
| <p align="center"> |
| <img style="width:100%;max-width:1100px" src="assets/benchmark-preview.png" alt="Agnes-3.0-Flash Preview benchmark reference results"> |
| </p> |
|
|
|
|
| The Agnes-3.0-Flash Preview scores in the chart correspond to the open-weight checkpoint released in this repository. |
| Reference results across contemporary models are shown below. The figures were compiled from different sources, harnesses, and model snapshots and do not constitute a controlled head-to-head comparison. |
| <div style="font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;width:100%;margin:0 auto;padding:16px 0;overflow-x:auto"> |
| <table style="width:100%;table-layout:fixed;border-collapse:collapse;font-size:11px;min-width:1180px"> |
| <thead><tr> |
| <th style="width:15%;padding:9px 6px;text-align:left;border-bottom:2px solid #3248AF;color:#3248AF;font-size:11px">Benchmark</th> |
| <th style="width:10%;padding:9px 5px;text-align:center;font-weight:700;border-bottom:2px solid #3248AF;color:#3248AF;background:rgba(50,72,175,.09);font-size:10px;line-height:1.3">Agnes-3.0-Flash Preview</th> |
| <th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Qwen3.6-35B-A3B<br><span style="opacity:.65">35B / 3B active</span></th> |
| <th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Kimi K2.5<br><span style="opacity:.65">1T / 32B active</span></th> |
| <th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Muse Glimmer<br><span style="opacity:.65">30B</span></th> |
| <th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Qwen3.5<br><span style="opacity:.65">27B</span></th> |
| <th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">DeepSeek V4 Flash 0731<br><span style="opacity:.65">284B / 13B active</span></th> |
| <th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Qwen3.8<br><span style="opacity:.65">27B</span></th> |
| <th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Gemini 3.5 Flash<br><span style="opacity:.65">undisclosed</span></th> |
| <th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">Qwen3.8 Flash Next<br><span style="opacity:.65">125B / 6B active</span></th> |
| <th style="width:8.33%;padding:9px 5px;text-align:center;border-bottom:2px solid #3248AF;font-size:10px;line-height:1.3">MiniMax M3<br><span style="opacity:.65">428B / 23B active</span></th> |
| </tr></thead><tbody> |
| <tr><td style="padding:7px 6px">IFBench</td><td style="padding:7px 5px;text-align:center;font-weight:700;color:#3248AF;background:rgba(50,72,175,.05)">74.20</td><td style="padding:7px 5px;text-align:center">64.4</td><td style="padding:7px 5px;text-align:center">43.7</td><td style="padding:7px 5px;text-align:center">77.0</td><td style="padding:7px 5px;text-align:center">75.6</td><td style="padding:7px 5px;text-align:center">75.8</td><td style="padding:7px 5px;text-align:center">79.5</td><td style="padding:7px 5px;text-align:center">76.3</td><td style="padding:7px 5px;text-align:center">81.3</td><td style="padding:7px 5px;text-align:center">82.9</td></tr> |
| <tr><td style="padding:7px 6px">SciCode</td><td style="padding:7px 5px;text-align:center;font-weight:700;color:#3248AF;background:rgba(50,72,175,.05)">38.08</td><td style="padding:7px 5px;text-align:center">35.8</td><td style="padding:7px 5px;text-align:center">39.6</td><td style="padding:7px 5px;text-align:center">43.6</td><td style="padding:7px 5px;text-align:center">39.5</td><td style="padding:7px 5px;text-align:center">50.3</td><td style="padding:7px 5px;text-align:center">46.6</td><td style="padding:7px 5px;text-align:center">53.1</td><td style="padding:7px 5px;text-align:center">50.6</td><td style="padding:7px 5px;text-align:center">45.4</td></tr> |
| <tr><td style="padding:7px 6px">GPQA Diamond</td><td style="padding:7px 5px;text-align:center;font-weight:700;color:#3248AF;background:rgba(50,72,175,.05)">85.05</td><td style="padding:7px 5px;text-align:center">84.1</td><td style="padding:7px 5px;text-align:center">78.9</td><td style="padding:7px 5px;text-align:center">83.5</td><td style="padding:7px 5px;text-align:center">85.8</td><td style="padding:7px 5px;text-align:center">90.8</td><td style="padding:7px 5px;text-align:center">90.5</td><td style="padding:7px 5px;text-align:center">92.2</td><td style="padding:7px 5px;text-align:center">92.3</td><td style="padding:7px 5px;text-align:center">92.9</td></tr> |
| <tr><td style="padding:7px 6px">AA-LCR</td><td style="padding:7px 5px;text-align:center;font-weight:700;color:#3248AF;background:rgba(50,72,175,.05)">68.33</td><td style="padding:7px 5px;text-align:center">66.7</td><td style="padding:7px 5px;text-align:center">59.0</td><td style="padding:7px 5px;text-align:center">80.0</td><td style="padding:7px 5px;text-align:center">72.3</td><td style="padding:7px 5px;text-align:center">79.7</td><td style="padding:7px 5px;text-align:center">82.0</td><td style="padding:7px 5px;text-align:center">81.0</td><td style="padding:7px 5px;text-align:center">79.7</td><td style="padding:7px 5px;text-align:center">74.0</td></tr> |
| <tr><td style="padding:7px 6px">AA-Omniscience Accuracy</td><td style="padding:7px 5px;text-align:center;font-weight:700;color:#3248AF;background:rgba(50,72,175,.05)">23.00</td><td style="padding:7px 5px;text-align:center">18.8</td><td style="padding:7px 5px;text-align:center">22.9</td><td style="padding:7px 5px;text-align:center">27.0</td><td style="padding:7px 5px;text-align:center">20.7</td><td style="padding:7px 5px;text-align:center">40.4</td><td style="padding:7px 5px;text-align:center">18.4</td><td style="padding:7px 5px;text-align:center">51.4</td><td style="padding:7px 5px;text-align:center">24.5</td><td style="padding:7px 5px;text-align:center">16.7</td></tr> |
| </tbody></table></div> |
|
|
| <p style="font-size:11px;opacity:.72"> |
| Higher is better for every row. Header parameter figures mix total and active counts, and harnesses and snapshot dates differ across sources, so treat cross-column comparisons as reference values rather than a controlled head-to-head evaluation. |
| </p> |
|
|
|
|
|
|
| ## Architecture |
| Agnes-3.0-Flash Preview is a hybrid-attention decoder: three of every four layers run a gated delta rule (recurrent, with per-layer state independent of sequence length), and the fourth runs standard global attention. Only 18 of the 72 layers therefore hold a KV cache that grows with context. |
| | | | |
| |---|---| |
| | Context length | **262 144** tokens | |
| | Decoder layers | 72 = 54 delta-rule recurrent + 18 global attention, alternating 3 : 1 | |
| | Hidden size | 5120 | |
| | Global attention | 24 query heads / 4 KV heads (6 : 1 GQA), head dim 256; RMS-norm on q and k, sigmoid-gated output | |
| | Delta-rule layers | 16 key heads / 48 value heads, head dim 128; causal conv (kernel 4) in front, gated RMS-norm; recurrent state in fp32 | |
| | Feed-forward | SwiGLU, intermediate size 17408; plus a parallel SwiGLU 2048 branch in every layer | |
| | Positions | 3-axis rotary (text / height / width), interleaved mrope sections 11 : 11 : 10, base 1e7, applied to the first 25 % of each head dim (64 dims) | |
| | Vocabulary | 248 320 | |
| | Vision tower | 27 layers, hidden 1152, patch 16, 2 × 2 spatial merge, projected to 5120 | |
|
|
| ## Quickstart |
| <div style="border-left:4px solid #3248AF;background:rgba(50,72,175,.08);border-radius:6px;padding:12px 16px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;font-size:14px;line-height:1.6"> |
| <div style="font-weight:700;color:#3248AF;margin-bottom:6px">REMOTE CODE REQUIRED</div> |
|
|
| <p style="margin:0"><b>Agnes-3.0-Flash Preview</b> ships its own model implementation. Always load it with <code>trust_remote_code=True</code>.</p> |
|
|
| </div> |
|
|
|
|
|
|
| ### Requirements |
| ```bash |
| pip install "transformers>=5.12" torch torchvision accelerate |
| ``` |
| Tested on transformers 5.12.1. Image and video inputs go through the bundled processor, which needs `torchvision`. |
|
|
| ### Transformers |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| path = "Agnes-AI/Agnes-3.0-Flash" |
| tok = AutoTokenizer.from_pretrained(path) |
| model = AutoModelForCausalLM.from_pretrained( |
| path, dtype="bfloat16", device_map="auto", trust_remote_code=True |
| ) |
| msgs = [{"role": "user", "content": "请用三句话解释什么是人工智能。"}] |
| ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device) |
| out = model.generate(ids, max_new_tokens=256) |
| print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| ### Images and video |
| Image and video inputs go through the bundled processor (also remote code): |
| ```python |
| from transformers import AutoProcessor |
| proc = AutoProcessor.from_pretrained(path, trust_remote_code=True) |
| msgs = [{"role": "user", "content": [{"type": "image", "image": "photo.jpg"}, |
| {"type": "text", "text": "描述这张图。"}]}] |
| inputs = proc.apply_chat_template(msgs, add_generation_prompt=True, tokenize=True, |
| return_dict=True, return_tensors="pt").to(model.device) |
| out = model.generate(**inputs, max_new_tokens=256) |
| print(proc.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]) |
| ``` |
|
|
| ### Reasoning effort |
| The chat template exposes three reasoning levels — `high` (default), `medium`, `low` — plus a thinking-off switch: |
| ```python |
| ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", |
| reasoning_effort="medium") # or enable_thinking=False |
| ``` |
|
|
| ### Tool calling |
| The chat template renders tool definitions for you. The model emits calls as `<tool_call><function=…><parameter=…>`, and you feed results back as a `tool` role message: |
| ```python |
| tools = [{ |
| "type": "function", |
| "function": { |
| "name": "get_weather", |
| "description": "Look up current weather for a city", |
| "parameters": { |
| "type": "object", |
| "properties": {"city": {"type": "string", "description": "City name"}}, |
| "required": ["city"], |
| }, |
| }, |
| }] |
| msgs = [{"role": "user", "content": "What's the weather in Beijing right now?"}] |
| ids = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True, |
| return_tensors="pt").to(model.device) |
| out = model.generate(ids, max_new_tokens=256) |
| reply = tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True) |
| |
| # <tool_call> |
| |
| # <function=get_weather> |
| |
| # <parameter=city> |
| |
| # Beijing |
| |
| # </parameter> |
| |
| # </function> |
| |
| # </tool_call> |
| |
| # run the tool, append the result, generate the final answer |
| msgs += [{"role": "assistant", "content": reply}, |
| {"role": "tool", "content": "Clear, 26°C, light northeasterly wind"}] |
| ``` |
| Over the OpenAI API pass `tools=` the same way. The server returns the text above verbatim by default; to get structured `tool_calls`, configure sglang with a tool-call parser matching this format (likewise a reasoning parser, if you want the thinking span in `reasoning_content`). |
|
|
| ### SGLang |
| `serve.sh` starts a server from a stock public image, overlaying three files onto the image's `sglang` package and nothing else. See `sglang_patch/README.md`. |
| ```bash |
| docker run --gpus all --shm-size 64g -p 30001:8080 \ |
| -v /path/to/agnes-3.0-flash:/model \ |
| lmsysorg/sglang:nightly-dev-20260908-20ca564b \ |
| bash /agnes-3.0-flash/serve.sh --served-model-name Agnes-3.0-Flash |
| ``` |
| `serve.sh` forwards extra command-line arguments to sglang, which is how `--served-model-name` takes effect; `--tp 2` works the same way. The server listens on port 8080 inside the container: |
| ```python |
| from openai import OpenAI |
| client = OpenAI(api_key="EMPTY", base_url="http://localhost:30001/v1") |
| response = client.chat.completions.create( |
| model="Agnes-3.0-Flash", |
| messages=[{"role": "user", "content": "Design a fault-tolerant event processing architecture."}], |
| temperature=1.0, |
| max_tokens=2000, |
| ) |
| print(response.choices[0].message.content) |
| ``` |
| Pass `stream=True` for streaming; `tools=` and `reasoning_effort=` are accepted the same way. |
|
|
| ## Hardware Requirements |
| | Resource | Recommendation | |
| |---|---| |
| | GPUs | 1 × NVIDIA H200 141 GB or NVIDIA H100 80 GB (or equivalent) at bf16 | |
| | Tensor parallel | `--tp 1`; `--tp 2` for maximum context and concurrency | |
| | Weights on disk | Approximately 66 GB for the bf16 checkpoint | |
| | Host memory | 128 GB or more recommended | |
|
|
| Actual context length and concurrency depend on KV-cache allocation, runtime overhead, and tensor-parallel configuration; validate the target workload on the intended hardware. |
|
|
| ## Recommended Inference Settings |
| | Setting | Recommended | |
| |---|---| |
| | `temperature` | 1.0 | |
| | `top_p` | 0.95 | |
| | `top_k` | 20 | |
| | `reasoning_effort` | `high` for hard reasoning, `low` for latency-sensitive traffic | |
| | `max_tokens` | 2000 or higher | |
|
|
| These are the checkpoint's own `generation_config.json` defaults. |
|
|
| ## Model Capabilities |
| | Capability | Support | |
| |---|---| |
| | Advanced reasoning | Yes, with `high` / `medium` / `low` effort levels | |
| | Coding and debugging | Yes | |
| | Long-context analysis | 262 144 tokens | |
| | Image understanding | Yes | |
| | Video understanding | Yes | |
| | Tool calling | Yes (`<tool_call>` / `<tool_response>`) | |
| | Streaming | Yes | |
| | OpenAI-compatible APIs | Chat Completions via sglang | |
|
|
| ## License |
| Released under the [Apache License 2.0](LICENSE). |
|
|
| ## Citation |
| ```bibtex |
| @misc{agnes30flash2026, |
| title = {Agnes-3.0-Flash Preview}, |
| author = {{Agnes AI}}, |
| year = {2026}, |
| month = sep, |
| howpublished = {Open-weights preview checkpoint}, |
| url = {https://agnes-ai.com/} |
| } |
| ``` |
