hero775 commited on
Commit
48f98a6
Β·
verified Β·
1 Parent(s): b533f18

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +82 -18
README.md CHANGED
@@ -16,6 +16,8 @@ tags:
16
  - ollama
17
  - batiai
18
  - on-device
 
 
19
  base_model: Qwen/Qwen3.6-35B-A3B
20
  pipeline_tag: text-generation
21
  library_name: llama.cpp
@@ -26,11 +28,14 @@ library_name: llama.cpp
26
  <p align="center">
27
  <a href="https://flow.bati.ai"><img src="https://img.shields.io/badge/BatiFlow-macOS%20AI%20Automation-blue?style=for-the-badge&logo=apple" alt="BatiFlow"></a>
28
  <a href="https://ollama.com/batiai/qwen3.6-35b"><img src="https://img.shields.io/badge/Ollama-batiai%2Fqwen3.6--35b-green?style=for-the-badge" alt="Ollama"></a>
 
29
  </p>
30
 
31
- > imatrix-calibrated GGUF quantizations of **Qwen/Qwen3.6-35B-A3B** (text-only) for on-device AI on Mac.
32
  > Built and verified by [BatiAI](https://bati.ai) for [BatiFlow](https://flow.bati.ai) β€” free, unlimited, on-device AI automation.
33
 
 
 
34
  ## Quick Start
35
 
36
  ```bash
@@ -43,7 +48,7 @@ ollama pull batiai/qwen3.6-35b:iq4
43
  ollama run batiai/qwen3.6-35b:iq4
44
  ```
45
 
46
- Aliases `:q3` and `:q4` point to the same blobs.
47
 
48
  ## Available Quantizations
49
 
@@ -52,17 +57,61 @@ Aliases `:q3` and `:q4` point to the same blobs.
52
  | `:iq3` / `:q3` | **IQ3_XXS** (imatrix) | **13 GB** | 16 GB | Mac mini / MacBook Air 16GB |
53
  | `:iq4` / `:q4` | **IQ4_XS** (imatrix) | **18 GB** | 24 GB | MacBook Pro / Mac Studio 24GB+ |
54
 
55
- Both quants use an **importance matrix** computed from wikitext-2-raw to preserve quality at low bit-widths. IQ quants generally outperform K-quants of similar size.
56
 
57
  ## Why Qwen 3.6 35B-A3B?
58
 
59
- - **MoE β€” 35B total, only ~3B active per token** β†’ 9Γ— less compute than a 27B Dense model
60
- - **262K native context** (1M with YaRN scaling)
61
- - **Gated DeltaNet hybrid attention** β€” long-context efficiency
62
- - **Function calling** β€” `qwen3_coder` parser (vLLM/SGLang)
63
- - **Agentic workflows** β€” tuned for repository-level coding, multi-tool reasoning
64
- - **Multilingual** β€” Korean / Japanese / Chinese / English
65
- - **Apache 2.0** β€” commercial-friendly
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
66
 
67
  ## MoE Advantage
68
 
@@ -71,10 +120,10 @@ Both quants use an **importance matrix** computed from wikitext-2-raw to preserv
71
  | Total params | 35B | 27B |
72
  | **Active params / token** | **3B** | **27B** |
73
  | Experts | 256 (8 routed + 1 shared) | β€” |
74
- | Typical VRAM | ~23 GB (IQ4) | ~28 GB |
75
  | Relative speed | **Faster** | Baseline |
76
 
77
- Only 9 of 256 experts fire per token β€” same reasoning, far less compute.
78
 
79
  ## RAM Requirements (on-device)
80
 
@@ -85,26 +134,37 @@ Only 9 of 256 experts fire per token β€” same reasoning, far less compute.
85
  | 32 GB | βœ… | βœ… comfortable |
86
  | 48 GB+ | βœ… | βœ… ideal |
87
 
88
- ## Benchmarks
89
 
90
- Benchmarks on Apple Silicon will be updated as measurements come in. To reproduce on your Mac:
91
 
92
  ```bash
93
  ollama run batiai/qwen3.6-35b:iq4 --verbose "Write a haiku about Seoul in autumn."
94
  ```
95
 
96
- The `--verbose` flag prints prompt-eval rate, token-generation rate, and memory usage.
 
 
97
 
98
  ## Note on the "3.6" Naming
99
 
100
- Upstream Qwen released this model as **Qwen 3.6** publicly. Internally the Hugging Face config still registers the architecture as `Qwen3_5MoeForConditionalGeneration` (a transitional class name carried over from the 3.5 line). llama.cpp handles this class via its `Qwen3_5MoeTextModel` converter, which is what these GGUFs were built from β€” text-only, vision tower excluded.
101
 
102
  ## Technical Details
103
 
104
  - **Original Model**: [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)
105
- - **Architecture**: MoE + Gated DeltaNet hybrid (35B total, ~3B active, 40 layers, hidden 2048)
 
 
 
 
 
 
106
  - **Experts**: 256 total, 8 routed + 1 shared per token
107
- - **Context Window**: 262,144 tokens native (extensible to ~1M via YaRN)
 
 
 
108
  - **License**: Apache 2.0
109
  - **Quantized with**: [llama.cpp](https://github.com/ggml-org/llama.cpp) build `bafae2765`
110
  - **Quantized by**: [BatiAI](https://bati.ai)
@@ -144,3 +204,7 @@ No third-party intermediaries. Direct from official Qwen weights.
144
  This repo mirrors the upstream license. `Qwen/Qwen3.6-35B-A3B` is released under **Apache 2.0** β€” commercial use permitted.
145
 
146
  BatiAI's quantization pipeline is MIT.
 
 
 
 
 
16
  - ollama
17
  - batiai
18
  - on-device
19
+ - agentic
20
+ - coding
21
  base_model: Qwen/Qwen3.6-35B-A3B
22
  pipeline_tag: text-generation
23
  library_name: llama.cpp
 
28
  <p align="center">
29
  <a href="https://flow.bati.ai"><img src="https://img.shields.io/badge/BatiFlow-macOS%20AI%20Automation-blue?style=for-the-badge&logo=apple" alt="BatiFlow"></a>
30
  <a href="https://ollama.com/batiai/qwen3.6-35b"><img src="https://img.shields.io/badge/Ollama-batiai%2Fqwen3.6--35b-green?style=for-the-badge" alt="Ollama"></a>
31
+ <a href="https://huggingface.co/Qwen/Qwen3.6-35B-A3B"><img src="https://img.shields.io/badge/Upstream-Qwen3.6--35B--A3B-orange?style=for-the-badge" alt="Upstream"></a>
32
  </p>
33
 
34
+ > **"Agentic Coding Power, Now Open to All"** β€” imatrix-calibrated GGUF quantizations of **Qwen/Qwen3.6-35B-A3B** (text-only) for on-device AI on Mac.
35
  > Built and verified by [BatiAI](https://bati.ai) for [BatiFlow](https://flow.bati.ai) β€” free, unlimited, on-device AI automation.
36
 
37
+ Released by Alibaba on **April 15, 2026** as the successor to Qwen 3.5 35B-A3B, with substantial upgrades in **agentic coding**, **frontend workflows**, **repository-level reasoning**, and **thinking preservation** for iterative development.
38
+
39
  ## Quick Start
40
 
41
  ```bash
 
48
  ollama run batiai/qwen3.6-35b:iq4
49
  ```
50
 
51
+ Aliases `:q3` / `:q4` point to the same blobs as `:iq3` / `:iq4`.
52
 
53
  ## Available Quantizations
54
 
 
57
  | `:iq3` / `:q3` | **IQ3_XXS** (imatrix) | **13 GB** | 16 GB | Mac mini / MacBook Air 16GB |
58
  | `:iq4` / `:q4` | **IQ4_XS** (imatrix) | **18 GB** | 24 GB | MacBook Pro / Mac Studio 24GB+ |
59
 
60
+ Both quants use an **importance matrix** computed from wikitext-2-raw to preserve quality at low bit-widths β€” IQ quants generally outperform K-quants of similar size.
61
 
62
  ## Why Qwen 3.6 35B-A3B?
63
 
64
+ Upstream headline: _"Agentic Coding Power, Now Open to All"_ β€” the model is tuned for multi-step coding agents, long-horizon repo reasoning, and tool use.
65
+
66
+ ### Benchmarks (official Qwen BF16 figures)
67
+
68
+ **Coding & Agentic**
69
+
70
+ | Benchmark | Qwen 3.6-35B-A3B | Qwen 3.5-35B-A3B | Gemma 4-31B |
71
+ |---|---:|---:|---:|
72
+ | SWE-bench Verified | **73.4** | 70.0 | 52.0 |
73
+ | SWE-bench Multilingual | **67.2** | β€” | 51.7 |
74
+ | SWE-bench Pro | **49.5** | β€” | 35.7 |
75
+ | Terminal-Bench 2.0 | **51.5** | 40.5 | 42.9 |
76
+ | QwenWebBench | **1397** | 978 | β€” |
77
+
78
+ **Math & Reasoning**
79
+
80
+ | Benchmark | Qwen 3.6-35B-A3B | Gemma 4-31B |
81
+ |---|---:|---:|
82
+ | AIME26 | **92.7** | 89.2 |
83
+ | GPQA | **86.0** | β€” |
84
+ | HMMT Feb 26 | **83.6** | β€” |
85
+ | HLE | **21.4** | β€” |
86
+ | LiveCodeBench v6 | **80.4** | β€” |
87
+
88
+ **General Knowledge**
89
+
90
+ | Benchmark | Qwen 3.6-35B-A3B |
91
+ |---|---:|
92
+ | MMLU-Pro | 85.2 |
93
+ | MMLU-Redux | 93.3 |
94
+ | SuperGPQA | 64.7 |
95
+ | C-Eval | 90.0 |
96
+
97
+ **Agent / Tool Use**
98
+
99
+ | Benchmark | Qwen 3.6-35B-A3B |
100
+ |---|---:|
101
+ | TAU3-Bench | 67.2 |
102
+ | MCP-Atlas | 62.8 |
103
+ | WideSearch | 60.1 |
104
+ | MCPMark | 37.0 |
105
+ | Tool Decathlon | 26.9 |
106
+
107
+ ### Key takeaways
108
+
109
+ - **SWE-bench Verified jumps +3.4** over Qwen 3.5 to **73.4** β€” top-tier agentic-coding among open models
110
+ - **Terminal-Bench 2.0 +11.0** over 3.5 β†’ genuine real-world command-line competence
111
+ - **QwenWebBench 1397** vs 978 for 3.5 β€” a **43% jump** in agentic web tasks
112
+ - **Beats Gemma 4-31B on every published coding & reasoning benchmark** despite Gemma being a similar-sized dense model (A3B only activates 3B params per token)
113
+
114
+ _Note: these are upstream BF16 figures. IQ3_XXS / IQ4_XS quantization may cost a few points on the hardest benchmarks β€” post your own bench results and we'll update this card._
115
 
116
  ## MoE Advantage
117
 
 
120
  | Total params | 35B | 27B |
121
  | **Active params / token** | **3B** | **27B** |
122
  | Experts | 256 (8 routed + 1 shared) | β€” |
123
+ | Typical VRAM (IQ4) | ~23 GB | ~28 GB |
124
  | Relative speed | **Faster** | Baseline |
125
 
126
+ Only 9 of 256 experts fire per token β€” same reasoning capacity, far less compute.
127
 
128
  ## RAM Requirements (on-device)
129
 
 
134
  | 32 GB | βœ… | βœ… comfortable |
135
  | 48 GB+ | βœ… | βœ… ideal |
136
 
137
+ ## On-device Benchmarks (measured)
138
 
139
+ Measured tokens/s on your Mac β€” update this card with your own numbers:
140
 
141
  ```bash
142
  ollama run batiai/qwen3.6-35b:iq4 --verbose "Write a haiku about Seoul in autumn."
143
  ```
144
 
145
+ | Mac | IQ3_XXS | IQ4_XS |
146
+ |-----|--------:|-------:|
147
+ | _pending community measurements_ | β€” | β€” |
148
 
149
  ## Note on the "3.6" Naming
150
 
151
+ Upstream Qwen released this model as **Qwen 3.6** publicly. Internally the Hugging Face config still registers the architecture as `Qwen3_5MoeForConditionalGeneration` (a transitional class name carried over from the 3.5 line). llama.cpp handles this via its `Qwen3_5MoeTextModel` converter, which is what these GGUFs were built from β€” **text-only, vision tower excluded**. For the vision-language capabilities (MMMU 81.7, MathVista 86.4, etc.), use the upstream multimodal weights directly.
152
 
153
  ## Technical Details
154
 
155
  - **Original Model**: [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)
156
+ - **Released**: 2026-04-15
157
+ - **Architecture**: MoE + **Gated DeltaNet hybrid attention**
158
+ - 40 layers, hidden 2048, expert-intermediate 512
159
+ - Layout: 10Γ— (3Γ— Gated DeltaNet β†’ MoE + 1Γ— Gated Attention β†’ MoE)
160
+ - Linear-attention heads: 32 V / 16 QK (head dim 128)
161
+ - Softmax-attention heads: 16 Q / 2 KV (head dim 256, RoPE dim 64)
162
+ - **Parameters**: 35 B total, ~3 B active per forward pass
163
  - **Experts**: 256 total, 8 routed + 1 shared per token
164
+ - **Context Window**: 262,144 tokens native (extensible to ~1,010,000 via YaRN)
165
+ - **Vocabulary**: 248,320 tokens (padded)
166
+ - **Training**: Multi-token Prediction (MTP) applied for speculative decoding
167
+ - **Modes**: thinking / non-thinking switchable
168
  - **License**: Apache 2.0
169
  - **Quantized with**: [llama.cpp](https://github.com/ggml-org/llama.cpp) build `bafae2765`
170
  - **Quantized by**: [BatiAI](https://bati.ai)
 
204
  This repo mirrors the upstream license. `Qwen/Qwen3.6-35B-A3B` is released under **Apache 2.0** β€” commercial use permitted.
205
 
206
  BatiAI's quantization pipeline is MIT.
207
+
208
+ ## Sources
209
+
210
+ Benchmark numbers in this card come from the official upstream [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) model card and Qwen's [research blog](https://qwen.ai/blog?id=qwen3.6-35b-a3b). Quantization and on-device numbers are measured by BatiAI.