Hugging Face's logo

AndrewThompson1233
/
maba-v1-architecture

Transformers
PyTorch
Safetensors
English
maba
maba-v1
maba-v1.1
architecture
recurrent
gated-deltanet
gdn
gdn-2
linear-attention
linear-recurrence
state-space-model
ssm
gqa
grouped-query-attention
swiglu
rmsnorm
rope
speculative-decoding
mtp
multi-token-prediction
scaling
100m
1b
3b
7b
30b
cpp
avx2
Model card Files
xet
 Community
AndrewThompson1233 commited on
Commit
4edf935
·
1 Parent(s): bdff173

commit39

Browse files
Files changed (1) hide show
  1. SCALING.md +6 -4
SCALING.md CHANGED
@@ -6,13 +6,15 @@ Technical specification and architectural comparison for scaling the Maba archit
6
 
7
  ## 1. Visual Comparison
8
 
9
- ![Maba Multi-Scale Comparison Against 2026 Architectures](assets/scaling_comparison.svg)
 
 
10
 
11
  ---
12
 
13
  ## 2. Scaling Topology Presets
14
 
15
- | Metric | Maba-100M | Maba-1B | Maba-3B | Maba-7B | Maba-30B (Agentic) |
16
  | :--- | :--- | :--- | :--- | :--- | :--- |
17
  | **Total Parameters** | 101,177,984 (101.2M) | 1,004,729,600 (1.00B) | 2,977,156,608 (2.98B) | 7,127,820,544 (7.13B) | 29,039,812,864 (29.04B) |
18
  | **Core Parameters** | 96,327,040 (95.21%) | 982,508,800 (97.79%) | 2,941,302,784 (98.80%) | 7,071,844,608 (99.21%) | 28,930,813,184 (99.62%) |
@@ -56,7 +58,7 @@ The decisive divergence lies in embedding topology and parameter allocation:
56
  | **KV Cache (131k FP16)** | **1,280.0 MB** | 768.0 MB | 1,536.0 MB | **2,048.0 MB** | 4,096.0 MB |
57
  | **Speculative Decoding** | Built-in MTP (k=2) | Built-in MTP | Built-in MTP | Built-in MTP (k=2) | Built-in MTP |
58
 
59
- ### Agentic Tier (27B - 31B)
60
 
61
  | Feature | Maba-30B (2026) | Muse-Glimmer-30B (Meta 2026) | Qwen3.8-27B (2026) | Gemma4-31B (Google 2026) |
62
  | :--- | :--- | :--- | :--- | :--- |
@@ -176,6 +178,6 @@ python3 -m maba.cli params --scale 3B
176
  # Audit 7B configuration
177
  python3 -m maba.cli params --scale 7B
178
 
179
- # Audit 30B agentic configuration
180
  python3 -m maba.cli params --scale 30B
181
  ```
 
6
 
7
  ## 1. Visual Comparison
8
 
9
+ <p align="center">
10
+ <img src="https://huggingface.co/AndrewThompson1233/maba-v1-architecture/resolve/main/assets/scaling_comparison.svg" width="900" alt="Maba Multi-Scale Comparison Against 2026 Architectures" />
11
+ </p>
12
 
13
  ---
14
 
15
  ## 2. Scaling Topology Presets
16
 
17
+ | Metric | Maba-100M | Maba-1B | Maba-3B | Maba-7B | Maba-30B |
18
  | :--- | :--- | :--- | :--- | :--- | :--- |
19
  | **Total Parameters** | 101,177,984 (101.2M) | 1,004,729,600 (1.00B) | 2,977,156,608 (2.98B) | 7,127,820,544 (7.13B) | 29,039,812,864 (29.04B) |
20
  | **Core Parameters** | 96,327,040 (95.21%) | 982,508,800 (97.79%) | 2,941,302,784 (98.80%) | 7,071,844,608 (99.21%) | 28,930,813,184 (99.62%) |
 
58
  | **KV Cache (131k FP16)** | **1,280.0 MB** | 768.0 MB | 1,536.0 MB | **2,048.0 MB** | 4,096.0 MB |
59
  | **Speculative Decoding** | Built-in MTP (k=2) | Built-in MTP | Built-in MTP | Built-in MTP (k=2) | Built-in MTP |
60
 
61
+ ### 30B Tier (27B - 31B)
62
 
63
  | Feature | Maba-30B (2026) | Muse-Glimmer-30B (Meta 2026) | Qwen3.8-27B (2026) | Gemma4-31B (Google 2026) |
64
  | :--- | :--- | :--- | :--- | :--- |
 
178
  # Audit 7B configuration
179
  python3 -m maba.cli params --scale 7B
180
 
181
+ # Audit 30B configuration
182
  python3 -m maba.cli params --scale 30B
183
  ```