commit5

README.md

@@ -9,26 +9,43 @@ pipeline_tag: text-generation
 
 # Maba v1.5 (103.5M)
 
-Trained checkpoint built on [AndrewThompson1233/maba-v1.5-exp-architecture](https://huggingface.co/AndrewThompson1233/maba-v1.5-exp-architecture).
+> [!WARNING]
+> **Research Proof-of-Concept - Not for General / Production Use**
+> This checkpoint is an empirical demonstration and verification artifact. It proves that the **[AndrewThompson1233/maba-v1.5-exp-architecture](https://huggingface.co/AndrewThompson1233/maba-v1.5-exp-architecture)** architecture is fully functional, trainable from scratch, and numerically stable on consumer/enterprise hardware (NVIDIA L4).
 
-* Base Architecture: [AndrewThompson1233/maba-v1.5-exp-architecture](https://huggingface.co/AndrewThompson1233/maba-v1.5-exp-architecture)
-* Parameters: **103.5M**
-* Core Ratio: **95.21%** (4.30% Vocab Tax)
-* Macro-Stack: **3:1** (15 DGDA Recurrence : 5 MABA-SA Attention)
+* Reference Architecture: [AndrewThompson1233/maba-v1.5-exp-architecture](https://huggingface.co/AndrewThompson1233/maba-v1.5-exp-architecture)
+* Parameters: **103,520,911 (103.5M)**
+* Core Computation Ratio: **95.21%** (4.30% Vocab Tax)
+* Topology: **3:1** (15 DGDA Recurrence : 5 MABA-SA Attention)
 * Positional Encoding: **Strict NoPE** (0 parameters)
-* Trained on: 3,044 dialogues on NVIDIA L4 (bfloat16)
+* Training Setup: 3,044 dialogues, 15 epochs on NVIDIA L4 (bfloat16)
 
-## Empirical Benchmark vs Qwen3.8-Flash-Next (101.7M)
+---
+
+## Empirical Benchmark: Maba v1.5 vs Qwen3.8-Flash-Next (~103M)
+
+Evaluated under identical training and hardware budgets (3,044 dialogues, bfloat16, NVIDIA L4):
 
-| Metric | Maba v1.5-exp | Qwen3.8-Flash-Next | Delta |
+| Benchmark / Architecture Metric | 🔵 Maba v1.5-exp | 🟣 Qwen3.8-Flash-Next | Advantage / Delta |
 | :--- | :---: | :---: | :---: |
-| **Retrieval Accuracy (MCQ)** | **87.5% (7/8)** | 75.0% (6/8) | **+12.5%** |
-| **Validation Loss** | **0.0697** | 0.0778 | **-10.4%** |
-| **Perplexity (PPL)** | **1.07** | 1.08 | **Maba wins** |
-| **Decoding Speed (L4)** | **7.0 tok/s** | 5.5 tok/s | **+27.3% faster** |
+| **Total Parameters** | **103,520,911 (103.5M)** | 101,701,120 (101.7M) | 0.2% parity |
+| **Core Computation Ratio** | **95.21%** | 74.99% | **+20.2% more active compute** |
+| **Vocab Tax** | **4.30%** (Factorized) | 25.01% (Direct) | **-20.7% parameter bloat** |
+| **Recurrence Engine (75%)** | **DGDA (Decoupled)** | GDN (Standard) | Decoupled erase/write gates |
+| **Attention Engine (25%)** | **MABA-SA (MLA + Top-32)** | QSA (GQA + Micro-block) | 80% KV latent compression |
+| **Positional Encoding** | **Strict NoPE** | 25% Partial RoPE | 0 positional parameters |
+| **Contrastive Retrieval (MCQ)** | **87.5% (7/8)** | 75.0% (6/8) | **+12.5% accuracy** |
+| **Validation Loss** | **0.0697** | 0.0778 | **-10.4% entropy** |
+| **Validation Perplexity (PPL)** | **1.07** | 1.08 | **Maba wins** |
+| **Attention Ablation (PPL Drop)** | **-29.2%** (51.21 -> 36.24) | -5.7% (51.21 -> 48.29) | **MABA-SA cuts error by 29%** |
+| **Needle-in-a-Haystack (4k)** | **91.7% (11/12)** | Lost-in-Middle (0.0) | Hybrid pooling prevents dilution |
+| **KV-Cache @ 4k Context** | **2.50 MB** | 8.00 MB | **-98.8% vs Dense (200 MB)** |
+| **Decode Throughput (L4)** | **7.0 tok/s** | 5.5 tok/s | **+27.3% faster generation** |
 
 <p align="center"><img src="assets/architecture_comparison.svg" width="920" alt="Benchmark" /></p>
 
+---
+
 ## How to Use
 
 ```python
@@ -38,6 +55,7 @@ from safetensors.torch import load_file
 from transformers import AutoTokenizer
 
 tokenizer = AutoTokenizer.from_pretrained('gpt2')
+
 if 'maba-v1.5-exp-architecture' not in sys.path:
     subprocess.run(['git', 'clone', 'https://huggingface.co/AndrewThompson1233/maba-v1.5-exp-architecture'], check=False)
     sys.path.insert(0, 'maba-v1.5-exp-architecture')