Upload 9 files

.gitattributes

@@ -34,3 +34,11 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
+assets/dark/banner.png filter=lfs diff=lfs merge=lfs -text
+assets/dark/chart.png filter=lfs diff=lfs merge=lfs -text
+assets/dark/pipeline.png filter=lfs diff=lfs merge=lfs -text
+assets/dark/vectors.png filter=lfs diff=lfs merge=lfs -text
+assets/light/banner.png filter=lfs diff=lfs merge=lfs -text
+assets/light/chart.png filter=lfs diff=lfs merge=lfs -text
+assets/light/pipeline.png filter=lfs diff=lfs merge=lfs -text
+assets/light/vectors.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,319 +1,281 @@
----
-license: apache-2.0
-base_model:
-  - XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
-  - ornith-ai/Ornith-1.5-9B
-tags:
-  - merge
-  - agsi
-  - qwen
-  - qwen3_5
-  - reasoning
-  - coding
-  - agentic
-  - terminal-use
-  - swe-bench
-  - tool-use
-language:
-  - en
-  - zh
-pipeline_tag: text-generation
-library_name: transformers
----
-
-<div align="center">
-
-# 🌌 MiMo-Ornith-9B-AGSI
-### High-Order Manifold Fusion for Autonomous Coding, Reasoning & Terminal Execution
-
-[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
-[![Architecture](https://img.shields.io/badge/Architecture-Qwen%209B%20Hybrid-orange.svg)](https://huggingface.co/models?other=qwen)
-[![Method](https://img.shields.io/badge/Merge%20Method-AGSI%20(DoRA--SLERP)-blueviolet.svg)](#-the-mathematics-of-agsi)
-[![Context Length](https://img.shields.io/badge/Context-131%2C072%20Tokens-green.svg)](#-deployment--inference)
-
-</div>
-
----
-
-## 📌 Executive Summary
-
-**MiMo-Ornith-9B-AGSI** is a non-linear parameter-space synthesis of two leading fine-tuned models derived from the Qwen 9B architecture:
-
-* **[XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B)**: A distillation checkpoint optimized for dense mathematical deduction, SWE-bench verified programmatic problem-solving, and long-horizon chain-of-thought (CoT) reasoning.
-* **[ornith-ai/Ornith-1.5-9B](https://huggingface.co/ornith-ai/Ornith-1.5-9B)**: A reinforcement-learning-driven agentic model specializing in terminal/CLI mastery, autonomous bash execution, self-debugging loops, and GrandCode algorithmic generation.
-
-Rather than relying on naive linear averaging (LERP) or global spherical interpolation (SLERP)—which flatten parameter tensors into uncalibrated vectors—this model was fused using **Adaptive Geodesic Spectral Interpolation (AGSI)**. AGSI executes row-wise hyperspherical geodesics on decoupled directional manifolds, enforces second-order spectral energy conservation, shields against anti-phase gradient interference, and dynamically modulates parameters across depth via a $C^2$-continuous quintic smoothstep curve.
-
----
-
-## 🔬 The Mathematics of AGSI
-
-Standard model interpolation techniques often suffer from:
-1. **Frobenius Attenuation**: Convex parameter averaging causes systematic shrinkage of matrix norms ($\|(1-t)W_A + tW_B\|_F < \|W\|_F$), resulting in signal degradation across 32 transformer layers.
-2. **Isotropic Collapsing**: Flattening multi-head projections into a single 1D vector treats distinct semantic subspaces as an isotropic sphere, corrupting specialized attention head alignments.
-3. **Anti-Phase Annihilation**: Conflicting updates between reinforcement learning (Ornith) and knowledge distillation (MiMo) cause destructive cancellation when vectors point in opposing directions ($\cos\theta < 0$).
-
-AGSI resolves these issues through a five-stage manifold interpolation framework:
-
-```
-                  ┌─────────────────────────────────────────┐
-                  │W_A, W_B (Weight Matrices in R^(DxK)) │
-                  └────────────────────┬────────────────────┘
-                                     │
-                         [ 1. DoRA-Style Decoupling ]
-                                     │
-                  ┌────────────────────┴────────────────────────┐
-                  ▼                                         ▼
-        m_A, m_B = ||W||_row                      U_A, U_B = W / ||W||_row
-        (Radial Magnitudes)                       (Unit Hypersphere S^(D-1))
-                  │                                         │
-                  │                        [ 2. Row-wise Geodesic SLERP ]
-                  │                                         │
-                  │                          θ_i = arccos(clamp(u_A · u_B))
-                  │                                         │
-                  │                        [ 3. Anti-Phase Dominance Gate ]
-                  │                                         │
-                  │                          If cos(θ) < -0.05:
-                  │                            Route to dominant feature
-                  │                                         │
-                  │                                         ▼
-                  │                                    U_fused ∈ S^(D-1)
-                  │                                         │
-                  └──────────────────────┬──────────────────────┘
-                                       │
-                  [ 4. Quadratic Spectral Energy Invariant ]
-                     m_target = sqrt((1-t)m_A^2 + t*m_B^2)
-                                       │
-                                       ▼
-                             W_cand = m_target ⊙ U_fused
-                                       │
-                   [ 5. Global Frobenius Norm Conservation ]
-                     W_final = W_cand * (F_target / F_cand)
-```
-
----
-
-### 1. Direction-Magnitude (DoRA) Decoupling
-
-Linear layers compute transformations $y = x W^T$, where individual rows $W_i \in \mathbb{R}^{D_{\text{in}}}$ represent the hyperplanes of specific neurons. AGSI isolates radial feature scale from directional orientation:
-
-$$m^{(i)} = \|W^{(i)}\|_2 = \sqrt{\sum_{j=1}^{D_{\text{in}}} (W_{ij})^2}$$
-
-$$u^{(i)} = \frac{W^{(i)}}{m^{(i)} + \epsilon}, \quad \text{where } u^{(i)} \in S^{D_{\text{in}}-1}$$
-
-By constraining directional updates to the unit hypersphere $S^{D_{\text{in}}-1}$, the model preserves the angular separation of neuron receptive fields.
-
----
-
-### 2. Row-Wise Hyperspherical Geodesics ($S^{D-1}$)
-
-For each individual neuron row $i$, the angular geodesic distance between checkpoint trajectories is calculated directly in its tangent space:
-
-$$\theta_i = \arccos\left(\text{clamp}\left(\langle u_A^{(i)}, u_B^{(i)} \rangle, -1 + \delta, 1 - \delta\right)\right)$$
-
-Rather than using Euclidean displacement, the directional basis moves along the great-circle arc:
-
-$$u_{\text{fused}}^{(i)} = c_A^{(i)} u_A^{(i)} + c_B^{(i)} u_B^{(i)}$$
-
-where the geodesic velocity coefficients are defined as:
-
-$$c_A^{(i)} = \frac{\sin((1 - t)\theta_i)}{\sin\theta_i + \epsilon}, \quad c_B^{(i)} = \frac{\sin(t \theta_i)}{\sin\theta_i + \epsilon}$$
-
-If $\sin\theta_i \to 0$ (quasi-collinear vectors), the interpolation smoothly transitions to normalized linear blending:
-
-$$\lim_{\theta \to 0} u_{\text{fused}}^{(i)} = (1 - t)u_A^{(i)} + t u_B^{(i)}$$
-
----
-
-### 3. Anti-Phase Interference Shielding
-
-When RL optimization (Ornith) and distillation gradients (MiMo) pull in opposing directions ($\theta_i > 90^\circ, \cos\theta_i < 0$), standard spherical combination results in substantial cancellation:
-
-$$\|u_A + u_B\| = \sqrt{2 + 2\cos\theta} < \sqrt{2} \approx 1.414 \quad (\text{vs. } 2.0 \text{ when aligned})$$
-
-AGSI monitors the directional dot product against a critical interference bound ($\tau = -0.05$, corresponding to $\theta \approx 92.86^\circ$). If destructive cancellation occurs:
-
-$$\text{Conflict Condition: } \langle u_A^{(i)}, u_B^{(i)} \rangle < \tau$$
-
-$$\gamma_A^{(i)} = \frac{m_A^{(i)}}{m_A^{(i)} + m_B^{(i)} + \epsilon}$$
-
-$$u_{\text{fused}}^{(i)} = \begin{cases} 
-0.85\, u_A^{(i)} + 0.15\, u_B^{(i)}, & \text{if } \gamma_A^{(i)} \ge 0.5 \\
-0.15\, u_A^{(i)} + 0.85\, u_B^{(i)}, & \text{if } \gamma_A^{(i)} < 0.5 
-\end{cases}$$
-
-This dominance gate projects conflicting features toward the checkpoint exhibiting higher parameter variance, preventing the formation of dormant neurons.
-
----
-
-### 4. Quadratic Spectral Energy Invariant
-
-In deep autoregressive networks normalized by RMSNorm, layer output variance relies heavily on parameter norm preservation:
-
-$$\mathbb{E}[\|W x\|_2^2] \approx \frac{1}{D_{\text{in}}} \|W\|_F^2 \, \mathbb{E}[\|x\|_2^2]$$
-
-To maintain stable forward-pass activations without logit saturation or signal decay, AGSI scales the interpolated direction vector by the root-mean-square energy:
-
-$$m_{\text{target}}^{(i)} = \sqrt{(1 - t)(m_A^{(i)})^2 + t(m_B^{(i)})^2}$$
-
-$$W_{\text{candidate}}^{(i)} = m_{\text{target}}^{(i)} \cdot \frac{u_{\text{fused}}^{(i)}}{\|u_{\text{fused}}^{(i)}\|_2}$$
-
-Finally, a global Frobenius norm correction is applied across the full matrix:
-
-$$W_{\text{final}} = W_{\text{candidate}} \times \left( \frac{\sqrt{(1 - t)\|W_A\|_F^2 + t\|W_B\|_F^2}}{\|W_{\text{candidate}}\|_F + \epsilon} \right)$$
-
----
-
-### 5. Quintic Smoothstep Depth & Functional Block Routing
-
-The mixing coefficient $t$ is non-static. It is governed by a $C^2$-continuous quintic polynomial function across the 32 transformer layers, augmented by functional block biases:
-
-$$\xi = \frac{l}{L - 1} \in [0, 1], \quad l \in \{0, 1, \dots, 31\}$$
-
-$$S(\xi) = 6\xi^5 - 15\xi^4 + 10\xi^3$$
-
-$$t(l, \text{block}) = \text{clamp}\left(t_{\text{base}} + \Delta t \cdot \left(S(\xi) - 0.5\right) + \beta_{\text{block}}, \, 0.0, \, 1.0\right)$$
-
-```
-  Mixing Ratio (t) -> Weight of Ornith-1.5
-  1.00 ┤                                                    
-  0.75 ┤                                                    
-  0.55 ┤                                   ╭──────────────  (Ornith Terminal Routing)
-  0.48 ┤                 ╭──────────────────╯               (Base Balance)
-  0.41 ┤  ────────────────╯                                   (MiMo Reasoning Anchor)
-  0.00 ┼───┬──────────────┬──────────────────┬──────────────┬
-      L0  L7            L16                L24            L31  (Depth)
-```
-
-#### Layer Profile Breakdown:
-* **Layers 0–7 ($t \approx 0.41 - 0.44$) — Syntactic Anchoring**: Biased toward MiMo-V2.6 to preserve input parsing, stable token embeddings, and baseline representations.
-* **Layers 8–23 ($t \approx 0.45 - 0.51$) — Cognitive & Algorithmic Core**: Near-equilibrium blending. Mathematical deductions and algorithm planning synthesize with Ornith's tool-use reasoning.
-* **Layers 24–31 ($t \approx 0.52 - 0.55$) — Action Policy & Output Heads**: Biased toward Ornith-1.5 to prioritize terminal command generation, tool-calling syntax, and execution policies.
-* **Attention vs. MLP Decoupling**:
-  * **Attention Projections (`q, k, v, o, linear_attn`)**: $\beta_{\text{attn}} = +0.04$ (Ornith priority for context selection and environment state tracking).
-  * **MLP / Feed-Forward (`gate, up, down_proj`)**: $\beta_{\text{mlp}} = -0.04$ (MiMo priority for factual retention and associative coding memories).
-
----
-
-## ⚙️ AGSI Configuration Profile
-
-```python
-# The Golden SOTA Preset applied during parameter synthesis
-BASE_ORNITH_RATIO        = 0.48   # Foundational balance (52% MiMo / 48% Ornith)
-DEPTH_MODULATION         = 0.14   # Dynamic span across transformer depth
-ATTN_ROUTING_BIAS        = 0.04   # Attention block offset (favors Ornith)
-MLP_ROUTING_BIAS         = -0.04  # MLP block offset (favors MiMo)
-ANTI_PHASE_THRESHOLD     = -0.05  # Critical angle cutoff (θ = 92.86°)
-SPECTRAL_ENERGY_EXPONENT = 2.0    # Second-order L2 moment conservation
-```
-
----
-
-## 🚀 Deployment & Inference
-
-This model is compatible with systems supporting the Qwen 9B architecture (e.g., standard Hugging Face `transformers`, `vLLM`, `SGLang`).
-
-### Serving with vLLM (Recommended)
-
-To run the model with vLLM, including support for streaming reasoning tokens (`<think>`) and agentic tool calls:
-
-```bash
-vllm serve DEST_REPO_ID \
-  --port 8000 \
-  --model DEST_REPO_ID \
-  --trust-remote-code \
-  --tensor-parallel-size 1 \
-  --max-model-len 131072 \
-  --gpu-memory-utilization 0.92 \
-  --reasoning-parser qwen3 \
-  --tool-call-parser hermes
-```
-
-### Python Inference via Transformers
-
-```python
-import torch
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-model_id = "DEST_REPO_ID"
-
-tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
-model = AutoModelForCausalLM.from_pretrained(
-    model_id,
-    torch_dtype=torch.bfloat16,
-    device_map="auto",
-    trust_remote_code=True
-)
-
-messages = [
-    {
-        "role": "system",
-        "content": "You are a master systems engineer and competitive programmer. Solve problems using precise step-by-step reasoning enclosed in <think> tags, then execute terminal commands or output production code."
-    },
-    {
-        "role": "user",
-        "content": "Write an optimized eBPF program in C that traces network socket latency outliers (>100ms) and provides a Python BCC analysis script."
-    }
-]
-
-prompt = tokenizer.apply_chat_template(
-    messages,
-    tokenize=False,
-    add_generation_prompt=True
-)
-
-inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
-outputs = model.generate(
-    **inputs,
-    max_new_tokens=4096,
-    temperature=0.6,
-    top_p=0.95,
-    repetition_penalty=1.05
-)
-
-response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
-print(response)
-```
-
----
-
-## 🎯 Prompt Formatting & Chat Template
-
-This model uses the unified Qwen chat template, configured to separate internal reasoning steps from final outputs:
-
-```xml
-<|im_start|>system
-You are a helpful assistant.<|im_end|>
-<|im_start|>user
-Write a bash script to monitor memory usage.<|im_end|>
-<|im_start|>assistant
-<think>
-1. Identify target metrics: available vs. used memory.
-2. Use /proc/meminfo or 'free -m' for portability.
-3. Handle logging and alert thresholds.
-</think>
-Here is the monitoring script:
-```
-
-```bash
-#!/usr/bin/env bash
-set -euo pipefail
-
-THRESHOLD=85
-CURRENT=$(free | awk '/Mem:/ {printf("%.0f"), $3/$2 * 100}')
-
-if [ "$CURRENT" -gt "$THRESHOLD" ]; then
-    echo "WARNING: Memory usage at ${CURRENT}%"
-fi
-```
-`<|im_end|>`
-
----
-
-## ⚖️ License & Attribution
-
-* **Base Model Checkpoints**:
-  * Checkpoint A: [XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B)
-  * Checkpoint B: [ornith-ai/Ornith-1.5-9B](https://huggingface.co/ornith-ai/Ornith-1.5-9B)
-* **Underlying Architecture**: Qwen Series (Alibaba Cloud)
-* **License**: Apache 2.0
+---
+license: apache-2.0
+base_model:
+  - XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
+  - ornith-ai/Ornith-1.5-9B
+tags:
+  - merge
+  - agsi
+  - qwen
+  - qwen3_5
+  - reasoning
+  - coding
+  - agentic
+  - terminal-use
+  - swe-bench
+  - tool-use
+language:
+  - en
+  - zh
+pipeline_tag: text-generation
+library_name: transformers
+---
+
+![MiMo-Ornith-9B-AGSI](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/dark/banner.png#hf-dark-mode-only)
+![MiMo-Ornith-9B-AGSI](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/light/banner.png#hf-light-mode-only)
+
+<div align="center">
+
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
+[![Architecture](https://img.shields.io/badge/Architecture-Qwen%209B%20Hybrid-orange.svg)](https://huggingface.co/models?other=qwen)
+[![Method](https://img.shields.io/badge/Merge%20Method-AGSI%20(DoRA--SLERP)-blueviolet.svg)](#-the-mathematics-of-agsi)
+[![Context Length](https://img.shields.io/badge/Context-131%2C072%20Tokens-green.svg)](#-deployment--inference)
+
+</div>
+
+---
+
+## 📌 Executive Summary
+
+**MiMo-Ornith-9B-AGSI** is a non-linear parameter-space synthesis of two leading fine-tuned models derived from the Qwen 9B architecture:
+
+* **[XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B)**: A distillation checkpoint optimized for dense mathematical deduction, SWE-bench verified programmatic problem-solving, and long-horizon chain-of-thought (CoT) reasoning.
+* **[ornith-ai/Ornith-1.5-9B](https://huggingface.co/ornith-ai/Ornith-1.5-9B)**: A reinforcement-learning-driven agentic model specializing in terminal/CLI mastery, autonomous bash execution, self-debugging loops, and GrandCode algorithmic generation.
+
+Rather than relying on naive linear averaging (LERP) or global spherical interpolation (SLERP)—which flatten parameter tensors into uncalibrated vectors—this model was fused using **Adaptive Geodesic Spectral Interpolation (AGSI)**. AGSI executes row-wise hyperspherical geodesics on decoupled directional manifolds, enforces second-order spectral energy conservation, shields against anti-phase gradient interference, and dynamically modulates parameters across depth via a $C^2$-continuous quintic smoothstep curve.
+
+---
+
+## 🔬 The Mathematics of AGSI
+
+Standard model interpolation techniques often suffer from:
+1. **Frobenius Attenuation**: Convex parameter averaging causes systematic shrinkage of matrix norms ($\|(1-t)W_A + tW_B\|_F < \|W\|_F$), resulting in signal degradation across 32 transformer layers.
+2. **Isotropic Collapsing**: Flattening multi-head projections into a single 1D vector treats distinct semantic subspaces as an isotropic sphere, corrupting specialized attention head alignments.
+3. **Anti-Phase Annihilation**: Conflicting updates between reinforcement learning (Ornith) and knowledge distillation (MiMo) cause destructive cancellation when vectors point in opposing directions ($\cos\theta < 0$).
+
+AGSI resolves these issues through a five-stage manifold interpolation framework:
+
+![AGSI pipeline](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/dark/pipeline.png#hf-dark-mode-only)
+![AGSI pipeline](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/light/pipeline.png#hf-light-mode-only)
+
+---
+
+### 1. Direction-Magnitude (DoRA) Decoupling
+
+Linear layers compute transformations $y = x W^T$, where individual rows $W_i \in \mathbb{R}^{D_{\text{in}}}$ represent the hyperplanes of specific neurons. AGSI isolates radial feature scale from directional orientation:
+
+$$m^{(i)} = \|W^{(i)}\|_2 = \sqrt{\sum_{j=1}^{D_{\text{in}}} (W_{ij})^2}$$
+
+$$u^{(i)} = \frac{W^{(i)}}{m^{(i)} + \epsilon}, \quad \text{where } u^{(i)} \in S^{D_{\text{in}}-1}$$
+
+By constraining directional updates to the unit hypersphere $S^{D_{\text{in}}-1}$, the model preserves the angular separation of neuron receptive fields.
+
+---
+
+### 2. Row-Wise Hyperspherical Geodesics ($S^{D-1}$)
+
+For each individual neuron row $i$, the angular geodesic distance between checkpoint trajectories is calculated directly in its tangent space:
+
+$$\theta_i = \arccos\left(\text{clamp}\left(\langle u_A^{(i)}, u_B^{(i)} \rangle, -1 + \delta, 1 - \delta\right)\right)$$
+
+Rather than using Euclidean displacement, the directional basis moves along the great-circle arc:
+
+$$u_{\text{fused}}^{(i)} = c_A^{(i)} u_A^{(i)} + c_B^{(i)} u_B^{(i)}$$
+
+where the geodesic velocity coefficients are defined as:
+
+$$c_A^{(i)} = \frac{\sin((1 - t)\theta_i)}{\sin\theta_i + \epsilon}, \quad c_B^{(i)} = \frac{\sin(t \theta_i)}{\sin\theta_i + \epsilon}$$
+
+If $\sin\theta_i \to 0$ (quasi-collinear vectors), the interpolation smoothly transitions to normalized linear blending:
+
+$$\lim_{\theta \to 0} u_{\text{fused}}^{(i)} = (1 - t)u_A^{(i)} + t u_B^{(i)}$$
+
+---
+
+### 3. Anti-Phase Interference Shielding
+
+When RL optimization (Ornith) and distillation gradients (MiMo) pull in opposing directions ($\theta_i > 90^\circ, \cos\theta_i < 0$), standard spherical combination results in substantial cancellation:
+
+$$\|u_A + u_B\| = \sqrt{2 + 2\cos\theta} < \sqrt{2} \approx 1.414 \quad (\text{vs. } 2.0 \text{ when aligned})$$
+
+AGSI monitors the directional dot product against a critical interference bound ($\tau = -0.05$, corresponding to $\theta \approx 92.86^\circ$). If destructive cancellation occurs:
+
+$$\text{Conflict Condition: } \langle u_A^{(i)}, u_B^{(i)} \rangle < \tau$$
+
+$$\gamma_A^{(i)} = \frac{m_A^{(i)}}{m_A^{(i)} + m_B^{(i)} + \epsilon}$$
+
+$$u_{\text{fused}}^{(i)} = \begin{cases} 
+0.85\, u_A^{(i)} + 0.15\, u_B^{(i)}, & \text{if } \gamma_A^{(i)} \ge 0.5 \\
+0.15\, u_A^{(i)} + 0.85\, u_B^{(i)}, & \text{if } \gamma_A^{(i)} < 0.5 
+\end{cases}$$
+
+This dominance gate projects conflicting features toward the checkpoint exhibiting higher parameter variance, preventing the formation of dormant neurons.
+
+![Geodesic SLERP vs anti-phase dominance gate](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/dark/vectors.png#hf-dark-mode-only)
+![Geodesic SLERP vs anti-phase dominance gate](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/light/vectors.png#hf-light-mode-only)
+
+---
+
+### 4. Quadratic Spectral Energy Invariant
+
+In deep autoregressive networks normalized by RMSNorm, layer output variance relies heavily on parameter norm preservation:
+
+$$\mathbb{E}[\|W x\|_2^2] \approx \frac{1}{D_{\text{in}}} \|W\|_F^2 \, \mathbb{E}[\|x\|_2^2]$$
+
+To maintain stable forward-pass activations without logit saturation or signal decay, AGSI scales the interpolated direction vector by the root-mean-square energy:
+
+$$m_{\text{target}}^{(i)} = \sqrt{(1 - t)(m_A^{(i)})^2 + t(m_B^{(i)})^2}$$
+
+$$W_{\text{candidate}}^{(i)} = m_{\text{target}}^{(i)} \cdot \frac{u_{\text{fused}}^{(i)}}{\|u_{\text{fused}}^{(i)}\|_2}$$
+
+Finally, a global Frobenius norm correction is applied across the full matrix:
+
+$$W_{\text{final}} = W_{\text{candidate}} \times \left( \frac{\sqrt{(1 - t)\|W_A\|_F^2 + t\|W_B\|_F^2}}{\|W_{\text{candidate}}\|_F + \epsilon} \right)$$
+
+---
+
+### 5. Quintic Smoothstep Depth & Functional Block Routing
+
+The mixing coefficient $t$ is non-static. It is governed by a $C^2$-continuous quintic polynomial function across the 32 transformer layers, augmented by functional block biases:
+
+$$\xi = \frac{l}{L - 1} \in [0, 1], \quad l \in \{0, 1, \dots, 31\}$$
+
+$$S(\xi) = 6\xi^5 - 15\xi^4 + 10\xi^3$$
+
+$$t(l, \text{block}) = \text{clamp}\left(t_{\text{base}} + \Delta t \cdot \left(S(\xi) - 0.5\right) + \beta_{\text{block}}, \, 0.0, \, 1.0\right)$$
+
+![Mixing ratio across transformer depth](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/dark/chart.png#hf-dark-mode-only)
+![Mixing ratio across transformer depth](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/light/chart.png#hf-light-mode-only)
+
+#### Layer Profile Breakdown:
+* **Layers 0–7 ($t \approx 0.41 - 0.44$) — Syntactic Anchoring**: Biased toward MiMo-V2.6 to preserve input parsing, stable token embeddings, and baseline representations.
+* **Layers 8–23 ($t \approx 0.45 - 0.51$) — Cognitive & Algorithmic Core**: Near-equilibrium blending. Mathematical deductions and algorithm planning synthesize with Ornith's tool-use reasoning.
+* **Layers 24–31 ($t \approx 0.52 - 0.55$) — Action Policy & Output Heads**: Biased toward Ornith-1.5 to prioritize terminal command generation, tool-calling syntax, and execution policies.
+* **Attention vs. MLP Decoupling**:
+  * **Attention Projections (`q, k, v, o, linear_attn`)**: $\beta_{\text{attn}} = +0.04$ (Ornith priority for context selection and environment state tracking).
+  * **MLP / Feed-Forward (`gate, up, down_proj`)**: $\beta_{\text{mlp}} = -0.04$ (MiMo priority for factual retention and associative coding memories).
+
+---
+
+## ⚙️ AGSI Configuration Profile
+
+```python
+# The Golden SOTA Preset applied during parameter synthesis
+BASE_ORNITH_RATIO        = 0.48   # Foundational balance (52% MiMo / 48% Ornith)
+DEPTH_MODULATION         = 0.14   # Dynamic span across transformer depth
+ATTN_ROUTING_BIAS        = 0.04   # Attention block offset (favors Ornith)
+MLP_ROUTING_BIAS         = -0.04  # MLP block offset (favors MiMo)
+ANTI_PHASE_THRESHOLD     = -0.05  # Critical angle cutoff (θ = 92.86°)
+SPECTRAL_ENERGY_EXPONENT = 2.0    # Second-order L2 moment conservation
+```
+
+---
+
+## 🚀 Deployment & Inference
+
+This model is compatible with systems supporting the Qwen 9B architecture (e.g., standard Hugging Face `transformers`, `vLLM`, `SGLang`).
+
+### Serving with vLLM (Recommended)
+
+To run the model with vLLM, including support for streaming reasoning tokens (`<think>`) and agentic tool calls:
+
+```bash
+vllm serve DEST_REPO_ID \
+  --port 8000 \
+  --model DEST_REPO_ID \
+  --trust-remote-code \
+  --tensor-parallel-size 1 \
+  --max-model-len 131072 \
+  --gpu-memory-utilization 0.92 \
+  --reasoning-parser qwen3 \
+  --tool-call-parser hermes
+```
+
+### Python Inference via Transformers
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "DEST_REPO_ID"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True
+)
+
+messages = [
+    {
+        "role": "system",
+        "content": "You are a master systems engineer and competitive programmer. Solve problems using precise step-by-step reasoning enclosed in <think> tags, then execute terminal commands or output production code."
+    },
+    {
+        "role": "user",
+        "content": "Write an optimized eBPF program in C that traces network socket latency outliers (>100ms) and provides a Python BCC analysis script."
+    }
+]
+
+prompt = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=4096,
+    temperature=0.6,
+    top_p=0.95,
+    repetition_penalty=1.05
+)
+
+response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
+print(response)
+```
+
+---
+
+## 🎯 Prompt Formatting & Chat Template
+
+This model uses the unified Qwen chat template, configured to separate internal reasoning steps from final outputs:
+
+```xml
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+Write a bash script to monitor memory usage.<|im_end|>
+<|im_start|>assistant
+<think>
+1. Identify target metrics: available vs. used memory.
+2. Use /proc/meminfo or 'free -m' for portability.
+3. Handle logging and alert thresholds.
+</think>
+Here is the monitoring script:
+```
+
+```bash
+#!/usr/bin/env bash
+set -euo pipefail
+
+THRESHOLD=85
+CURRENT=$(free | awk '/Mem:/ {printf("%.0f"), $3/$2 * 100}')
+
+if [ "$CURRENT" -gt "$THRESHOLD" ]; then
+    echo "WARNING: Memory usage at ${CURRENT}%"
+fi
+```
+`<|im_end|>`
+
+---
+
+## ⚖️ License & Attribution
+
+* **Base Model Checkpoints**:
+  * Checkpoint A: [XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B)
+  * Checkpoint B: [ornith-ai/Ornith-1.5-9B](https://huggingface.co/ornith-ai/Ornith-1.5-9B)
+* **Underlying Architecture**: Qwen Series (Alibaba Cloud)
+* **License**: Apache 2.0