# Fuse3 llama.cpp Integration

Full llama.cpp support for Fuse3 (fuse-1 Lite) — LFM2 host + Qwen3.6 coding experts.

## Files

| File | Purpose |
|------|---------|
| `fuse3.cpp` | C++ graph builder (place in `src/models/`) |
| `fuse3.h` | C++ header (place in `src/models/`) |
| `fuse3_converter.py` | Python GGUF converter (place in `conversion/`) |
| `export_gguf.py` | Standalone GGUF exporter (no llama.cpp fork needed to create GGUF) |
| `INTEGRATION.md` | Step-by-step patches for llama.cpp source files |

## Quick Start

### 1. Create the GGUF file (no fork needed)

```bash
pip install gguf safetensors torch
python export_gguf.py --model-dir /path/to/fuse-1-Lite --output fuse-1-Lite-f16.gguf --outtype f16
```

### 2. Build llama.cpp with Fuse3 support

```bash
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp

# Apply patches (see INTEGRATION.md)
# Copy fuse3.cpp and fuse3.h to src/models/
# Copy fuse3_converter.py to conversion/
# Modify src/llama-arch.h, src/llama-arch.cpp, src/llama-model.cpp, src/CMakeLists.txt

# Build
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release
```

### 3. Run inference

```bash
./build/bin/llama-cli \
  -m fuse-1-Lite-f16.gguf \
  -p "Write a Python function to check if a number is prime." \
  -n 512 --temp 0.1
```

## Architecture

Fuse3 extends llama.cpp's native LFM2 implementation:

- **Host layers**: Reuses LFM2's attention (`Lfm2AttentionDecoderLayer`) and short-conv (`Lfm2ShortConvDecoderLayer`) graph builders
- **Dense FFN**: Standard LFM2 SwiGLU FFN (always dense, not MoE)
- **Expert augmentation**: After each augmented layer's FFN, a custom MoE block runs:
  1. Router: `sqrtsoftplus(x @ router_weight)` → top-k selection → normalize
  2. Experts: Per-expert SwiGLU FFN (gate, up, down projections)
  3. Scale: `softplus(expert_scale)` clamped to 0.1
  4. Add: `host_output + scale * expert_output`

## GGUF Tensor Names

### Standard LFM2 tensors (host)
- `token_embd.weight`, `token_embd_norm.weight`, `output.weight`
- `blk.{i}.attn_norm.weight`, `blk.{i}.attn_q/k/v.weight`, `blk.{i}.attn_output.weight`
- `blk.{i}.attn_q_norm.weight`, `blk.{i}.attn_k_norm.weight`
- `blk.{i}.ffn_norm.weight`, `blk.{i}.ffn_gate/up/down.weight`
- `blk.{i}.shortconv_conv/inproj/outproj.weight`

### Fuse3-specific tensors
- `blk.{i}.fuse3_router.weight` — `{n_embd, n_exp}` router gate
- `blk.{i}.fuse3_expert_scale.weight` — `{1}` learned scale parameter
- `blk.{i}.fuse3_experts.gate.weight` — `{n_embd, n_ff_exp, n_exp}` stacked expert gates
- `blk.{i}.fuse3_experts.up.weight` — `{n_embd, n_ff_exp, n_exp}` stacked expert up projections
- `blk.{i}.fuse3_experts.down.weight` — `{n_ff_exp, n_embd, n_exp}` stacked expert down projections

### Custom GGUF metadata keys
- `fuse3.augmented_layers` — array of layer indices with experts
- `fuse3.expert_counts` — per-layer expert count array
- `fuse3.swiglu_limit` — float, expert activation clamp (default 10.0)
- `fuse3.expert_scale_init` — float, initial scale (not used at inference)

## Quantization

The GGUF file can be quantized using llama.cpp's standard quantization tools:

```bash
# Q4_K_M (recommended)
./build/bin/llama-quantize fuse-1-Lite-f16.gguf fuse-1-Lite-q4_k_m.gguf q4_k_m

# Q8_0
./build/bin/llama-quantize fuse-1-Lite-f16.gguf fuse-1-Lite-q8_0.gguf q8_0
```

Note: Only the host LFM2 weights are quantized. The expert weights are kept in f16
for quality (they're small — 512 intermediate × ~30 experts × 30 layers ≈ 200MB).