Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -23,10 +23,32 @@ widget:
|
|
| 23 |
|
| 24 |
# Qwen3-0.6B-Particle-SousVide-R128-Perfect
|
| 25 |
|
|
|
|
|
|
|
| 26 |
**Qwen3-0.6B + 6 custom technologies — gguf2bin runtime (C99, AVX2, Vulkan)**
|
| 27 |
|
| 28 |
> **Perfect r128 20k** · **1.00 loss** (3.52→1.00) · **62.7% SecEval** · **520k** lines 465MB (110 shards `rdru200m` 6.9GB + `code_search_net` 20k)
|
| 29 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
### Technologies
|
| 31 |
|
| 32 |
| Tech | Combines | What it does |
|
|
|
|
| 23 |
|
| 24 |
# Qwen3-0.6B-Particle-SousVide-R128-Perfect
|
| 25 |
|
| 26 |
+
> **⚡ 40.1 tok/s on your CPU (i5-6200U 2C/4T) + 37 MB RAM — gguf2bin interface**
|
| 27 |
+
|
| 28 |
**Qwen3-0.6B + 6 custom technologies — gguf2bin runtime (C99, AVX2, Vulkan)**
|
| 29 |
|
| 30 |
> **Perfect r128 20k** · **1.00 loss** (3.52→1.00) · **62.7% SecEval** · **520k** lines 465MB (110 shards `rdru200m` 6.9GB + `code_search_net` 20k)
|
| 31 |
|
| 32 |
+
### ⚡ Speed on YOUR CPU + RAM (gguf2bin)
|
| 33 |
+
|
| 34 |
+
**Your CPU (i5-6200U 2C/4T DDR3L 9.4 GB/s) — measured `bench -n32 min3` with `--fast`:**
|
| 35 |
+
|
| 36 |
+
| Model | Weights (mmap) | Runtime RAM | decode | prefill | Your RAM knob |
|
| 37 |
+
|-------|----------------|-------------|--------|---------|---------------|
|
| 38 |
+
| **Qwen3-0.6B Q4** | 339 MB | **511 MB** | **24.7** | 38.6 | `--q8-kv` → 3.8× less, `-c 512` → 139 MB |
|
| 39 |
+
| **+MV 0.5** | 339 MB | 511 MB | **40.1 (+62%)** | 47.9 | `--mv 0.5` |
|
| 40 |
+
| **+BVH 15%** | 339 MB | 511 MB | 24.8 (2.5× on ctx32k) | — | `--bvh` |
|
| 41 |
+
| **+Particle-SousVide** | 339 MB | 511 MB | 24.7 | 38.6 | train 5× faster |
|
| 42 |
+
| **2GB machine** | 339 MB | **37 MB** | 24.7 | — | `--swap D:\kv.swap` or `--max-ram 2048` |
|
| 43 |
+
|
| 44 |
+
**What counts on 2GB?** Weights = mmap evictable (❌), KV cache = `F32→Q8_0` (`--q8-kv`), buffers + tokenizer ~60 MB (✅). `model.g2bx` never loads to RAM, only `KV + 37 MB`.
|
| 45 |
+
|
| 46 |
+
```bash
|
| 47 |
+
gguf2bin2_new.exe chat model.g2bx --cyber adapter.lora --mv 0 --fast # 24.7 t/s, 511 MB
|
| 48 |
+
gguf2bin2_new.exe chat model.g2bx --cyber adapter.lora --mv 0.5 --fast # 40.1 t/s, same RAM
|
| 49 |
+
gguf2bin2_new.exe run model.g2bx "Hello" --max-ram 2048 # auto Q8 + ctx halved → 37 MB
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
### Technologies
|
| 53 |
|
| 54 |
| Tech | Combines | What it does |
|