VoidWalkercero commited on
Commit
156743c
·
verified ·
1 Parent(s): 8f62377

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +22 -0
README.md CHANGED
@@ -23,10 +23,32 @@ widget:
23
 
24
  # Qwen3-0.6B-Particle-SousVide-R128-Perfect
25
 
 
 
26
  **Qwen3-0.6B + 6 custom technologies — gguf2bin runtime (C99, AVX2, Vulkan)**
27
 
28
  > **Perfect r128 20k** · **1.00 loss** (3.52→1.00) · **62.7% SecEval** · **520k** lines 465MB (110 shards `rdru200m` 6.9GB + `code_search_net` 20k)
29
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  ### Technologies
31
 
32
  | Tech | Combines | What it does |
 
23
 
24
  # Qwen3-0.6B-Particle-SousVide-R128-Perfect
25
 
26
+ > **⚡ 40.1 tok/s on your CPU (i5-6200U 2C/4T) + 37 MB RAM — gguf2bin interface**
27
+
28
  **Qwen3-0.6B + 6 custom technologies — gguf2bin runtime (C99, AVX2, Vulkan)**
29
 
30
  > **Perfect r128 20k** · **1.00 loss** (3.52→1.00) · **62.7% SecEval** · **520k** lines 465MB (110 shards `rdru200m` 6.9GB + `code_search_net` 20k)
31
 
32
+ ### ⚡ Speed on YOUR CPU + RAM (gguf2bin)
33
+
34
+ **Your CPU (i5-6200U 2C/4T DDR3L 9.4 GB/s) — measured `bench -n32 min3` with `--fast`:**
35
+
36
+ | Model | Weights (mmap) | Runtime RAM | decode | prefill | Your RAM knob |
37
+ |-------|----------------|-------------|--------|---------|---------------|
38
+ | **Qwen3-0.6B Q4** | 339 MB | **511 MB** | **24.7** | 38.6 | `--q8-kv` → 3.8× less, `-c 512` → 139 MB |
39
+ | **+MV 0.5** | 339 MB | 511 MB | **40.1 (+62%)** | 47.9 | `--mv 0.5` |
40
+ | **+BVH 15%** | 339 MB | 511 MB | 24.8 (2.5× on ctx32k) | — | `--bvh` |
41
+ | **+Particle-SousVide** | 339 MB | 511 MB | 24.7 | 38.6 | train 5× faster |
42
+ | **2GB machine** | 339 MB | **37 MB** | 24.7 | — | `--swap D:\kv.swap` or `--max-ram 2048` |
43
+
44
+ **What counts on 2GB?** Weights = mmap evictable (❌), KV cache = `F32→Q8_0` (`--q8-kv`), buffers + tokenizer ~60 MB (✅). `model.g2bx` never loads to RAM, only `KV + 37 MB`.
45
+
46
+ ```bash
47
+ gguf2bin2_new.exe chat model.g2bx --cyber adapter.lora --mv 0 --fast # 24.7 t/s, 511 MB
48
+ gguf2bin2_new.exe chat model.g2bx --cyber adapter.lora --mv 0.5 --fast # 40.1 t/s, same RAM
49
+ gguf2bin2_new.exe run model.g2bx "Hello" --max-ram 2048 # auto Q8 + ctx halved → 37 MB
50
+ ```
51
+
52
  ### Technologies
53
 
54
  | Tech | Combines | What it does |