Text Generation
memra
GGUF
nvfp4
speculative-decoding
mtp
conversational
blackwell
qwen3
Eval Results (legacy)
Instructions to use tiyuvta/Qwen3.8-27B-NVFP4-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- memra
How to use tiyuvta/Qwen3.8-27B-NVFP4-MTP-GGUF with memra:
# memra serves NVIDIA Blackwell workstation and consumer cards (sm_120a), with a # compile-gated Hopper lane. Prebuilt binaries need Linux x86_64 and driver 580+, # and no CUDA toolkit. curl -fsSL https://raw.githubusercontent.com/avifenesh/memra/main/tools/install.sh | sh
# One chat-templated generation. In a repo with several GGUF files, append # :<substring> to choose one, for example hf:tiyuvta/Qwen3.8-27B-NVFP4-MTP-GGUF:Q4_K_M MEMRA_CHAT=1 run-gen hf:tiyuvta/Qwen3.8-27B-NVFP4-MTP-GGUF --prompt "Explain KV caches in one sentence."
# Or an OpenAI-compatible server on 127.0.0.1:8080. MEMRA_MODELS="model=hf:tiyuvta/Qwen3.8-27B-NVFP4-MTP-GGUF" memra-server
- Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -50,6 +50,20 @@ masked head — see
|
|
| 50 |
model's *own generations* (163k-token corpus over real agentic session
|
| 51 |
prompts; external text used as prompts only — the ranked distribution is
|
| 52 |
100% model-generated). Ranks artifact: `q38-ranks-sxc32768.gguf(.txt)`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
- **Source model:** ranks generated by, and head extracted from, THIS repo's
|
| 54 |
`Qwen3.8-27B-NVFP4-Q5K-mtp.gguf` (minted from
|
| 55 |
[Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) via the
|
|
|
|
| 50 |
model's *own generations* (163k-token corpus over real agentic session
|
| 51 |
prompts; external text used as prompts only — the ranked distribution is
|
| 52 |
100% model-generated). Ranks artifact: `q38-ranks-sxc32768.gguf(.txt)`.
|
| 53 |
+
- **Flavors (2026-08-15):** three ranks corpora, pick by workload — each ships as a
|
| 54 |
+
ranks list (`.txt`, drives memra's load-time trim on safetensors trunks via
|
| 55 |
+
`MEMRA_FRSPEC_TRIM`) and as a pre-trimmed draft head (`.gguf`, serve with
|
| 56 |
+
`MEMRA_MTP_DRAFT`):
|
| 57 |
+
| flavor | files | corpus |
|
| 58 |
+
|---|---|---|
|
| 59 |
+
| **agentic** (serving default) | `q38-ranks-sxc32768.gguf(.txt)` + `mtp-...frspec-sxc32768.gguf` | 163k own-generated tokens over real agentic sessions |
|
| 60 |
+
| **prose** | `q38-ranks-prose-32768.{gguf,txt}` + `mtp-...frspec-prose32768.gguf` | 154k own-generated tokens over essay/story/letter prompts, ~15% non-English |
|
| 61 |
+
| **mixed** | `q38-ranks-mixed-32768.txt` + `mtp-...frspec-mixed32768.gguf` | 50/50 normalized count-blend of both streams (same rank law) |
|
| 62 |
+
|
| 63 |
+
A mask can never change OUTPUT (verification is the target's job on the full
|
| 64 |
+
vocab); it only moves draft acceptance. Short generic probes measure the three
|
| 65 |
+
within noise of each other (acceptance 0.40 prose-text / 0.59-0.62 code); the
|
| 66 |
+
differences live in domain tail tokens.
|
| 67 |
- **Source model:** ranks generated by, and head extracted from, THIS repo's
|
| 68 |
`Qwen3.8-27B-NVFP4-Q5K-mtp.gguf` (minted from
|
| 69 |
[Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) via the
|