Upload deploy/DEPLOYMENT.md with huggingface_hub

deploy/DEPLOYMENT.md

@@ -0,0 +1,86 @@
+# Muse-Glimmer-30B-Abliterated-GGUF — Deployment Kit
+
+Serve the abliterated Muse Glimmer 30B locally with **llama.cpp**, optionally with the
+**DFlash** block-diffusion drafter for ~1.6–3× faster decoding (lossless — identical output).
+
+---
+
+## 1. Requirements
+
+- A recent **llama.cpp** build with CUDA/Metal (DFlash support merged in [ggml-org/llama.cpp#26841](https://github.com/ggml-org/llama.cpp/pull/26841)). Build from `master`.
+- Disk for the quant you pick (Q4_K_M ≈ 16 GB, Q8_0 ≈ 28 GB) + the DFlash drafter (≈ 4.8 GB).
+- A GPU with enough VRAM for the quant, or CPU (slower).
+
+## 2. Get the files
+
+Text quant (pick one) + the DFlash drafter, from this repo:
+
+```bash
+hf download Blackfrost-Research/Muse-Glimmer-30B-Abliterated-GGUF \
+  Muse-Glimmer-30B-Abliterated-Q8_0.gguf \
+  dflash-Muse-Glimmer-30B-Abliterated-F16.gguf \
+  --local-dir ./muse
+# for image input, also grab a projector:
+hf download Blackfrost-Research/Muse-Glimmer-30B-Abliterated-GGUF \
+  mmproj-Muse-Glimmer-30B-Abliterated-F16.gguf --local-dir ./muse
+```
+
+## 3. Serve
+
+**Plain (no speculation):**
+```bash
+llama-server -m ./muse/Muse-Glimmer-30B-Abliterated-Q8_0.gguf \
+  -ngl 999 --jinja --host 0.0.0.0 --port 8080 -c 16384 \
+  --temp 1.0 --top-p 0.95 --top-k 64
+```
+
+**With DFlash speculative decoding (recommended — faster, same output):**
+```bash
+llama-server -m ./muse/Muse-Glimmer-30B-Abliterated-Q8_0.gguf \
+  -md ./muse/dflash-Muse-Glimmer-30B-Abliterated-F16.gguf \
+  --spec-type draft-dflash --spec-draft-n-max 15 \
+  -ngl 999 -ngld 999 --jinja --host 0.0.0.0 --port 8080 -c 16384 \
+  --temp 1.0 --top-p 0.95 --top-k 64
+```
+
+- `--spec-type draft-dflash` selects the block-diffusion draft path (block size 16, drafter injects the target's hidden states into its attention). DFlash must run under `llama-server` (it needs the shared target context) — not `llama-cli`.
+- `--spec-draft-n-max 15` is the draft block length (clamped to the trained 16).
+- `-md` = the DFlash drafter file; `-ngld 999` offloads it to GPU too.
+
+**Multimodal (image input):** add the projector to either command:
+```bash
+  --mmproj ./muse/mmproj-Muse-Glimmer-30B-Abliterated-F16.gguf
+```
+
+Or run the ready-made script in this kit: `bash serve.sh` (see `serve.sh`).
+
+## 4. Query (OpenAI-compatible)
+
+```bash
+curl http://localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{
+  "messages":[{"role":"user","content":"Write a binary search in Python."}],
+  "max_tokens": 2048, "temperature": 1.0, "top_p": 0.95, "top_k": 64
+}'
+```
+
+## 5. Important notes
+
+- **Heavy thinker.** Muse spends tokens on a reasoning channel first. Use a **generous `max_tokens` (≥ 1024)** or the answer can come back empty (the budget is consumed by reasoning). llama-server returns the reasoning in `reasoning_content` and the answer in `content`.
+- **Sampling (Meta guidance):** `temperature 1.0, top_p 0.95, top_k 64`. Steer depth with a `Reasoning strength: low/medium/high/xhigh` line in the system prompt.
+- **Flash attention:** `-fa on` is fastest and lets DFlash reach its peak speedup. If your GPU/CUDA combo hangs at load with `-fa on` (seen on some brand-new archs paired with an older CUDA toolkit), use `-fa off` — everything works, at a modest speed cost.
+- **Persona:** ships with an "AI assistant" system template baked in; override with your own system message.
+
+## 6. Reference performance
+
+Single NVIDIA RTX PRO 6000 (Blackwell), Q8_0, `-fa off`:
+
+| config | decode tok/s | speedup |
+|---|--:|--:|
+| baseline | ~46 | 1.0× |
+| + DFlash | ~73 | **1.6×** |
+
+DFlash speedup rises with structured/code output and with `-fa on`. Meta reports up to 3.1× on an RTX 5090.
+
+---
+
+*Built by [Blackfrost](https://x.com/Blackfrost_AI) · Las Vegas, NV. Not affiliated with Meta.*