Add collapsible H200, RTX 5090, laptop and experimental IQ2_M quickstarts

DEVICE-QUICKSTART.md

@@ -0,0 +1,130 @@
+## 🚀 Start on your device
+
+Choose the setup you actually use. **You only need one edition and one quantization.** The large server uses Transformers; the desktop and laptop examples use smaller GGUF files through llama.cpp or Ollama. These are different ways to run the same release, not four new models.
+
+<details>
+<summary>🖥️ H200 / large server — full BF16 with Transformers</summary>
+
+For a server with enough GPU memory. Validated with **Transformers 5.16.1 on NVIDIA H200**. The weights alone are about **70 GB**; leave additional room for images, cache and intermediate calculations. This example is not suitable for a single 32-GB RTX 5090 or an 8-GB laptop GPU.
+
+Install a CUDA-enabled PyTorch build compatible with your driver using the [official PyTorch installer](https://pytorch.org/get-started/locally/), then:
+
+```bash
+python -m pip install transformers==5.16.1 accelerate pillow
+```
+
+```python
+import torch
+from transformers import AutoProcessor, Qwen3_5MoeForConditionalGeneration
+
+repo = "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT"
+processor = AutoProcessor.from_pretrained(repo)
+model = Qwen3_5MoeForConditionalGeneration.from_pretrained(
+    repo, dtype=torch.bfloat16, device_map="auto"
+)
+messages = [{"role": "user", "content": [
+    {"type": "text", "text": "Explain gravity briefly."}
+]}]
+inputs = processor.apply_chat_template(
+    messages, tokenize=True, add_generation_prompt=True,
+    return_dict=True, return_tensors="pt", enable_thinking=False
+).to(model.device)
+with torch.inference_mode():
+    output = model.generate(**inputs, max_new_tokens=256, do_sample=False)
+print(processor.batch_decode(
+    output[:, inputs.input_ids.shape[1]:], skip_special_tokens=True
+)[0])
+```
+
+For image input, add an image content item supported by `AutoProcessor`. Large images or long conversations need more memory.
+
+</details>
+
+<details>
+<summary>🎮 RTX 5090 · 32 GB — Q4_K_M with llama.cpp</summary>
+
+Use the **GGUF / Llama edition**, not the 70-GB BF16 download. Q4_K_M is about **20.22 GiB**, plus a **0.84-GiB vision projector** and runtime memory. This model/backend combination was benchmarked on RTX 5090. The commands below start a short-context interactive server, not the full benchmark configuration.
+
+Install the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli) and a CUDA-enabled [llama.cpp build](https://github.com/ggml-org/llama.cpp). The release was tested with llama.cpp commit `427291b5b34cd914a31b3fd3b61a68f6184f4b9f`. Run the following in Bash on Ubuntu; `llama-server` must be on your PATH:
+
+```bash
+mkdir -p qwen-v2-q4
+cd qwen-v2-q4
+repo="oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Llama"
+hf download "$repo" Qwen3.6-35B-v2-Q4_K_M.gguf mmproj-Qwen3.6-35B-v2-F16.gguf SHA256SUMS LICENSE --local-dir .
+sha256sum --check --ignore-missing SHA256SUMS
+llama-server -m Qwen3.6-35B-v2-Q4_K_M.gguf \
+  --mmproj mmproj-Qwen3.6-35B-v2-F16.gguf \
+  -ngl 99 -c 4096 -np 1 --host 127.0.0.1 --port 8080 \
+  --chat-template-kwargs '{"enable_thinking":false}'
+```
+
+Stop if checksum verification reports a mismatch. In another terminal on the same machine:
+
+```bash
+curl http://127.0.0.1:8080/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"messages":[{"role":"user","content":"Explain gravity briefly."}],"max_tokens":256,"temperature":0,"stream":false}'
+```
+
+The service is local-only. On a remote server, use SSH forwarding instead of exposing it publicly. Other GPU workloads reduce available memory; keep the initial context small. See the [llama.cpp server documentation](https://github.com/ggml-org/llama.cpp/tree/master/tools/server).
+
+</details>
+
+<details>
+<summary>💻 Laptop · 32 GB RAM / 8 GB VRAM — IQ4_XS with Ollama</summary>
+
+For a laptop such as an **RTX 4060 Laptop with 8 GB VRAM and 32 GB system RAM**. IQ4_XS is about **17.86 GiB**, plus the projector and runtime memory. It does **not** fit entirely in 8 GB VRAM: Ollama must split work between GPU and CPU/RAM. Expect slower responses than the RTX 5090. This is a starting configuration; the full laptop benchmark is still pending.
+
+Install [Ollama](https://ollama.com/download), Python and the [HF CLI](https://huggingface.co/docs/huggingface_hub/guides/cli). Keep Ollama running. In PowerShell:
+
+```powershell
+New-Item -ItemType Directory -Path .\qwen-v2-iq4 -Force | Out-Null
+Set-Location .\qwen-v2-iq4
+$qwenRepo = "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Ollama"
+hf download $qwenRepo Qwen3.6-35B-v2-IQ4_XS.gguf mmproj-Qwen3.6-35B-v2-F16.gguf import_ollama.py SHA256SUMS LICENSE --local-dir .
+python -m pip install requests
+python .\import_ollama.py --quant IQ4_XS
+ollama run qwen3.6-35b-v2:iq4_xs --think=false
+```
+
+The importer checks the selected model/projector hashes and includes both files. Its default context is 4,096 tokens with a 1,024-token output cap. For a lighter first request, use this PowerShell example instead of interactive chat:
+
+```powershell
+$qwenRequest = @{
+    model = "qwen3.6-35b-v2:iq4_xs"
+    messages = @(@{ role = "user"; content = "Explain gravity briefly." })
+    think = $false
+    stream = $false
+    options = @{ num_ctx = 2048; num_predict = 256 }
+} | ConvertTo-Json -Depth 5
+$qwenReply = Invoke-RestMethod -Uri "http://127.0.0.1:11434/api/chat" -Method Post -ContentType "application/json" -Body $qwenRequest -TimeoutSec 300
+$qwenReply.message.content
+```
+
+Close memory-heavy apps, run one model at a time and check GPU/CPU allocation with `ollama ps`. Importing into Ollama can keep a second copy of the weights: allow roughly 40 GB of free disk for this download plus its imported copy. These memory/disk figures are planning estimates, not measured laptop peaks. The importer was tested with Ollama 0.33.3; the full IQ4_XS laptop route remains unbenchmarked. [Ollama chat API](https://docs.ollama.com/api/chat).
+
+</details>
+
+<details>
+<summary>🧪 Smaller experimental option — IQ2_M with Ollama</summary>
+
+**About 11.70 GiB**, plus the **0.84-GiB projector** and runtime memory. Smaller than IQ4_XS, but substantial losses in accuracy, reasoning, code and formatting are possible. This is the released **IQ2_M**, not a separate Q2_K file. Its effective precision is mixed; some tensors remain at higher precision.
+
+Use it to explore lower-memory operation, not as the preferred option for important answers. It still does **not** fit completely in an 8-GB GPU, and lower file size does not guarantee higher speed. Start with text-only questions and a short context; quality and performance on the laptop have not been measured yet.
+
+With Ollama, Python and the HF CLI installed, run in PowerShell:
+
+```powershell
+New-Item -ItemType Directory -Path .\qwen-v2-iq2 -Force | Out-Null
+Set-Location .\qwen-v2-iq2
+$qwenRepo = "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Ollama"
+hf download $qwenRepo Qwen3.6-35B-v2-IQ2_M.gguf mmproj-Qwen3.6-35B-v2-F16.gguf import_ollama.py SHA256SUMS LICENSE --local-dir .
+python -m pip install requests
+python .\import_ollama.py --quant IQ2_M
+ollama run qwen3.6-35b-v2:iq2_m --think=false
+```
+
+For the lighter API example above, change only `model` to `qwen3.6-35b-v2:iq2_m`; keep `num_ctx = 2048` and `num_predict = 256`. Allow roughly 27 GB free disk for the download plus an imported copy. No Transformers/BF16 download is needed for this option.
+
+</details>

README.md

@@ -60,6 +60,8 @@ You set the tone: professional, casual, blunt or playful. The aim is personality
 | I use llama.cpp or a compatible GGUF app | [GGUF / Llama edition](https://huggingface.co/oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Llama); Q4_K_M is a starting point if it fits your memory |
 | I work with Python or want the full weights | [BF16 / FT edition](https://huggingface.co/oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT) |
 
+**Choose your hardware:** [H200 server · RTX 5090 · laptop · experimental IQ2_M](DEVICE-QUICKSTART.md).
+
 Smaller files need less memory, but the lowest-bit versions can lose substantial quality. Vision needs the included image processor/projector as well as the language-model weights.
 
 ### 💬 Try asking

SHA256SUMS

@@ -3,6 +3,7 @@
 0e712dd8e4bc65088e170826e98bdc440fb19a2cfdb7613c643e9dabb624ecd8  BENCHMARK-EXTENSION-QWEN38.json
 67d070e3b12bcd185626054c81cdf4d528a9a1af26774050e1d50be370b5ee14  BENCHMARK-PLAN.md
 5c1a6fa0bd2d21e400b4c84001abec8085e3f5d7fbe5d217e7a0af7f32f42c6d  DATASETS.md
+b483301314df6773850288b6291d49d082a7f816f7266d8cd62ef803b4f4e154  DEVICE-QUICKSTART.md
 5fa5bdc4687e5ed723af58eb436f545cc62589e5395d5041722639f03bb5ec9b  LAPTOP-BENCHMARK-MATRIX.json
 ad3521db2dbcdf0b2ff300349d312c0fb44a1ebccc84571712602c512bf4a733  Qwen3.6-35B-v2-IQ1_M.gguf
 61a1a351bdc64ad14616277b0f7c630abe297733a29d96659da3bb7b369c769d  Qwen3.6-35B-v2-IQ2_M.gguf
@@ -11,7 +12,7 @@ ad3521db2dbcdf0b2ff300349d312c0fb44a1ebccc84571712602c512bf4a733  Qwen3.6-35B-v2
 3cc8cd0bad0c37fba8e038f68c089cdf2e5f55619d48487fdd9bfff0686ef830  Qwen3.6-35B-v2-Q4_K_M.gguf
 a3a5b8afaf96d6d4c9dfb257e54406a04a8363791f980647f788c67ff3966615  Qwen3.6-35B-v2-Q5_K_M.gguf
 71099584272a77b22c4aa1152271bf7d85595e1b869ffc883440a9d0e225c073  Qwen3.6-35B-v2-Q8_0.gguf
-277c04f7ca2ff3d347a080e516dd6600f798048878003bf53fc7901d9d82b7b0  README.md
+81ee951e2874a3bd16458ce27fea3288edd0a34da193720c15f50c276ee88dc3  README.md
 a69838e3728d2aa08941191fce1d7c5af6f0703a9a4444e8d6f0ef18cb4ecf8a  RELEASE-NAMING.json
 d431e82d0ab961c261e525fc08fc3e64c0916fbd4d3f52b839b3d2eda0766338  assets/qwen3.6-35b-v2-banner.png
 fc5f718f531acdfe2e45bb09655d40f525eab939f9a3ab501287758f901cb3ad  mmproj-Qwen3.6-35B-v2-F16.gguf