Not-For-All-Audiences

hermes-lineage

mtp

coding

tool-calling

reasoning

roleplay

bf16

Model card Files Files and versions

xet

Community

Instructions to use oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

Transformers

How to use oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT")
model = AutoModelForMultimodalLM.from_pretrained("oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT

SGLang

How to use oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT with Docker Model Runner:
```
docker model run hf.co/oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT
```

Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT / LAPTOP-CAPABILITY-PLAN.md

oktayd

Explain MoE and inherited stages; add per-test benchmark dropdowns and laptop capability roadmap

68be41c verified 20 days ago

preview code

Raw

History Blame Contribute Delete

5.33 kB

Laptop assistant: from quick tests to useful workflows

Status: local preparation in progress; no laptop winner selected yet. This is a deployment roadmap, not a claim that the full integration already works.

💻 Coding and agent skills — what we will actually test

Track	Cases	Success evidence	Current status
HumanEval+	5	Generated functions pass the selected task's base and extended tests in an isolated container	Existing outputs; execution/grading must be reconciled
MBPP+	5	Correct results on base and extended tests, including edge cases	Existing outputs; execution/grading must be reconciled
LiveCodeBench	9	Program passes task tests within sandbox limits	Existing outputs; execution/grading must be reconciled
Q36 bug fixes	5	Reproduce a failing test, apply a patch, pass regression tests	New tasks to prepare
Q36 small app / website tasks	5	Build succeeds; required UI behavior passes browser checks	New tasks to prepare
Tool choice and arguments	5	Correct tool and schema-valid arguments; no invented tool result	Local smoke fixtures prepared separately
Multi-step planning	5	Respect dependencies and reach a verifiable final state	New interactive fixtures to prepare
Recovery from tool errors	5	Respond to a simulated failure, choose a valid alternative, verify recovery	New interactive fixtures to prepare
Memory and conflicting evidence	5	Retrieve the correct note, cite it, distinguish updated from stale facts	New isolated test vault to prepare
Stopping and task completion	5	Stop after success; report missing prerequisites instead of inventing completion	New interactive fixtures to prepare
Transfer to changed constraints	5	Solve held-out variants after a changed rule or environment	New tasks to prepare
Q36 browser workflows	5	Complete a task on a local test website and verify the resulting page state	New interactive fixtures to prepare

64 case slots in this extension, including 19 existing coding slots. These are compact diagnostics, not full leaderboard scores. The agent tasks measure useful components of general problem solving, not proof of AGI. Existing Hermes-format and JSON-schema diagnostics remain in the broad suite and are not silently counted twice.

How a capability earns a card entry

Freeze tasks and evaluators before comparing runtime settings. Count one primary attempt per case. Report passes / executed-and-graded cases, task coverage, failures and ungraded cases together. Publish results even when they are weak; only describe a workflow as demonstrated when its actual end-to-end task passes. A successful JSON tool-call test alone does not establish autonomous coding or desktop control.

Generated code runs in a disposable container without network access, host credentials or user-folder mounts, with CPU, RAM, process and time limits. Host tools and paid services are not available to benchmark prompts.

⚡ Pick a laptop runtime

Smoke-test IQ4_XS, Q3_K_M, IQ2_M and IQ1_M; exclude the BF16 server package. Start with IQ2_M to check loading and the harness.
Compare Ollama and llama.cpp, each with a documented baseline and an optimized profile: 16 primary configurations. FreeToken is an additional experimental backend only if a compatible build actually runs.
Use the same prompts, image inputs, context, output cap and thinking setting. Tune only on a separate warm-up set, then freeze settings.
Record answer quality separately from prefill/decode tokens/s, end-to-end speed, time to first visible output, CPU/RAM/VRAM, temperatures, timeouts and incomplete outputs.
Select a measured quality/performance/memory compromise, not simply the fastest token rate. An OOM or unsupported backend is a technical failure, not a wrong answer.

The target laptop has 32 GB RAM and an RTX 4060 Laptop GPU with 8 GB VRAM. These 35B-weight models need CPU/RAM offload. MoE reduces active computation, not the need to store the expert weights.

🛠️ Hermes, routing and personal memory

After selecting the runtime, test it as a local endpoint behind the existing Hermes/router installation. Keep the previous route available for rollback. First use a dedicated test workspace and synthetic notes; connect the real Obsidian vault only after the test flow passes.

Useful scenarios to validate:

Code companion: inspect a sample repository, fix one bug, run tests and summarize the patch.
Project memory: find a note, cite its source and use its latest decision in a response.
Research notebook: turn supplied material into a draft note with provenance, then ask before writing into the real vault.
Task recovery: recognize an unavailable tool, ask for what is missing or use an explicitly allowed alternative.
Private local assistant: use the configured local route and verify that external fallback is disabled for private inputs.

Obsidian storage, retrieval/indexing, memory updates and routing belong to the host application. They are not built into the model weights. Exact endpoint instructions and working configuration examples will be added after local validation; no production routing has been changed by this document.