Instructions to use chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M

Use Docker

docker model run hf.co/chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M

Ollama
How to use chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF with Ollama:
```
ollama run hf.co/chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M
```
Unsloth Desktop

How to use chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF with Docker Model Runner:
```
docker model run hf.co/chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M
```

Lemonade

How to use chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "chatqaq/Qwen3.6-27B-Claude-Mythos-Distilled-MTP-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

lrsoft commited on Jun 22

Commit

a6de945

verified ·

1 Parent(s): 3654f62

Update README.md

Browse files

Files changed (1) hide show

README.md +76 -18

README.md CHANGED Viewed

@@ -8,34 +8,92 @@ base_model:
 # Qwen3.6-27B-Claude-Mythos-Distilled
 QLoRA fine-tuned version of Qwen3.6-27B trained on a 25K synthetic instruction dataset focused on advanced reasoning, coding, cybersecurity analysis, and agentic workflows.
-## Base Model
 Qwen3.6-27B
-## Method
 QLoRA (4-bit NF4) fine-tuning
-## Dataset
 25K synthetic SFT dataset (Claude Mythos-style reasoning distribution, fully synthetic)
 https://huggingface.co/datasets/WithinUsAI/claude_mythos_distilled_25k
-## System Prompt
-"You are a distilled mirror of Claude Mythos..."
-## Export
-GGUF Q8/Q4 quantized
-## Intended Use
-- Coding assistance
-- Technical reasoning
-- Agentic workflows
-- Research / analysis tasks
-## Limitations
-- Synthetic training data (no real frontier model outputs)
-- May hallucinate or overconfidently reason
-- Not safety-aligned for production-critical use
-## License
-Follows Qwen3.6-27B base model license

 # Qwen3.6-27B-Claude-Mythos-Distilled
+🚀 Try our ecosystem:
+- Free AI Chat (no login): https://freeaichat.chatqaq.com/
+- AI Atlas (AI news & insights): https://ai-atlas-a.chatqaq.com/zh/
+---
 QLoRA fine-tuned version of Qwen3.6-27B trained on a 25K synthetic instruction dataset focused on advanced reasoning, coding, cybersecurity analysis, and agentic workflows.
+基于 Qwen3.6-27B 的 QLoRA 微调模型，使用 25K 高质量合成指令数据训练，重点增强复杂推理、代码能力、网络安全分析与 Agent 工作流能力。
+---
+## Base Model / 基座模型
 Qwen3.6-27B
+---
+## Method / 训练方法
 QLoRA (4-bit NF4) fine-tuning
+QLoRA（4-bit NF4）参数高效微调
+---
+## Dataset / 训练数据
 25K synthetic SFT dataset (Claude Mythos-style reasoning distribution, fully synthetic)
+25,000 条合成 SFT 数据（Claude Mythos 风格推理分布，完全合成生成）
 https://huggingface.co/datasets/WithinUsAI/claude_mythos_distilled_25k
+---
+## System Prompt / 系统提示词
+Light system prompt used during training:
+训练时使用的轻量系统提示词：
+> You are a highly capable assistant optimized for technical reasoning, coding, and multi-step problem solving.
+> You provide structured, precise, and actionable responses.
+---
+## Export / 导出格式
+GGUF Q8 / Q4 quantized
+GGUF Q8 / Q4 量化版本
+---
+## Intended Use / 适用场景
+- Coding assistance / 编程辅助
+- Technical reasoning / 技术推理
+- Agentic workflows / Agent 工作流
+- Research & analysis tasks / 研究与分析任务
+- System design support / 系统设计辅助
+---
+## Key Characteristics / 主要特点
+- Strong structured reasoning behavior / 强结构化推理能力
+- Code-oriented response style / 偏工程化代码输出风格
+- Multi-step problem decomposition / 多步问题拆解能力
+- Synthetic high-signal instruction tuning / 高信号合成数据训练
+---
+## Limitations / 局限性
+- Fully synthetic training data (no real frontier model outputs)
+  完全合成数据训练（非真实前沿模型输出）
+- May hallucinate or over-generalize in complex domains
+  在复杂任务中可能产生幻觉或过度泛化
+- Not safety-certified for production systems
+  未经过生产级安全验证
+- Not a replacement for real-world expert validation
+  不能替代真实领域专家审查
+---
+## License / 许可
+Follows Qwen3.6-27B base model license
+遵循 Qwen3.6-27B 基座模型许可协议