Instructions to use mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M

Use Docker

docker model run hf.co/mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M

LM Studio
Jan

vLLM

How to use mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M

Ollama
How to use mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2 with Ollama:
```
ollama run hf.co/mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M
```
Unsloth Desktop

How to use mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2 with Docker Model Runner:
```
docker model run hf.co/mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M
```

Lemonade

How to use mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M

Run and chat with the model

lemonade run user.asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2

GGUF quantizations of a fine-tuned model for translating Japanese ASMR transcriptions (ASR/Whisper output) into Simplified Chinese.

The model normalizes imperfect audio transcriptions, applies domain-specific glossaries, and translates character dialogue while retaining emotion and nuances.

This variant preserves the upstream Qwen3.5 MTP / speculative-decoding head in GGUF format so it can be used with MTP-capable llama.cpp builds.

Echo Mode

The model echoes the source Japanese text in an "input" field and records applied terms in a per-entry "glossary" object alongside the target translation. This provides an explicit source anchor that can reduce omitted or drifted segments, but it does not guarantee immunity to long-context repetition or noisy-ASR failures.

Available Quantizations

Quantization	Filename	Size	Description
q4_k_m	`asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2-q4_k_m.gguf`	5.4 GB	Good balance of quality and size
q6_k	`asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2-q6_k.gguf`	7.0 GB	Higher quality, moderate size
q8_0	`asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2-q8_0.gguf`	9.1 GB	Near-lossless quality
bf16	`asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2-bf16.gguf`	17.1 GB	Full BF16, no quantization loss

Prompt Example

将以下日语ASMR逐字稿翻译成简体中文。

音轨：track01_示例音轨
场景说明：主角与青梅竹马在校园下午的对话...

术语表（请严格使用zh栏位的译名）：
{
  "cvs": [],
  "characters": [],
  "terms": [{"ja": "放課後", "zh": "放学后"}]
}

翻译前请静默修正以下Whisper识别错误：
- 重复片语（连续3次以上且无变化）：仅保留一次
- 错字／同音异字：依上下文修正
- 字幕版权行（字幕：／翻訳：／QQ／LINE水印）：text设为null
- 错误专有名词：依术语表修正

翻译规则：
- 呻吟与气息声（あ、ん、はあ）→ 自然对应（啊、嗯、哈、呼）
- 拟声词：日语形式翻译（パンパン→啪啪）；中文形式保留原样
- 保留角色语气与口吻
- text字段只输出译文，不加注释或括号说明
- input字段为ids所对应的原始日文片段
- glossary字段只记录该输出项实际使用的日文→中文术语；没有时输出空对象

输入：逐字稿JSON数组 — {"id": <n>, "text": "<日文>", "start": <ms>, "end": <ms>}

输出：将连续构成同一句话的片段合并，JSON数组格式：
{"ids": [<n>, ...], "input": "<合并后的原始日文，以空格连接>", "glossary": {"<日文>": "<简体中文>"}, "text": "<简体中文>", "start": <最早ms>, "end": <最晚ms>}

字幕版权行：{"ids": [<n>], "input": "<原始日文>", "glossary": {}, "text": null, "start": <ms>, "end": <ms>}
每个输入id必须恰好出现在一个输出项中。
input字段为ids所对应的原始日文片段以空格连接，glossary记录该项实际使用的术语，text为其简体中文翻译。

逐字稿：
[
  {"id": 1, "text": "ねぇ、放課後、", "start": 3000, "end": 5000},
  {"id": 2, "text": "一緒に帰らない?", "start": 5000, "end": 7000}
]

Example Output:

[{"ids": [1, 2], "input": "ねぇ、放課後、 一緒に帰らない?", "glossary": {"放課後": "放学后"}, "text": "呐，放学后，要不要一起回去？", "start": 3000, "end": 7000}]

Usage

llama-server

llama-server -m asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2-q4_k_m.gguf -c 8192 --port 8080

llama-cli

llama-cli -m asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2-q4_k_m.gguf -c 8192 -p "<your prompt>" -n 2048

llama-cli with MTP

llama-cli -m asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2-q4_k_m.gguf -c 8192 --spec-type draft-mtp --spec-draft-n-max 6 -p "<your prompt>" -n 2048

Structured Decoding (Recommended)

This model outputs JSON arrays. Using structured decoding (e.g. GBNF grammar or JSON schema constraints) avoids wasted computation on malformed output and guarantees valid JSON on every generation.

JSON Schema:

{
  "type": "array",
  "items": {
    "type": "object",
    "properties": {
      "ids": {
        "type": "array",
        "items": {
          "type": "integer"
        },
        "minItems": 1
      },
      "input": {
        "type": "string"
      },
      "glossary": {
        "type": "object",
        "additionalProperties": {
          "type": "string"
        }
      },
      "text": {
        "anyOf": [
          {
            "type": "string"
          },
          {
            "type": "null"
          }
        ]
      },
      "start": {
        "type": "integer"
      },
      "end": {
        "type": "integer"
      }
    },
    "required": [
      "ids",
      "input",
      "glossary",
      "text",
      "start",
      "end"
    ],
    "additionalProperties": false
  },
  "minItems": 1
}

Supported by llama.cpp (--json-schema), vLLM, and outlines.

Training Details

Base model: unsloth/Qwen3.5-9B
Method: LoRA (r=16, alpha=16)
Target modules: o_proj, gate_proj, k_proj, down_proj, v_proj, up_proj, q_proj
Locale: zh-cn (Simplified Chinese)
Mode: Echo Mode
Max sequence length: 8192
Precision: bf16

Version 0.2 — Native 8k Continuation

This release continues the v0.1 Simplified Chinese echo adapter directly to a maximum training sequence length of 8,192 tokens. It does not stack adapters and does not use RoPE scaling or YaRN.

The deterministic continuation consumed 800 training draws: 520 coherent same-track, 120 far-context/glossary-recall, and 160 ordinary replay draws. The published adapter reached absolute step 200. Adapter weights SHA-256: 8000e9e54726e1943fd0c63c7f1249a351f462c2902274539e9245f483443740.

Pinned base revision: 005429cee5cb648998cf2b70eebdd83175989c9a.

Validation Scope

This variant received a small held-out semantic check: three ordinary rows and three long rows, each from a distinct locked-test family and with zero source overlap with the continuation ledger. All six grammar-constrained Q8 requests completed with exact ID coverage.

Manual reading classified two rows as pass, two as marginal, and two long rows as major failures. One long noisy-ASR row developed repeated speaker-prefix/template contamination. Another confused an earlier weapon scene with a later ear-cleaning scene.

The three exact original 4k constituent prompts from the repetition failure were then evaluated separately with the identical Q8, grammar, wrapper, seed, and decoding settings. The severe speaker-prefix collapse did not reproduce at 4k. This points to cross-window contamination when noisy/repetitive material is concatenated into a long prompt.

Context Budgeting

8192 is the total runtime context, not a safe input-only budget. Reserve room for generated JSON. Do not fill the entire context with the prompt and expect a non-trivial response.

Recommended Use

Treat long-context use of this variant as experimental. Prefer shorter windows around 4k for noisy ASR, and inspect or retry outputs when concatenating multiple scenes. Structured decoding guarantees the schema but cannot prevent semantically repeated or contaminated text.

Known Limitations

This variant did not receive the full 100-row 4k plus 100-row 8k validation used for zh-tw-echo.
Long concatenated noisy ASR can trigger speaker-prefix repetition and cross-scene contamination.
Shortening to original 4k windows removed the severe collapse, but did not make badly transcribed lines semantically reliable.
There is no complete controlled v0.1-versus-v0.2 final comparison.
Validation used Q8_0; other quantizations were not independently quality-scored.

Content Notice

The training domain includes adult ASMR dialogue and may produce sexually explicit text. This model is intended for transcription translation and subtitle-processing workflows.

MTP-preserving export

This revision preserves and verifies all 15 upstream MTP tensors against the pinned base checkpoint after the LoRA merge. The native-8192 adapter is unchanged; no retraining was performed. The MTP head itself is not fine-tuned. Earlier revisions of this v0.2 repository omitted MTP. Filenames and translation weights remain compatible; use a fresh repository revision to avoid cached non-MTP files. All four quantizations have verified MTP metadata and tensor inventory. Q4_K_M was GPU-smoke-tested with llama.cpp b9247, context 8192, and --spec-type draft-mtp --spec-draft-n-max 3. This is a runtime smoke test, not a new quality evaluation or a speedup guarantee. See mtp-preservation.json and mtp-release-verification.json for provenance and checks.

Downloads last month: 385

GGUF

Model size

9B params

Architecture

qwen35

Hardware compatibility

4-bit

6-bit

8-bit

16-bit

Model tree for mmis1000/asmr-qwen3.5-9b-zh-cn-echo-gguf-v0.2

Base model

Qwen/Qwen3.5-9B-Base

Finetuned

Qwen/Qwen3.5-9B

Finetuned

unsloth/Qwen3.5-9B

Quantized

(32)

this model