Instructions to use YTan2000/Muse-Glimmer-30B-TQ3_4S with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use YTan2000/Muse-Glimmer-30B-TQ3_4S with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0
# Run inference directly in the terminal:
llama cli -hf YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0
# Run inference directly in the terminal:
llama cli -hf YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0

Use Docker

docker model run hf.co/YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0

LM Studio
Jan

vLLM

How to use YTan2000/Muse-Glimmer-30B-TQ3_4S with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "YTan2000/Muse-Glimmer-30B-TQ3_4S"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "YTan2000/Muse-Glimmer-30B-TQ3_4S",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0

Ollama
How to use YTan2000/Muse-Glimmer-30B-TQ3_4S with Ollama:
```
ollama run hf.co/YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0
```
Unsloth Desktop

How to use YTan2000/Muse-Glimmer-30B-TQ3_4S with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use YTan2000/Muse-Glimmer-30B-TQ3_4S with Docker Model Runner:
```
docker model run hf.co/YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0
```

Lemonade

How to use YTan2000/Muse-Glimmer-30B-TQ3_4S with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0

Run and chat with the model

lemonade run user.Muse-Glimmer-30B-TQ3_4S-Q8_0

List all available models

lemonade list

Hermes Agent

How to use YTan2000/Muse-Glimmer-30B-TQ3_4S with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use YTan2000/Muse-Glimmer-30B-TQ3_4S with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "YTan2000/Muse-Glimmer-30B-TQ3_4S:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

YTan2000 commited on 29 days ago

Commit

f6aec5c

verified ·

1 Parent(s): baf0c4b

Upload folder using huggingface_hub

Browse files

Files changed (6) hide show

.gitattributes +4 -0
Muse-Glimmer-30B-TQ3_4S.gguf +3 -0
README.md +124 -0
benchmark.png +3 -0
mmproj-Muse-Glimmer-30B-Q8_0.gguf +3 -0
thumbnail.png +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Muse-Glimmer-30B-TQ3_4S.gguf filter=lfs diff=lfs merge=lfs -text
+benchmark.png filter=lfs diff=lfs merge=lfs -text
+mmproj-Muse-Glimmer-30B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+thumbnail.png filter=lfs diff=lfs merge=lfs -text

Muse-Glimmer-30B-TQ3_4S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cd6d4fafccaf7ccdf219db21adbdf8a44b6af9d06436859aade12705404a3d68
+size 14806958880

README.md ADDED Viewed

	@@ -0,0 +1,124 @@

+---
+license: apache-2.0
+language:
+- en
+library_name: gguf
+pipeline_tag: text-generation
+tags:
+- gguf
+- llama.cpp
+- muse-glimmer
+- turboquant
+- tq3_4s
+- vision
+- image-text-to-text
+base_model:
+- unsloth/Muse-Glimmer-30B-GGUF
+---
+# Muse-Glimmer-30B-TQ3_4S
+![Muse-Glimmer-30B-TQ3_4S](thumbnail.png)
+## Required Runtime
+This model uses the custom `TQ3_4S` tensor type. It requires
+[turbo-tan/llama.cpp-tq3](https://github.com/turbo-tan/llama.cpp-tq3).
+Stock `llama.cpp` builds without TurboQuant support cannot load it.
+## Model Files
+| File | Size | Purpose |
+|---|---|---|
+| `Muse-Glimmer-30B-TQ3_4S.gguf` | 13.78 GiB | Main model (4.25 bpw) |
+| `mmproj-Muse-Glimmer-30B-Q8_0.gguf` | 2.0 GiB | Vision projection (image input) |
+## Base Model
+- Upstream parent: `unsloth/Muse-Glimmer-30B-GGUF` (from `meta-models/Muse-Glimmer-30B`, Apache-2.0)
+- Quantization: TurboQuant `TQ3_4S` (four-scale turbo quant) with out6k recipe —
+  output and embedding tensors preserved at q6_K precision
+- Native context: 131,072 tokens
+## Recommended Runtime
+Text-only:
+```bash
+./build/bin/llama-server \
+  -m Muse-Glimmer-30B-TQ3_4S.gguf \
+  --host 127.0.0.1 --port 8080 \
+  -c 32768 -np 1 -ngl 99 -fa on \
+  --reasoning-format deepseek --jinja
+```
+With vision (mmproj):
+```bash
+./build/bin/llama-server \
+  -m Muse-Glimmer-30B-TQ3_4S.gguf \
+  --mmproj mmproj-Muse-Glimmer-30B-Q8_0.gguf \
+  --host 127.0.0.1 --port 8080 \
+  -c 32768 -np 1 -ngl 99 -fa on \
+  --reasoning-format deepseek --jinja
+```
+Optional — DFlash speculative decoding (raises decode ~20%):
+```bash
+# separate drafter model required
+--spec-type draft-dflash -md <drafter>.gguf --spec-draft-n-max 3
+```
+## Benchmarks
+Measured on **NVIDIA RTX 3090 24 GB**, `turbo-tan/llama.cpp-tq3` build `f755f1ac1`,
+thinking ON, temperature 0.
+![Benchmark summary](benchmark.png)
+### Evalplus (official scorer)
+| Benchmark | pass@1 |
+|---|---:|
+| HumanEval | **93.3** |
+| HumanEval+ | **89.0** |
+| MBPP | **89.7** |
+| MBPP+ | **74.6** |
+### Hard86
+| Benchmark | Result |
+|---|---|
+| Hard86 | **74/86** (86.0%) |
+### Task suites (task breakdown)
+| Suite | Score | Pass rate |
+|---|---:|---|
+| instructfollow | 96.7 | 14/15 |
+| coding | 87.5 | 10/12 |
+| dataextract | 82.8 | 9/15 |
+| reasonmath | 80.0 | 12/15 |
+| toolcall | 80.0 | 11/15 |
+| speed | 70.8 | 9/9 |
+### Speed
+| Config | Result |
+|---|---:|
+| llama-bench pp2048 | 1,155 tok/s |
+| llama-bench tg128 | 43.3 tok/s |
+| Decode, 8K context, no drafter | 44.6 tok/s |
+| Decode, 8K context, DFlash drafter (n_max=3) | 53.7 tok/s (+20%) |
+Generation throughput during task suites: 47.9 tok/s.
+## Validation
+- Strict server smoke (`--reasoning off`): content exactly `ok` ✅
+- Drafter signature verified in server logs: `block_size=16, mask_token_id=201818, n_extract=5`
+## License
+Apache-2.0. Use is also subject to the base model license and the license terms of the runtime.

benchmark.png ADDED Viewed

Git LFS Details

SHA256: b97a599f407c3b09ef21f0a20755ca1a36ec35102a46a6e464b98196f0829768
Pointer size: 131 Bytes
Size of remote file: 131 kB

mmproj-Muse-Glimmer-30B-Q8_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:01ff73c95108e1754a4c145176c6d3ba44338942285cb87dcac7f4f193192ea2
+size 2051685088

thumbnail.png ADDED Viewed

Git LFS Details

SHA256: d14cbe60493c8911ccc8b585d7ef493dad14c0f7b383f71dc5ec09ca897b1f7d
Pointer size: 131 Bytes
Size of remote file: 406 kB