Instructions to use bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16

Use Docker

docker model run hf.co/bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16

LM Studio
Jan

vLLM

How to use bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16

Ollama
How to use bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF with Ollama:
```
ollama run hf.co/bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16
```
Unsloth Desktop

How to use bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF with Docker Model Runner:
```
docker model run hf.co/bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16
```

Lemonade

How to use bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16

Run and chat with the model

lemonade run user.Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF-BF16

List all available models

lemonade list

Hermes Agent

How to use bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF

GGUF conversion of huihui-ai/Huihui-GLM-4.7-Flash-abliterated at full BF16 precision.

The standard convert_hf_to_gguf.py produced broken output for glm4_moe_lite at the time of creation (January 2026), so this was produced via binary patching of verified working GGUF files.

Model Details

Property	Value
Architecture	GLM-4.7-Flash (30B-A3B MoE, DeepSeek2-like)
Active Parameters	~3B per token
Total Parameters	~30B
Experts	64 routed + 1 shared (4 active per token)
Precision	BF16 (full precision, no quantization)
Context Length	Up to 202K tokens (tested at 128K)
Files	2 split GGUF files (~56GB total)
Tensors	844 patched, 0 errors

How It Was Made

Instead of using the broken converter, this model was created by:

Starting with unsloth/GLM-4.7-Flash-BF16 split GGUF files (known working, correct structure)
Loading abliterated weights from huihui-ai's safetensors
Binary patching each tensor in-place, handling:
- MLA kv_b_proj split: Unified kv_b_proj (8960x512) reshaped and split into separate k_b (20x512x192, transposed) and v_b (20x256x512) tensors
- Expert stacking: 64 individual expert weights merged into fused 3D tensors per layer
- F32/BF16 dtype matching: Norm weights and biases kept as F32, main weights as BF16

This approach inherits all gating function fixes and correct GGUF structure from unsloth's conversion.

Note: Internal GGUF metadata (e.g., general.name, general.quantized_by) reflects the original unsloth source files. Only tensor data was replaced.

Verification

844/844 tensors patched with 0 errors
Byte-level verification: SHA256 hashes differ from base (weights changed), structure preserved (same shapes/offsets)
Coherence tests: Math, code generation, reasoning, knowledge, creative writing all pass
Long generation: 600+ tokens with no degradation
Multi-turn: Correct context handling across conversation turns
Abliteration confirmed: Base model refuses sensitive prompts; this model responds

Attribution

Base model: zai-org/GLM-4.7-Flash (MIT License)
Abliteration: huihui-ai/Huihui-GLM-4.7-Flash-abliterated (MIT License)
GGUF structure: unsloth/GLM-4.7-Flash-GGUF (BF16 split files)
GGUF conversion: Binary patching method (this repo)

Safety Warning

This is an abliterated (uncensored) model. Safety filtering has been significantly reduced. This model:

May generate sensitive, controversial, or inappropriate content
Is NOT suitable for public-facing or production applications
Is intended for research and experimental use only
Should be monitored during use

The creator bears no responsibility for any consequences arising from the use of this model. Users must ensure compliance with local laws and ethical standards.

License

MIT (inherited from base models)

Files

Huihui-GLM-4.7-Flash-abliterated-BF16-00001-of-00002.gguf  (49.9 GB)
Huihui-GLM-4.7-Flash-abliterated-BF16-00002-of-00002.gguf  (10.0 GB)

Downloads last month: 29

GGUF

Model size

30B params

Architecture

deepseek2

Hardware compatibility

16-bit

Model tree for bloopez/Huihui-GLM-4.7-Flash-abliterated-BF16-GGUF

Base model

zai-org/GLM-4.7-Flash

Finetuned

huihui-ai/Huihui-GLM-4.7-Flash-abliterated

Quantized

(11)

this model