Instructions to use jwest33/medgemma-4b-it-null-space-abliterated-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use jwest33/medgemma-4b-it-null-space-abliterated-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M

Use Docker

docker model run hf.co/jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use jwest33/medgemma-4b-it-null-space-abliterated-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jwest33/medgemma-4b-it-null-space-abliterated-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jwest33/medgemma-4b-it-null-space-abliterated-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M

Ollama
How to use jwest33/medgemma-4b-it-null-space-abliterated-GGUF with Ollama:
```
ollama run hf.co/jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use jwest33/medgemma-4b-it-null-space-abliterated-GGUF with Docker Model Runner:
```
docker model run hf.co/jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M
```

Lemonade

How to use jwest33/medgemma-4b-it-null-space-abliterated-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jwest33/medgemma-4b-it-null-space-abliterated-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.medgemma-4b-it-null-space-abliterated-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

MedGemma 4B Instruct - Null-Space Abliterated

google/medgemma-4b-it with refusal behavior removed via orthogonal projection. Uses null-space constraints and adaptive layer weighting to preserve model capabilities.

Note: This model will produce uncensored outputs. Use responsibly.

Abliteration Techniques Used

Winsorization: Clips outlier activations at the 99th percentile for cleaner refusal direction estimation (recommended for Gemma models)
Null-Space Projection: Preserves model capabilities by constraining weight updates to the null space of preservation activations
- Preservation Prompts: Dynamically generated using Gemma Scope 2 complete SAE circuit analysis to ensure complete coverage of shared features activated by harmful prompts, without overextending into unrelated capability space [gemma-3-4b-pt SAE models were used for circuit analysis of MedGemma3]
Adaptive Weighting: Applies Gaussian-weighted per-layer ablation strength, focusing on middle-to-later layers where refusal behavior concentrates
Norm Preservation: Maintains original Frobenius norms of weight matrices after projection

Parameter	Value
Harmful Prompts	5000
Harmless Prompts	637
Winsorization	99.5th percentile
Null-Space Constraints	rank ratio: 0.95
Directional Multiplier	1.10
SAE Targeted Coverage	1.00

Credits

Base Model: google/medgemma-4b-it by Google
SAE Analysis: GemmaScope sparse autoencoders by Google DeepMind
Norm-Preserving Biprojected Abliteration — Jim Lai (grimjim) (2025)
AlphaEdit: Null-Space Constrained Knowledge Editing — Fang et al. (ICLR 2025)
Refusal in Language Models Is Mediated by a Single Direction — Arditi et al. (2024)
Representation Engineering — Zou et al. (2023)

Toolkit Used

github.com/jwest33/abliterator

License

This model inherits the Gemma license from the base model. Please review and comply with Google's usage terms.

Disclaimer

This model is provided for research and educational purposes. The creators are not responsible for any misuse. Users are solely responsible for ensuring their use complies with applicable laws and ethical standards.

Downloads last month: 204

GGUF

Model size

4B params

Architecture

gemma3

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Model tree for jwest33/medgemma-4b-it-null-space-abliterated-GGUF

Base model

google/gemma-3-4b-pt

Finetuned

google/medgemma-4b-pt

Finetuned

google/medgemma-4b-it

Quantized

(50)

this model

Papers for jwest33/medgemma-4b-it-null-space-abliterated-GGUF