Instructions to use llmfan46/gemma-4-31B-it-uncensored-heretic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use llmfan46/gemma-4-31B-it-uncensored-heretic with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="llmfan46/gemma-4-31B-it-uncensored-heretic")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("llmfan46/gemma-4-31B-it-uncensored-heretic")
model = AutoModelForMultimodalLM.from_pretrained("llmfan46/gemma-4-31B-it-uncensored-heretic", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use llmfan46/gemma-4-31B-it-uncensored-heretic with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "llmfan46/gemma-4-31B-it-uncensored-heretic"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llmfan46/gemma-4-31B-it-uncensored-heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/llmfan46/gemma-4-31B-it-uncensored-heretic

SGLang

How to use llmfan46/gemma-4-31B-it-uncensored-heretic with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "llmfan46/gemma-4-31B-it-uncensored-heretic" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llmfan46/gemma-4-31B-it-uncensored-heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "llmfan46/gemma-4-31B-it-uncensored-heretic" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llmfan46/gemma-4-31B-it-uncensored-heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use llmfan46/gemma-4-31B-it-uncensored-heretic with Docker Model Runner:
```
docker model run hf.co/llmfan46/gemma-4-31B-it-uncensored-heretic
```

Thought experiment... Heretic and Beyond...

by IrisColt - opened Jun 3

Discussion

IrisColt

Jun 3

Okay, I was not sure where to post this on Hugging Face, so I am leaving it under one of your flagship models, and one of my all-time favorites ever, Gemma 4 31B Uncensored Heretic.

This is just a thought experiment, since I do not have the resources to work with LLMs this large. Still, I wanted to describe something I would be curious to test, based on experiments I have done with T2I models like Illustrious.

As I understand it, this model is the result of applying your carefully refined Heretic process. In theory, you could get a delta, or diff, by comparing the original/base Gemma 4 safetensors file(s) with these Heretic version(s). Using that diff, transforming the original model into the Heretic model would be equivalent to applying that difference at face value (1.000 strength).

What my curiosity keeps pushing me to ask is: what would happen if we went a little further in that same direction? For example, what if we applied the diff at 1.00X, 1.0XX or 1.XXX strength instead? I do not know what a safe value for XX would be. With Illustrious, if I remember correctly, values around 1.035 were already close to the point where the images went from interesting to visual chaos.

So my main question is: if we carefully push the model further along the direction in parameter space that the Heretic version points toward, how does it behave? How far can we go before it breaks down? And what kind of personality would we be creating in the process?

End of soliloquy, heh.

llmfan46

Owner Jun 3

•

edited Jun 3

Increasing settings for more strength/aggressive in most cases means higher KL divergence and in most cases this translates to mean more capabilities lost in the process and less reasoning stability from the model which translates to overall worse quality, my methodology is to use KL divergence as a pointer, not as an end-all and be-all result that tells you everything select the ones with the best low KL divergence to low refusals ratio, then I pick up the finalists based and then benchmark these finalists from there.

Now you can definitely increase the strength, but what kind of results you will get is up to luck, but in most cases you will get either a model that is damaged and/or a model that behaves strangely with weird quirks, most people would prefer stability and predictability from a model, however what you could do is finetune the model with Unsloth Studio and that might or might heal the damage caused by the original high KL divergence, but at this point you will get a model that is probably quite far away from the original base/vanilla model and you can already achieve this without causing the model damage by increasing the strength in settings, but instead either by abliterating the model and finetuning it like I did with the Ortenzya model, or by doing merging and uncensoring like the Harmonia model.

IrisColt

Jun 4

Increasing settings for more strength/aggressive in most cases means higher KL divergence and in most cases this translates to mean more capabilities lost in the process...

Appreciate it! That connects a few dots for me. Sometimes I'm dumb, heh

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment